Friday, September 28, 2007
One of the summer’s highlights was a talk at AEI by Chicago labor economist Derek Neal. (Footnote: AEI talks generally make me want to impale myself on a Powerpoint projector, but this one was exceptional.) For those who weren’t there, you can watch the video here.
Neal found that low-performing kids in Chicago got shafted when the Chicago accountability system went into place, and again after NCLB was implemented. His talk wasn’t about teacher labor markets, but he made a critical point in this area. If the measurement of teacher effectiveness doesn’t take into account that some kids start off further behind that others and I am labeled a bad teacher as a result, why would I teach in a low-performing school? We have a hard enough time staffing these schools to begin with, in part because of salary differentials but also because of working conditions. If these teachers feel disrespected as professionals because the measurement system doesn’t acknowledge that they have a tougher job, I predict that we’re going to have a harder time recruiting and retaining teachers in these schools. This is conjecture, I know – we really have very little evidence about such a system because no one has implemented a comprehensive teacher effectiveness plan yet. (If you know of any studies on this issue, please email them to me.)
The best part, I thought, was towards the end of the discussion, where Doug Mesecar (Asst. Secretary at Ed) and Neal go back and forth in response to Mesecar’s question, “Are you saying our teachers are not professionals?”, i.e. that they're not good enough to get everyone to proficiency no matter how far behind they start. Most folks back down when challenged with the “soft bigotry of low expectations” rhetoric, but Neal was having none of it.
That’s it for teacher effectiveness, folks – I hope that this week has made you think through some of the issues we don’t hear much about in this debate.
Stay tuned – next week I’ll focus on NCLB.
Thursday, September 27, 2007
W, speaking in NYC, related that NYC's gains are, in part, a function of NCLB. He forgot to mention that NCLB also led to the development of the iPhone - because if two things happen at the same time, one must have caused the other. Duh!
I'm convinced that NCLB is responsible for these phenomena, too, since they all happened after 2001:
- Hannah Montana
- A-Rod’s 53 home runs (Joe Torre closed the achievement gap)
- Feist’s numerical prowess (Feist sings the song from the iPod commercial: 1-2-3-4-5-6-9-10. And NAEP math scores are up?)
- The 2006 outbreak of e.coli in spinach (Ed Trust accuses Fairtest of poisoning crops to lower scores)
- The Tour de France doping scandal (as Sherman Dorn wrote, Vi8gra for ur tests)
- The rise of crystal meth (teachers, I know it's tough, but...)
Everything in the world is measured with error. This isn't the sexiest topic - so I'll try to keep it short - but it's important. I want to draw your attention to two different problems:
Repeated measurements of anything necessarily have some random variation. Step onto the scale daily for a month (assuming you're not preparing to hibernate for the winter, in which case your weight changes are not "random variation") and this will quickly become apparent. In terms of measuring teacher effects, this random variation is less of a problem if we are willing to assess teacher effectiveness across multiple years of data (i.e. my students' test scores in 2005, 2006, and 2007).
But I've heard a lot of plans kicked around that propose to reward some proportion of teachers based on one year of data; certainly, from an incentives perspective, this makes sense. We want to give you an incentive this year to push hard. But when Mrs. Scott is awarded merit pay in 2005, but not in 2006, and then she's a merit pay-worthy teacher again in 2007, this system doesn't have a lot of face validity for educators or the public.
2) A student's baseline is not necessarily a good control:
To be sure, value-added models are a tremendous improvement upon NCLB's proficiency system. (Note: value-added models with unrealistic proficiency targets aren't really an improvement - more on this next week.) Value-added approaches give us a more accurate portrayal of how a school or teacher is really doing. Here's a description of value-added Tennessee style from the Center for Greater Philadelphia at Penn:
Because individual students rather than cohorts are traced over time, each student serves as his or her own "baseline" or control, which removes virtually all of the influence of the unvarying characteristics of the student, such as race or socioeconomic factors.
Sounds about right. Right? The clearest example of why a student's baseline test score does not "remove virtually all the influence of unvarying characteristics of the student" is the following: Mrs. Jones' class enrolls only wealthy children, while Mrs. Scott's class enrolls only kids who qualify for free lunch. Tests are given in the spring of each school year, so we're going to measure May to May. We know that poor students have lower rates of learning in the summer compared to their more advantaged peers. But if we only take into account their initial score, not their socioeconomic status, we're going to come up with a biased estimate of teacher effectiveness. In this case, teachers who teach low-income kids look like poorer teachers simply because of summer learning loss.
Check out the powerpoint at the Center for Greater Philadelphia - link above. It's a great overview of value-added.
Wednesday, September 26, 2007
Number of New York City's most influential role models (i.e. teachers) that will never wear a
Yankee uniform: ~74,000
Number of Boston's most influential role models that will never wear a
Yankee uniform: ~5,500
Number of children's books authored by Yankees players: 2
Number of children's books authored by Red Sox players: 0 (can they read?)
Starting salary of new New York City teachers: $43,362
Starting salary of new Boston teachers: $42,355
2007 salary of Alex Rodriguez: $22,708,525
2007 salary of David "Big Papi" Ortiz: $13,250,000
Dollars per years of education, NYC teachers (assuming 16 yrs): $2,710.13
Dollars per years of education, Boston teachers: $2,647.19
Dollars per years of education, Alex Rodriguez (12 yrs): $1,892, 377.08
Dollars per years of education, David Ortiz (11 yrs): $1,204,545.45
Value of renewed Snapple contract with NYC schools, 2006: $33 million
Number of flavors of Snapple iced tea (note: iced tea not sold in schools): 23
Dollars invested in Boston's "war on obesity" (announced Dec 2006): $279,000
Percent of Boston 3 year olds that are obese: 14%
Percent of NYC K-5 children that are obese: 24%
Percent of NYC African-American children that are obese: 23%
Percent of NYC Hispanic children that are obese: 31%
Percent of NYC Asian-American children that are obese: 14.4%
Percent of NYC white children that are obese: 16%
Grams of sugar in Snapple juice drinks: 41 grams
Grams of sugar in 12 ounce Coke: 39 grams
Weight of Alex Rodriguez in pounds: 210
Weight of David Ortiz in pounds: 230
Dollars earned per pound, Alex Rodriguez: $108,135.83
Dollars earned per pound, David Ortiz: $57,608.70
If you are interested in the relationship between teacher salaries and teacher quality, check out this Economic Policy Institute report. Or click here for A-Rod or Jorge Posada's children's books.
Wiki-alert: If I have any of the factoids above wrong, please email me at eduwonkette (at) gmail (dot) com and I will correct them.
IMAGINE, FOR A MOMENT, that a program designed to aid disadvantaged students might, instead, be seriously undermining their performance. Imagine that the schools administering the programs were told that the programs might be having this boomerang effect -- but that no one investigated further because the programs were so popular and the prospect of change was so politically controversial.
They go on to make the now familiar claim that black law students would be better off had they not attended selective institutions. Does research have anything to say about this issue?
Funny that they forgot to mention it, but rising star and Princeton economist Jesse Rothstein and his colleague Albert Yoon have actually spent a lot of time rigorously looking at this question. (There's something in the eduwater where Jesse grew up, as his dad is Richard Rothstein.) And here’s what they find:
We find that the data are inconsistent with large mismatch effects, particularly with respect to employment outcomes. While moderate mismatch effects are possible, they are concentrated among the students with the weakest entering academic credentials.
By weakest academic credentials, Rothstein means students in the bottom 20% of the LSAT score and GPA distribution. And what would happen to black student representation if we took the advice of Amar and Sander? Rothstein and Yoon simulated such a parallel world, and found the following:
In the absence of affirmative action, we estimate that the number of black students entering law school would fall by about 60 percent, while black representation at the most selective schools would fall by 90 percent. The decline in black representation would extend well beyond the most selective schools; all but the least selective schools would enroll one-third to one-half fewer black students than they do today. This contradicts claims about the centrality of the so-called “cascade effect,” whereby affirmative action affects diversity only at the elite schools.
Rothstein and Yoon conclude:
Many potentially successful black law students would be excluded, far more than the number who would be induced to pass the bar exam by the elimination of mismatch effects. Accordingly, we find that eliminating affirmative action would dramatically reduce the production of black lawyers.
So to Amar and Sander - it's not exactly true that no one has been able to study this because the data are on lock down. That said, I don't see why they shouldn't have access to the data - but please cc Jesse Rothstein so he can check your numbers. His papers on this issue are available here; one is forthcoming in the University of Chicago Law Review.
Bonus prize!!!: for anyone who teaches undergrad methods or statistics, this is a great exam question, i.e.:
In this opinion piece, the authors argue that affirmative action in law school hurts its intended beneficiaries, black students. Use what you've learned this semester to identify some potential problems with their causal claim.
Data from one selective California law school from 2005 show that students who received large preferences were 10 times as likely to fail the California bar as students who received no preference. After the passage of Proposition 209, which limited the use of racial preferences at California's public universities, in-state bar passage rates for blacks and Latinos went up relative to out-of-state bar passage rates. To the extent that students of color moved from UC schools to less elite ones (as seems likely), the post-209 experience is consistent with the mismatch theory.
In general, research shows that 50% of black law students end up in the bottom 10th of their class, and that they are more than twice as likely to drop out as white students. Only one in three black students who start law school graduate and pass the bar on their first attempt; most never become lawyers. How much of this might be attributable to the mismatch effect of affirmative action is still a matter of debate, but the problem cries out for attention.
Measuring the effects of teachers on students is challenging because students are not randomly assigned to teachers. In everyday conversation and journalistic accounts, the distinction between treatment and selection effects is rarely acknowledged. Ironically, teachers know this statistical trade secret well, but get lambasted when they point out that all teachers are not working with the same inputs.
Imagine a study evaluating the effectiveness of an exercise program. The first trial randomly assigns study participants to two conditions: running five miles a day or watching television. If, two months later, exercisers are in better shape than the TV watchers, we can conclude that the program is responsible. The average difference in fitness between these two groups is called a treatment effect.
Now, imagine a second trial, where participants themselves decide whether they want to exercise or watch TV (baseball post-season, anyone?). People know how athletic they are. They have a pretty good idea of whether they would like to, or even can, run five miles a day. Suppose that the track team signs up for the exercise program, while those who have a special distaste for physical activity (or were cut from the track team) opt for television. Two months later, we find that those participating in the exercise condition have lower blood pressure and less body fat.
What can we make of the effects of the program? Not much. Common sense dictates that members of the track team likely had lower blood pressure and less body fat to begin with. In this scenario, we can't just compare average differences in fitness between exercisers and TV watchers because of selection bias.
If students were randomly assigned to teachers, differences in teacher performance would represent a treatment, or a “teacher effect.” These differences could thus be attributed to something the teacher did. As the example above illustrates, accurately measuring the effects of teachers would require this type of assignment. Of course, students are not randomly assigned to teachers. Parents do not mindlessly flip a coin and leave their child’s placement with a bad teacher up to chance; we know that principals and guidance counselors often heed parents’ wishes in the teacher placement process. Parents aside, we also know that principals non-randomly assign kids to teachers based on their sense of which teachers are good with certain kinds of kids. (In the worst examples of this I’ve seen, new teachers are given the biggest behavior problems to “make them or break them.”)
If performance is measured by absolute criteria, and students with poorer performance are not distributed equally across teachers, some teachers will spuriously appear to be doing much better than others. Because of the way we are currently measuring teacher effectiveness (even with growth models that account for students’ initial score - we rarely hear of models that will take into account demographic and social characteristics that also affect student achievement), a high-performing teacher is one that enrolls high-performing students in the first place.
Any legitimate model of measuring teacher effectiveness needs to take non-random assignment seriously. Researchers who study teacher effects have developed a formidable repertoire of tools to deal with this problem, but they all involve 1) controlling for characteristics of kids that are politically unpopular to control for, like race and poverty, and 2) using multiple observations of the same teacher (i.e the last three years of this teacher's performance) to establish a “permanent” teacher effect. To date, the former has been politically unpopular, while the latter would be unacceptable to those who want to measure a teacher’s performance one year at a time.
Because of these political tensions, my fear is that we end up with a model of teacher effectiveness that a) isn’t valid and b) provides disincentives for teaching the lowest performing kids. More on both of these issues in the next two days.
Charlie Clotfelter is a professor of public policy and economics at Duke. He has studied more things than I can mention here, but two of his most important strands of work focus on teacher quality and school desegregation. Since this is teacher quality week, I'll focus my attention there, but see his book After Brown as well.
Findings on teacher quality:
- Race and experienced teachers: Most of the research on who is taught by inexperienced teachers focuses on how new teachers are more likely to teach in schools with more black kids. Clotfelter and colleagues Sunny Ladd and Jake Vigdor show that even within the same school, black kids are more likely to be taught by inexperienced teachers than white kids. (See publication #3 at the link for more info.)
- Teacher credentials and student achievement: Clotfelter et al., using better data than previously available (a 10 year time frame and data for all teachers in North Carolina), find that a teacher's experience, test scores and regular licensure all have positive effects on student achievement, with larger effects for math than for reading. Of particular interest is their finding that the benefit of experience peaks at 21-27 years of teaching, though like much other research, they find that much of the returns to experience occur in the first few years of teaching.
Tuesday, September 25, 2007
1) "The Sign" - Ace of Base (for NAEP part 1)
2) "Big Girls Don't Cry" - Fergie (for NAEP part 2)
3) "You're So Vain" - Carly Simon (for George Miller)
4) "Lost in Your Eyes" - Debbie Gibson (for Buck McKeon)
5) "One Shot" - Eminem (NCLB anthem)
6) "Not Ready to Make Nice" - The Dixie Chicks (for irony)
7) "Opposites Attract" - Paula Abdul (for Reg Weaver)
8) "Drunk and Hot Girls" - Kanye West (shout out to Eli and Bill)
9) "Friends in Low Places" - Garth Brooks (for POTUS)
10) "The Right Stuff" - New Kids on the Block (positive psychology part 1)
11) "I'm Too Sexy" - Right Said Fred (positive psychology part 2)
12) "Faith" - George Michael (positive psychology part 3)
Today I want to focus on how the debate has overemphasized the contributions of individual teachers without thinking about the ways that schools function like organisms. No teacher is an island. In multiple ways, schools and colleagues enable and constrain the work that a teacher does in his or her own classroom. There are three key points to this argument:
1) Teachers are interdependent
This means two things. First, teachers provide instructional support to their peers. When Mrs. Jones heads down to Mrs. Scott's room to ask for help, trade ideas, talk over what to do about a difficult student, or plan a lesson together, we see evidence of the interdependence that makes schools a unique enterprise. To reduce teacher effectiveness to a measure of what one teacher does in her own classroom ignores that teaching is, in the best circumstances, a collective accomplishment. Identifying these "peer effects" formally is difficult for a host of boring methodological reasons, but nonetheless something that educational research needs to do.
The second issue applies specifically to middle and high schools, where students are taught by multiple teachers. When Mrs. Jones, the reading teacher, improves Mikey's reading skills so that Mrs. Scott can teach him how to do word problems, we see that no teacher is an island. When Mrs. Scott gets Mikey's attention problems under control so Mrs. Jones can get Mikey to sit still and read, we see another example of interdependence.
We don't think about interdependence because we are deluged with images of individual teachers administering vigilante justice in unruly schools (think "Dangerous Minds"). Those who haven't spent much time in urban schools assume that teachers are isolates. But I would submit that even in the toughest schools, there are teachers working together in ways that violate the independence assumption of the teacher quality debate.
2) Teachers play many roles in a school
There is an oft-cited maxim that returns to experience plateau after five years (which is true), with the implicit assumption that more experienced teachers aren't adding more value (which is not). Experienced teachers often serve as anchoring forces in addition to teaching students in their own classrooms. Some of these functions sit under the collaboration and mentoring points discussed above, but others involve schoolwide leadership.
If we opt to hire and reward teachers based only on their own students' performance, it follows that we would save a lot of money by shedding more experienced (and thus more expensive) teachers. Indeed, this is a fear of many teachers in New York City, which will begin allocating dollars, rather than teaching positions, to schools.
Thinking about returns to experience this way would be a mistake even for those concerned only with student achievement, as pulling out the core of our schools - experienced educators - may destabilize schools altogether.
3) Schools, as organizations, create the conditions for the work individual teachers do in their classrooms: Let's imagine that we want to compare the teachers in two different schools for the purposes of merit pay or job evaluations. In School A, resources are flush, class size is small, and the hallways are quiet. In School B, there are no books, class size is 50% larger, and the school is chaotic.
If we are going to make across school comparisons of teachers, we need to take into account that individual schools can make teachers more or less effective. What we really need to know is how Mrs. Jones at School A would have performed if she was a teacher at School B. Economists call this "firm-specific capital"; sociologists call this a "school effect." Irrespective of what we call it, we need to take school conditions into account when we think about comparing teachers across schools.
Please send Bill Gates an iPod nano. If you do, I'll gift him Kanye West's albums, beginning with "College Dropout."
Looks like no one at Ed in '08 is a hip-hop fan - or perhaps they are and like song 8 on Kanye's new "Graduation" album, called "Drunk and Hot Girls." Here's the flavor:
We go through too much bullshit just to mess with these drunk and hot girls
We go through too much bullshit just to mess with these drunk and hot girls
Driving around town looking for the best spot for the
(Drunk and hot girls)
Up in the club look at here what we got some
(Drunk and hot girls)
Stop dancing with your girlfriend and come dance with me
Stop talking about your boyfriend since he is not me
Stop running up my tab cause these drinks is not free
You drunk and hot girl
Or maybe it was "School Spirit Skit 2" that turned homies Bill and Eli on to Kanye:
You keep it going man, you keep those books rolling,
You pick up those books you're going to read
And not remember and you roll man.
You get that a sociate degree, okay,
Then you get your bachelors,
then you get your masters
Then you get your master's masters,
Then you get your doctron,
You go man, then when everybody says quit
You show them those degrees man,
when Everybody says hey,
you're not working,
You're not making in money,
You say look at my degrees and you look at my life,
Yeah I'm 52,
so what, hate all you want,
But I'm smart, I'm so smart, and I'm in school,
And these guys are out here making Money all these ways,
and I'm spended mine to be smart.
You know why?
Because when I die, buddy, you know
What going to keep me warm, that right, those degrees.
Now, don't count Kanye out yet, as he has an answer to all this in his song "Champion" on his most recent album, "Graduation:"
Last week I paid a visit to the institute
They got the crop out keepin' kids in the school
I guess I'll clean up my act like Prince'll do
If not for the pleasure, least for the principle
Note to Bill and Eli - use those Benjamins to hire McKinsey to track down more appropriate tunes to reach out to the African-American community.
For the record, my vote goes to Mary J. Blige.
More commentary on the choice of Kanye for Ed in '08's ad campaign over at Matthew Tabor and District Administration.
There have always been multiple justifications for desegregation - among the most cited are 1) separate schools will always have resource inequalities, and 2) social interaction in the early years can spur social integration later on.
What were the effects of desegregation on its intended beneficiaries - black students - and if these effects were positive, what mechanisms explain these effects? Sarah Reber, a UCLA economist, wanted to know, too. In this important paper, she finds the following:
In Louisiana, substantial reductions in segregation between 1965 and 1970 were accompanied by large increases in per-pupil funding. This additional funding was used to "level up" school spending in integrated schools to the level previously experienced only in the white schools...the increase in funding associated with desegregation was more important than the increased exposure to whites. A simple cost-benefit calculation suggests that the additional school spending was more than offset by higher earnings due to increased educational attainment...the results of this paper are consistent with earlier work suggesting that desegregation improved educational attainment for blacks and sheds new light on the potential mechanism behind this improvement in Louisiana: increased funding for blacks' schools.
While Reber only addresses one of the goals of deseg - academic outcomes - this study by Amy Stuart Wells has also looked at non-cognitive outcomes of deseg. (See her report "How Desegregation Changed Us.")
Monday, September 24, 2007
1) The winner take all phenomenon: Whether it's the market for music, books, or anything else, attention flows to people based on their relative position (think Harry Potter, Norah Jones, or the difference between being on versus off the NYT Book Review Bestsellers list). Attention breeds more attention, and it snowballs. This is also true in educational research. We know a lot about a small number of researchers. There's a lot more out there, though.
2) The Beltway problem: To be fair about Beltway bashing - all "policy reports" aren't coming from inside the Beltway, and undoubtedly there are some useful and rigorous reports being produced. But a lot of them would never see the light of day if they were peer-reviewed. For journalists and lay consumers of educational research, it's often difficult to tell the difference between "good" and "bad" research. (Disclaimer: this is not to say that I'm the authority on research quality. I'm not. But I do have some opinions. You should weigh in, too.)
3) The mystification tendency: There's too great a divide between researchers and teachers. Researchers don't (or don't know how to) translate their work for broader audiences. Yet new policies are justified to educators in terms of "the research," without ever telling teachers where or to whom they might look if they wanted to learn more about "the research."
"Cool people you should know" is my humble attempt to address these problems. From time to time, I'll profile a researcher you may not know about, but should. I'll list some of their coolest findings, with the hope that the good work these folks are doing finds its way into popular parlance.
Cool people to date:
1) Charlie Clotfelter
2) Tom Corcoran
3) Doug Lauen
4) Linda Renzulli
5) Jesse Rothstein
6) Claudia Buchmann and Claudia Goldin
7) Randy Reback
8) Jennifer Russell
9) Karolyn Tyson
I've nominated four of this fall's best academic books on education for the first (quasi-recurring) eduwonkette gold star award. Here are the books and short blurbs from the publishers about each of them. Give me a couple of weeks to get through them all and I'll announce a winner. In no particular order:
Mitchell Stevens, Creating a Class: College Admissions and the Education of Elites
In real life, Mitchell Stevens is a professor in bustling New York. But for a year and a half, he worked in the admissions office of a bucolic New England college that is known for its high academic standards, beautiful campus, and social conscience. Ambitious high schoolers and savvy guidance counselors know that admission here is highly competitive. But creating classes, Stevens finds, is a lot more complicated than most people imagine.
Admissions officers love students but they work for the good of the school. They must bring each class in "on budget," burnish the statistics so crucial to institutional prestige, and take care of their colleagues in the athletic department and the development office. Stevens shows that the job cannot be done without "systematic preferencing," and racial affirmative action is the least of it. Kids have an edge if their parents can pay full tuition, if they attend high schools with exotic zip codes, if they are athletes--especially football players--and even if they are popular.Jack Buckley and Mark Schneider, Charter Schools - Hope or Hype?
Over the past several years, privately run, publicly funded charter schools have been sold to the American public as an education alternative promising better student achievement, greater parent satisfaction, and more vibrant school communities. But are charter schools delivering on their promise? Or are they just hype as critics contend, a costly experiment that is bleeding tax dollars from public schools? In this book, Jack Buckley and Mark Schneider tackle these questions about one of the thorniest policy reforms in the nation today.
Using an exceptionally rigorous research approach, the authors investigate charter schools in Washington, D.C., carefully examining school data going back more than a decade, interpreting scores of interviews with parents, students, and teachers, and meticulously measuring how charter schools perform compared to traditional public schools. Their conclusions are sobering.
Buckley and Schneider show that charter-school students are not outperforming students in traditional public schools, that the quality of charter-school education varies widely from school to school, and that parent enthusiasm for charter schools starts out strong but fades over time. And they argue that while charter schools may meet the most basic test of sound public policy--they do no harm--the evidence suggests they all too often fall short of advocates' claims.
Sociologist Bruce Fuller traveled the country—sitting in preschool classrooms, delving into the birth of universal preschool in California and Oklahoma, and interviewing this robust movement’s eager leaders—to understand the ideologies of childhood and the raw political forces at play. He details how these new progressives earnestly seek to extend the rigors of public schooling down into the lives of very young children. Fuller then illuminates the stiff resistance by some children’s activists, ethnic leaders, and conservatives, who hold less trust in government solutions and more faith in nonprofits and local groups in contributing to the upbringing of young children. Drawing on the voices of teachers, community activists, and political leaders actively shaping this debate,Standardized Childhood shows why the universal preschool movement is attracting such robust support—and strident opposition—nationwide.
David Kirp, The Sandbox Investment: The Preschool Movement and Kids-First Politics
The rich have always valued early education, and for the past forty years, millions of poor kids have had Head Start. Now, more and more middle class parents have realized that a good preschool is the smartest investment they can make in their children's future in a competitive world. As The Sandbox Investment shows, their needs are key to the growing call for universal preschool.
Writing with the verve of a magazine journalist and the authority of a scholar, David L. Kirp makes the ideal guide to this quiet movement. He crouches in classrooms where committed teachers engage lively four-year-olds, and reveals the findings of an extraordinary longitudinal study that shows the life-changing impact of preschool. He talks with cutting-edge researchers from neuroscience and genetics to economics, whose findings increasingly show how powerfully early childhood shapes the arc of children's lives.
Imagine that you were rewarded for doing a particular part of your job well, while other equally important components remained unacknowledged. It's likely that you would begin to focus your time and attention on the rewarded task and shortchange the unrewarded one. Herein lies the problem with the dominant conception of teacher effectiveness today, which assumes that the only goal of schooling is to raise students’ test scores.
Public schools, like most organizations, have many goals. Certainly, a central goal of American schools is to prepare children for their futures through improving their academic skills.
A second goal of public schools is to prepare children to become active citizens in a democratic society. Students, at the very least, must have the social skills and academic tools necessary to serve on a jury, vote, and understand the rights and responsibilities implied by our social contract.
A third goal of public schools is social mobility. The social mobility goal sees schools as breaking the link between parents and children. In this view, schools level the playing field by providing a venue in which each student can showcase his natural talent and merit. (See David Labaree for a fuller discussion of these goals.)
While they are not mutually exclusive, the three goals introduce very different metrics of educational success. However, the current policy debate about teacher quality, which privileges students' standardized test scores as the sole measure of teacher performance, is strangely out of sync with the longstanding American acknowledgement of the multiple goals of education.
To be sure, America needs students with strong academic skills, and we need good teachers to get there. But since the doors of our first common schools swung open, public education has been about more than academic achievement. Before moving forward with plans to identify and reward effective teachers, we must first answer a critical question: what does it mean for a teacher to be good?
We want our children to grow up not only to be skilled workers, but good citizens, good neighbors, and good parents. Social and civic development are important for each of these goals. Even those unconvinced about the intrinsic value of these non-cognitive skills would agree that task persistence, flexibility, eagerness to learn, and civic mindedness matter because they can boost academic achievement. (And if you are interested in the economic return to non-cognitive skills, see this summary of Nobel prize winning economist James Heckman's recent work.)
Unfortunately, current proposals to measure and reward teacher effectiveness solely on the basis of test scores leave these important education goals out of the picture, and strip away much of the stuff of good teaching. According to these proposals, a good teacher is simply one that increases students' test scores, and such teachers should be rewarded financially.
The danger of an exclusive focus on academic benchmarks is that teachers may overemphasize academic development at the expense of the other goals of education. This would be a loss not only for individual students, but for society at large.
As local, state, and federal policymakers design teacher incentive policies, they should address the full range of skills that constitute good teaching. We need to develop strategies to assess teachers' contributions to these other goals. In all other professions, one's boss, colleagues, and clients weigh in on job evaluations. Schools should adopt similar techniques to assess teachers' effectiveness in promoting skills not measured by standardized tests. Principals, colleagues, and parents all have a role to play in this process. But ignoring these skills altogether comes at too high a price.
Despite their image as meritocratic beacons of opportunity, the selective colleges serve less as vehicles of upward mobility than as transmitters of privilege from generation to generation.
Karabel goes on to point out that kids from the top socioeconomic quartile are 25 times more likely to attend a top tier college than those from the bottom quartile.
All of this raises the question of whether attending a highly selective college actually matters. Take a look at an old paper by Princeton economist Alan Krueger, who found that when you make an apples-to-apples comparison (students attending highly selective colleges versus those admitted to highly selective colleges who went to less selective colleges instead) you're no better off attending an elite school, at least in terms of income. Great summary ("Children Smart Enough to Get into Elite Schools May Not Need to Bother") of his study here.
There was one notable exception. Low-income students who attended highly selective schools did benefit financially, which makes Karabel's call for more attention to these students' representation an important addition to the usual chorus of whining from elite parents.
Sunday, September 23, 2007
1) Every child should be prepared to succeed when they show up in the classroom.
2) Every classroom should be led by an excellent teacher.
3) Every teacher should work in an outstanding school.
What remains to be seen is how the details shake out (i.e. paying for it). Teacherken has more analysis over at dailykos. For McSteamy's plan, click here. And we're still waiting for Addison to leave private practice and give us something concrete - only vague details on her website.
Let me note that I am not opposed to measuring and rewarding teacher effectiveness in principle. But it’s more complicated than most commentators would like to acknowledge, and I hope this week’s postings will help us think about that complexity.
Monday: Tunnel vision syndrome - The teacher effectiveness debate focuses only on a narrow set of the goals of public education, which may endanger other important goals we have for our schools.
Tuesday: No teacher is an island - The teacher effectiveness debate ignores that teachers play many roles in a school. Experienced teachers often serve as anchoring forces in addition to teaching students in their own classrooms. If we don’t acknowledge this interdependence, we may destabilize schools altogether.
Wednesday: Ignoring the great sorting machine - If students were randomly assigned to classrooms and schools, measuring teacher effects would be a much more straightforward enterprise. But when Mrs. Jones is assigned the lowest achievers, and Mrs. Scott’s kids are in the gifted and talented program, matters are complicated immeasurably.
Thursday: Overlooking the oops factor - Everything in the world is measured with error, and the best research on teacher effectiveness takes this very seriously. Yet many of those hailing teacher effectiveness proposals missed out on Statistics 101.
Friday: Disregarding labor market effects - The nature of evaluation affects not only current teachers, but who chooses to join the profession in the future and where they are willing to teach. If we don’t acknowledge that kids that are further behind are harder to pull up, we risk creating yet another incentive for teachers to avoid the toughest schools.
(Kickline roster (from left to right): Eli Broad (Broad Foundation), Kati Haycock (Ed Trust), Michael Bloomberg (NYC), Michael Petrilli and Checker Finn (Fordham).)
• A selective reading of educational research: The loudest outlets pick and choose which studies are relevant, often leading to a skewed view of what we know and don’t know about how to improve schools.
• An inattention to the costs and benefits of policies: Policy solutions are endorsed as if they have no downside. But we know that all actions have positive and negative consequences. The education policy debate would benefit from such an acknowledgement.
• A fundamental disrespect for the knowledge of teachers and principals who work in public schools: Too often, teachers and administrators are dismissed as “self-interested” or “protecting the status quo” when they question what policymakers wreak on their classrooms and schools. In no other profession are we willing to discount the opinions of those closest to the work at hand. Education should be no different.
Rather than stepping into this ideological boxing ring, this blog takes a different approach.
Eduwonkette will consider some of the most contentious education policy debates from a different perspective, including teacher quality, testing and the No Child Left Behind Act, merit pay, the achievement gap, charter schools, school choice, social promotion, and school funding. In addition to providing regular commentary on the real-time education news, each week I’ll pick one of these issues to discuss in more depth.
My hope is that this blog provides a forum for a broader discussion about the possibilities for, and barriers to, improving schools for American children.
But eduwonkette is not all about being wonky. We'll have some fun, too. See below for a preview.
Feel free to email me at eduwonkette (at) gmail.com with comments, ideas, or questions.
There's no bettter way to kick off eduwonkette than with a cameo by the original Eduwonk. Here Andy Rotherham fulfills his lifelong dream of pop stardom.
You didn't really think that was Britney performing at the VMAs, did you?