(Props to despair.com for the image.)
The identification of especially "effective schools" for poor and minority students dates back to Ron Edmonds' work in the late 1970s (hat tip to Sherman Dorn and anonymous 6:16AM). The Education Trust (1999, 2001) and Heritage Foundation (1999) have carried the Edmonds flag since then. Both have published reports on schools that are high poverty/high minority, but are "beating the odds."
What does it mean to be "high-flying" or "beating the odds?" The 2001 Education Trust report defined high-flying schools as those schools in the top 1/3 of their states' test scores that are also “high-poverty” (more than 50 percent of students eligible for free or reduced price lunch). Over 3,500 schools were identified. According to Ed Trust, if this many schools can make it happen, all schools should be able to.
It turns out that a school could make the Ed Trust's list if it posted high achievement in only one subject, in one grade, for one year. Doug Harris, an economist at the University of Wisconsin - Madison, reanalyzed these data (paper here), and found that the number of high flyers was, unfortunately, too good to be true. In his analysis, the Ed Trust definition classifies 15.6% of high poverty schools as “high flyers.” Requiring high performance in two subjects, for two grades, for two years reduces the number of high-flying schools to 1.1% of high poverty schools.
These high-flying schools are what statisticians call “outliers” – data points that are quite atypical of the general relationship between school poverty and school achievement. (See also my previous posts on KIPP on this issue - and for more on the problems with Ed Trust and No Excuses lists, see the Harris paper and Chapter 2 in Richard Rothstein’s Class and Schools.)
2) If some schools with high concentrations of minority and poor students are getting good results, poverty must not affect academic achievement - at least not in ways that can't be overcome by good schools.
After the release of “Dispelling the Myth Revisited,” the Education Trust's Kati Haycock commented, "How many effective schools do we have to see in this country before we conclude that it's not the kids?" (See the NY Times article.) Advocates like Haycock often argue that educators' low expectations, not students’ out-of-school conditions, remain the biggest challenge faced by poor children.
One way of thinking about the impact of poverty on student achievement is to consider the odds that a low-poverty school will exhibit consistent high achievement compared to a high poverty school. Harris found that:
Low-poverty schools are 22 more likely to reach consistently high academic achievement compared with high-poverty schools. Schools serving student populations that are both low-poverty and low-minority are 89 times more likely to be consistently high performing compared with high-poverty, high-minority schools.
This argument has been made at length elsewhere, so I won’t go on. For more, see Annette Lareau’s Unequal Childhoods or Richard Rothstein’s Class and Schools.
3) If some schools can get exceptional results in spite of the challenges their students face, all schools should be able to.
Let’s think about what it would mean for the bottom 50% of any group to perform at the level of the top 1%.
Comical examples first – this argument implies that if we all went crazy at the gym, we could all achieve the physiques of the top 1% - i.e. Heidi Klum and David Beckham. It also suggests that if students currently scoring in the 25th percentile of the SAT distribution studied harder, they could make it to the 99th percentile.
Proponents of this argument will respond by saying “we’re just asking for basic proficiency.” But that does not change the fact that they are asking 99% of high poverty schools to suddenly do what only 1% have been able to. We don’t use exceptional performance – the top 1% - in any other field to argue that everyone else can do just as well. It makes no sense to do so in education.
4) These high-achieving schools employ shared "best practices." These "best practices" have a positive causal effect on educational success.
Boring alert - this claim is a basic misunderstanding of the concept of conditional probability. Conditional probability is the probability of one event happening given that another has occurred.
In this case, what organizations like the Ed Trust really want to know is the probability of a school being a high-flyer given that it uses one of these “best practices” (i.e. high expectations, collaborative decision-making, etc). Instead, “high-flyer/no excuses/it’s being done” studies look at the characteristics of a school given that it is a high-flyer, and then attribute these schools' success to these characteristics.
But if you want to identify the effect of particular practices on achievement, you want to know the probability of a school being a high flyer given that it uses best practice #10. There are lots of schools where teachers have high expectations, work collaboratively, make decisions based on what is best for students, etc that do not achieve exceptional results.
Of course, it is the case that there are practices and structures that increase the odds of success. But even if everyone used these practices, I suspect that most high-poverty schools would still be low-flying.
5) If schools aren't achieving results on par with these "high-flying" schools, it is the fault of the schools and their educators.
Other folks have argued this point more eloquently than I can, so here's an excerpt from last week's Richard Rothstein/Russlyn Ali debate. Rothstein wrote:
Pretending that more effective schools can close achievement gaps on their own — promises the impossible, setting schools and teachers up for failure. Why shouldn't the public conclude that schools are incompetent if educators cannot achieve what some foolishly promise? I am often accused of letting schools "off the hook" by making this argument. Not at all — both schools and social policy need improvement. But claims that schools alone can close achievement gaps let politicians and business leaders "off the hook." We let them claim one day that it's too expensive to provide health insurance to all children, and on the next pose as advocates for minorities by demanding that schools close the gap.
To sum up - I am not arguing that schools don't matter or that schools can't have important impacts on kids' life chances. They can, and they do. But the existence of a small number of schools with exceptional results is not evidence that all schools can produce them.