(Props to despair.com for the image.)
The identification of especially "effective schools" for poor and minority students dates back to Ron Edmonds' work in the late 1970s (hat tip to Sherman Dorn and anonymous 6:16AM). The Education Trust (1999, 2001) and Heritage Foundation (1999) have carried the Edmonds flag since then. Both have published reports on schools that are high poverty/high minority, but are "beating the odds."
What does it mean to be "high-flying" or "beating the odds?" The 2001 Education Trust report defined high-flying schools as those schools in the top 1/3 of their states' test scores that are also “high-poverty” (more than 50 percent of students eligible for free or reduced price lunch). Over 3,500 schools were identified. According to Ed Trust, if this many schools can make it happen, all schools should be able to.
It turns out that a school could make the Ed Trust's list if it posted high achievement in only one subject, in one grade, for one year. Doug Harris, an economist at the University of Wisconsin - Madison, reanalyzed these data (paper here), and found that the number of high flyers was, unfortunately, too good to be true. In his analysis, the Ed Trust definition classifies 15.6% of high poverty schools as “high flyers.” Requiring high performance in two subjects, for two grades, for two years reduces the number of high-flying schools to 1.1% of high poverty schools.
These high-flying schools are what statisticians call “outliers” – data points that are quite atypical of the general relationship between school poverty and school achievement. (See also my previous posts on KIPP on this issue - and for more on the problems with Ed Trust and No Excuses lists, see the Harris paper and Chapter 2 in Richard Rothstein’s Class and Schools.)
2) If some schools with high concentrations of minority and poor students are getting good results, poverty must not affect academic achievement - at least not in ways that can't be overcome by good schools.
After the release of “Dispelling the Myth Revisited,” the Education Trust's Kati Haycock commented, "How many effective schools do we have to see in this country before we conclude that it's not the kids?" (See the NY Times article.) Advocates like Haycock often argue that educators' low expectations, not students’ out-of-school conditions, remain the biggest challenge faced by poor children.
One way of thinking about the impact of poverty on student achievement is to consider the odds that a low-poverty school will exhibit consistent high achievement compared to a high poverty school. Harris found that:
Low-poverty schools are 22 more likely to reach consistently high academic achievement compared with high-poverty schools. Schools serving student populations that are both low-poverty and low-minority are 89 times more likely to be consistently high performing compared with high-poverty, high-minority schools.
This argument has been made at length elsewhere, so I won’t go on. For more, see Annette Lareau’s Unequal Childhoods or Richard Rothstein’s Class and Schools.
3) If some schools can get exceptional results in spite of the challenges their students face, all schools should be able to.
Let’s think about what it would mean for the bottom 50% of any group to perform at the level of the top 1%.
Comical examples first – this argument implies that if we all went crazy at the gym, we could all achieve the physiques of the top 1% - i.e. Heidi Klum and David Beckham. It also suggests that if students currently scoring in the 25th percentile of the SAT distribution studied harder, they could make it to the 99th percentile.
Proponents of this argument will respond by saying “we’re just asking for basic proficiency.” But that does not change the fact that they are asking 99% of high poverty schools to suddenly do what only 1% have been able to. We don’t use exceptional performance – the top 1% - in any other field to argue that everyone else can do just as well. It makes no sense to do so in education.
4) These high-achieving schools employ shared "best practices." These "best practices" have a positive causal effect on educational success.
Boring alert - this claim is a basic misunderstanding of the concept of conditional probability. Conditional probability is the probability of one event happening given that another has occurred.
In this case, what organizations like the Ed Trust really want to know is the probability of a school being a high-flyer given that it uses one of these “best practices” (i.e. high expectations, collaborative decision-making, etc). Instead, “high-flyer/no excuses/it’s being done” studies look at the characteristics of a school given that it is a high-flyer, and then attribute these schools' success to these characteristics.
But if you want to identify the effect of particular practices on achievement, you want to know the probability of a school being a high flyer given that it uses best practice #10. There are lots of schools where teachers have high expectations, work collaboratively, make decisions based on what is best for students, etc that do not achieve exceptional results.
Of course, it is the case that there are practices and structures that increase the odds of success. But even if everyone used these practices, I suspect that most high-poverty schools would still be low-flying.
5) If schools aren't achieving results on par with these "high-flying" schools, it is the fault of the schools and their educators.
Other folks have argued this point more eloquently than I can, so here's an excerpt from last week's Richard Rothstein/Russlyn Ali debate. Rothstein wrote:
Pretending that more effective schools can close achievement gaps on their own — promises the impossible, setting schools and teachers up for failure. Why shouldn't the public conclude that schools are incompetent if educators cannot achieve what some foolishly promise? I am often accused of letting schools "off the hook" by making this argument. Not at all — both schools and social policy need improvement. But claims that schools alone can close achievement gaps let politicians and business leaders "off the hook." We let them claim one day that it's too expensive to provide health insurance to all children, and on the next pose as advocates for minorities by demanding that schools close the gap.
To sum up - I am not arguing that schools don't matter or that schools can't have important impacts on kids' life chances. They can, and they do. But the existence of a small number of schools with exceptional results is not evidence that all schools can produce them.
9 comments:
I agree with much of what you say here -- but there are some specific education reforms that do work to close the achievement gap and that have not been implemented widely, , including increased access to preK and smaller classes.
Though the people at Ed Trust almost never focus on such concrete reforms but instead cite the more amorphous qualities of "high expectations" etc., even guys like Rothstein give these options short shrift.
Of course, we should have national health insurance, more equitable income distribution, etc. etc. but why do I need an education writer like Rothstein to focus on this?
Instead, he should be writing about the ways our schools should and can be improved to have better outcomes. Yet often he comes off sounding far more defeatist than anything else.
I prefer Jonathan Kozol's attitude: until we offer class sizes that approach those offered to the elite, the achievement gap should not be written off as impossible to solve.
I was looking forward to your picking my work apart, but I find you are just repeating stale and worn arguments.
The most interesting thing you said was: "Of course, it is the case that there are practices and structures that increase the odds of success. But even if everyone used these practices, I suspect that most high-poverty schools would still be low-flying."
Wouldn't it be worth trying to identify those practices and structures and then making sure all children--particularly poor children--have access to them? Then we could have a real discussion of the effects of poverty on learning. Right now it is impossible to separate out the effects of bad schooling on students who grow up in poverty because so many poor children are forced to attend schools that do not use the "practices and structures that increase the odds of success."
Karin Chenoweth
Author
It's Being Done: Academic Success in Unexpected Schools (Harvard Education Press, 2007)
I think Karin has a point, though I don't think its going to be enough to close the achievement gap.
It's true that no matter how much I studied David Beckham's "best practices" its very (very, very...) unlikely that I could implement them in a way that raised my soccer achievement to the level of his. However, if I want to increase my soccer achievment, studying his best practices is probably a good place to start.
On the other hand, if my coach took the view that all I needed to do to play as well as David Beckham was work harder at implementing best practices during in the 6 hours a day I attended practice, he would clearly be missing significant impediments to my progress.
The problem I have with so many of the "it's being done" arguments is that they seem so often to be reduced to "you could do it if you just tried harder."
From what I've seen there's pretty compelling evidence that small classes and strong teacher collaboration are elements of an effective school. But even those are not completey in a single school's control. Small classes require a budget that can support the staffing, and strong teacher collaboration requires a supportinve culture and, most likely, a relatively stable staff.
It would be really interesting to see a study of what effect implementing certain reforms had in real life situations.
But it also seems very unlikely that school achievement really is completely unaffected by all the other "gaps" in childrens lives.
I believe the "Logic" image is from despair.com... you should give credit for images that are proprietary.
So what does the literature on class size say? I know there were some positive results from the Tennessee experiment, but has anyone refuted Jepsen and Rivkin?
I had the pleasure of hearing Karin Chenoweth present her work at EdTrust last month. Administrators and staff from two of the schools she wrote about were also there. I had the same reaction to their stories as I typically have when I hear or read about "It's being done" schools: 1) These are genuinely inspiring educators, and 2) These are not models for all schools.
Let me quickly explain what I mean by #2. One of the things that the "high flyer" schools seem to have in common is a superhuman dedication, top to bottom, from all of the adults in the school. This is moving, heartwarming, laudable....and unsustainable.
Teaching is the easiest job in the world to do badly. It's also the hardest job in the world to do well. I'm not merely referring to craft and expertise, but the pure, raw energy it takes to work in a troubled school. The gains are incremental and the entire enterprise can at its worst seem Sisyphean. It takes a special breed of person to do the hard work that needs to be done, day after day, and not be physically and morally exhausted by it after a very short while.
Unless one is ready to suggest that we can run schools effectively like fast-food restaurants -- lots of cheap, young staff; churn 'em and burn 'em -- we will probably get more of what we have right now. The top schools will continue to be filled with dynamic and charismatic school leaders with the ability to attract true believers and produce remarkable results. Everything else will continue to be an undifferentiated mass of mediocrity, or worse.
In other words, outlier schools are filled with outlier people. There simply aren't enough of them to go around.
rpondiscio@aol.com
I have seen the Haycock scatter plot of poverty vs achievement, and I have seen it replicated for my own state using 3rd grade reading data. It always strikes me that the hard and fast determinants are at the higher end of the income scale. At higher income levels, achievement is tightly clustered at the top. As income declines, the data becomes much more widely scattered--in fact covering a range from no success to the so-called outliers at the top. It would appear to be far less true that poverty determines lack of success than that affluence determines success.
I support and honor the troops, but not the war in Iraq. For years, the Bush spin machine successfully ridiculed that logic.
I support best practices and honor the the best high performing high poverty schools, but reject NCLB.
The last liberal believers in Bush's law borrow the science of the tobacco industry (not all smokers get cancer and some nonsmokers do)to claim that a few outliers prove that NCLB could work.
Through the miracle of the blogoshere, I can express my opinions but I get uneasy when I see them on the screen. Had I had such an opportunity in 2002 I could have described the damage caused by Voodoo economics as it accelerated the migration of inner city jobs to the exurbs, the market forces that are worsening health services for my students, and the drug war that backfired when a) they took discretion and out of the judical system and b) committed to data-driven policing. I could have explained how NCLB was based on those principles by placing too much faith in expectations, the market and taking the human element out.
I wouldn't have written that because I then gave cautious support to NCLB. And I would have never put so much energy into condemning liberals for adopting the methods of the Republican machine in order to achieve our principles. So what changed?
My criticisms of the Chenoweth/ Ed Trust approach are based on fear. They have been so effective in putting a smiley "Morning in America" face on a law that is condemning schools to nonstop test prep.
But I need to move beyond my fears. Just in the last year, an enormous body of research has shown how and why NCLB has failed. So far, the spin machine has been holding its own but what about the next two years? If NCLB gains traction and shows results, then I need to shut up. But if the research continues to show that hard accountability imposed on a national level is backfiring, then even the most determined NCLB supporters will have to rethink their game plan. By the time a new president takes office (I support Obama and his plan, but all of the Dems are good),and teachers have overwhelmingly voted with their feet, we'll have to take an apporoach other than the blame game. (Polls say that most teachers used to support NCLB but I don't think I personally know anybody who still does.)
So then, we can invite the Raise Expectations school to agree to disagree on hard accountability and focus on areas where we agree wholeheartedly. Chenoweth et.al. agree that poor students need real rigor, relevance, and respectful relationships, that we need to address the drop out crisis, and that rote standardization is not the answer. So why not craft a law that DIRECTLY addresses those issues?
I'm not afraid to make concessions on Standards, differiential pay, local adjustments in collective bargaining etc. I'm just afraid of the No Child Left Untested that is driving my students out of school and desecrating the principles of public education for all.
But I see a pendulum that is swinging and I don't think we need to be afraid anymore. The fill in the bubble method of achieving a civil rights revolution on the cheap is declining. I intend to swallow my old fears, bite my tongue and invite liberal supporters of NCLB into a two-year process to draft a completely different type of law.
John Thompson
Michael Winerip in Sunday's Times, unfortunately buried in the regional parenting section, starts his article with this:
THE federal No Child Left Behind law of 2002 rates schools based on how students perform on state standardized tests, and if too many children score poorly, the school is judged as failing.
But how much is really the school’s fault?
A new study by the Educational Testing Service — which develops and administers more than 50 million standardized tests annually, including the SAT — concludes that an awful lot of those low scores can be explained by factors that have nothing to do with schools. The study, “The Family: America’s Smallest School,” suggests that a lot of the failure has to do with what takes place in the home, the level of poverty and government’s inadequate support for programs that could make a difference, like high-quality day care and paid maternity leave.
The E.T.S. researchers took four variables that are beyond the control of schools: The percentage of children living with one parent; the percentage of eighth graders absent from school at least three times a month; the percentage of children 5 or younger whose parents read to them daily, and the percentage of eighth graders who watch five or more hours of TV a day. Using just those four variables, the researchers were able to predict each state’s results on the federal eighth-grade reading test with impressive accuracy.
Commentary and links at ednotes.
Post a Comment