Can School Choice Close the Achievement Gap? Evidence from Chicago

Part 3 on school choice. Previous posts archived here.

The central claim of school choice proponents is that increased choice will give poor and minority kids access to higher quality schools and thus close the achievement gap. But we actually have very shaky evidence about the effects of schools of choice because of students' non-random assignment to these schools.

Along comes a spectacular new paper by economists Julie Cullen and Brian Jacob on school choice in Chicago, which provides the most rigorous evidence to date on this question. Chicago has one of the largest school choice systems in the country - more than a third of Chicago elementary students attend a school other than the neighborhood school assigned to them. Cullen and Jacob exploit the fact that most schools of choice assign slots by lottery in kindergarten and first grade; with longitudinal data, they can follow these students for five years.

What do they find? While lottery winners attend schools that are of higher quality in terms of their peers' achievement and the value-added by the school, it turns out that attending a school of choice confers no academic benefits on students, and these effects are consistent across demographic subgroups. Their conclusion has significant implications for current education policy, including the choice provisions of NCLB and many urban districts' efforts to expand choice:

Given the multiple disadvantages faced by poor families and the multiplicity of support services, along with the uncertainty regarding the impact of school quality on student outcomes, simply attending a better school may not be the most effective intervention....The original analysis conducted in this chapter suggests that schools are a blunt instrument for improving the achievement of disadvantaged students....We cautiously conclude that access to "better" schools is likely to be less effective than more targeted interventions.

See also the Cullen, Jacob, and Levitt paper on school choice in Chicago at the high school level, which also finds no academic benefits but important social benefits (i.e. fewer disciplinary problems and lower arrest rates).

Cool people you should know: Doug Lauen

Cool people you should know #3!

This week I've been talking about school choice, so who better to introduce to you than an up and coming school choice researcher. Doug Lauen is an Assistant Professor of Public Policy at the University of North Carolina at Chapel Hill. His dissertation was about the causes and consequences of Chicago's public school choice programs. For all of you school choice junkies, you can find his papers here.

In his most recent paper, Lauen addressed a gaping hole in school choice research. Most research has focused on individual students and families, and ignored the role of one's surroundings - for example, how disadvantaged one's neighborhood is - in affecting one's likelihood of choosing a non-neighborhood public school or a private school. Lauen takes into account not only individual characteristics, but contextual characteristics. Some of his findings:

• Attending a predominantly black elementary school, living in a predominantly black neighborhood, or living in a neighborhood with a high degree of concentrated disadvantage decreases the chances of attending a selective enrollment school.
  
• Affluence has the opposite effect, increasing students' chances of attending a selective school.
  

Do Parents Choose School Quality or School Racial/Class Composition?

Part 2 of 3 on school choice. Yesterday's posting here.

A central concern in the school choice debate is whether parents choose schools based on school quality (i.e. academic performance and school culture/climate) or school status (race and class composition). Survey data addressing school choice certainly suggest that parents care about academic performance. But stated preferences (what people say) and revealed preferences (what people really do) are very different things. My conclusion from reading this literature is that parents of all backgrounds - i.e. low/high income or white/minority - are choosing on racial and class composition as much or more than they are choosing on "true" school quality as captured by schools' academic performance and climates.

I've been reading NCES Commissioner Mark Schneider and colleague Jack Buckley's book, Charter Schools: Hope or Hype?. One chapter of this book reports on an innovative study of parental preferences through which parents' use of a website (DCschoolsearch.com) was tracked, such that the researchers could document which features of the schools parents looked at, and in what order. Guess what the heaviest hitter was? Demographics.

The Schneider/Buckley study adds to a growing body of evidence on this issue. A new paper by Teachers College professors Aaron Pallas and Carolyn Riehl found that even after controlling for the academic quality of the school, schools in New York City with higher concentrations of black and ELL kids received fewer applications; this is in a district that is only 15% white, so non-white families are driving these patterns. Another study by Salvatore Saporito and Annette Lareau, while finding white aversion to schools with high black concentrations, found that African-Americans also attempt to avoid schools with high poverty rates. This is not just a US phenomenon - researchers looking at the UK (see Stephen Ball), Chile (see Greg Elacqua), and New Zealand (see Edward Fiske and Sunny Ladd) have identified similar patterns - school socioeconomic and racial composition are central elements of the choice process.

For a qualitative look at this process, check out Jennifer Holme's study. She's a professor of education at the UT-Austin, and also found that parents prioritize school race/class composition, rather than school quality, in her study of home purchasing behavior. For example, one parent interviewed explained her choices in the following way:

We also looked at the mix, the ethnic mix...to see if we could determine if there was a lot of foreign students, I guess you'd call them....We just felt like if there is...a language barrier or things like that to overcome, that might cheat the education process a bit...I believe it's good to be shown other cultures and everything, but there definitely seemed to be a correlation...I just thought we were going to try to avoid that.

Considered together, these studies provide sobering evidence about the potential of increased school choice to spur schools to improve the quality of education they offer. If parents aren't choosing on school quality, "market pressures" can't work to improve school quality. Perhaps more importantly, these studies offer another reminder of the central role that race continues to play in American education.

The Turnaround at Evander Childs: A NYC Small School Trick?

New York's Fund for Public Schools, which has raised substantial funds for NYC's reforms, has launched a new ad campaign called "Keep it Going New York City." One ad showcases the succesful creation of new small schools within large high schools. Watch this ad called "Evander Childs Turnaround" - the main idea here is that Evander Childs, a high school in the Bronx, was failing, dangerous, and a poor environment for learning. Enter Bloomberg/Klein, the Children First reforms, and five new small schools, and Evander is reborn - teachers say it's different, students say they like going to school there, and a principal beams that the graduation rate has increased from 30% to 80%. Evander certainly has received a lot of attention - Joel Klein visited the school to deliver his spring statement on small schools' superior graduation rates. A NY Times Editorial praised new small schools for increasing graduation rates. The final line of the ad: "The building may be the same, but the school is very different." Should we be cheering?

The answer is a resounding "no." The building is the same, but the students definitely aren't. As the tables below demonstrate, the new students were much higher performing before they even entered the building. In the first table, I use the NYC School Report cards to compare incoming 9th graders in 2004-2005 (the last year that Evander took 9th graders) with the 9th graders at the other small schoools. On every dimension, the Evander incoming 9th graders are lagging behind academically - they are more likely to be in special education or to be classified as ELL, they are much more likely to be overage for their grade (i.e. they had been retained before), their attendance rates in junior high school were much lower, and they were much less likely to be proficient in reading and math. Of particular note is the praise showered on Bronx Lab at the end of the 2004-2005 school year - see this NY Times article - but 46.6% of their kids were proficient in reading and 52.7% in math when they walked in the door, while Evander's entering students passed at rates of only 11.1% in reading and 12.8% in math. How did the reporter miss this? How did the NYT editorial board miss these numbers before writing a glowing endorsement? (See Leo Casey's analysis at EdWize that shows similar comparisons for other schools in NYC- the pattern is very similar.)

Comparing these schools is either incredibly foolish or incredibly dishonest - and I don't think the folks running NYC schools are foolish. Click below to enlarge these tables.

(Note: 2004-2005 was the last year Evander Childs took new ninth graders; the 2005-2006 numbers for Evander thus represent 9th and 10th grade transfer students, I imagine - someone with more knowledge of this particular case can comment here.)

* I am not sure what to make of these free lunch figures. Given the disadvantage profile suggested by the performance of kids at Evander (especially that so many of them are over-age for grade and ELL), it is difficult to believe that they don't qualify for free lunch in equal numbers as their peers at the small schools. Another interpretation is that their families are less involved, and thus less likely to turn in free lunch forms; alternatively, their feeder schools may be less organized to get these forms turned in. Other interpretations welcome.

TFA Part II: Confessions of a Temporary Teacher

Last week, in response to the Times article on TFA, I wrote a post evaluating the costs and benefits of TFA. From what we read in the popular press, we would believe that TFA is closing the achievement gap and that the strongest determinant of student achievement is teacher effectiveness. The central ideas that I take issue with are 1) that schools and teachers can do it all and 2) that "smart people" can make it happen. Ironically, TFA's stated theory of change - which acknowledges that educational inequality is, in part, a function of social inequality - is much more thoughtful than this caricature (watch this slideshow), but it's been lost in translation. In what follows, the royal "you" is a presumptuous one - i.e. me - but nonetheless, these are the parts of the TFA narrative that I personally find problematic. My thoughts on what you know after teaching for two years:

You know that “being smart” purchases precious little in the classroom. You realize that being able to write an honors thesis gives you limited insight into teaching your kids to write a five-paragraph essay. You learn that those maligned education school courses on literacy might have come in handy. You learn this when you plant yourself in the back of your colleague’s classroom – who did go through one of those “pointless” ed school programs, as it is now trendy to say – to figure out how to teach what your overrated Ivy League brain does not know how to do.

You know that what happens outside of the school affects – and sometimes even determines – what happens inside. You learn this when homework is missing not because of Thursday night’s sitcom, but because Deneidra got evicted. You learn this when someone’s brother or father or cousin gets shot while running drugs. Or maybe you learn this when Kenneth doesn’t feel like doing his Do-Now because he spent Sunday talking to his dad through glass upstate.

You know that the “no excuses” mantra is more religion than reality. Of course, you do your best anyway – you drive to school on Saturdays to prepare your kids for the upcoming AP test, quiz Valeen on her vocabulary words over the phone, or have Millie, a dazzling high school junior, over to prepare for the SAT. But you also understand that no amount of SAT prep on your living room floor is going to prepare Millie for variations on gondola is to Italy as subway is to New York City. And maybe her score goes up 100, even 200 points if you’re lucky – but so does the score of the daughter of two lawyers who has been surrounded by books since birth and also has a private tutor three days a week - and her baseline score was 300 points higher to begin with.

So you learn that closing the achievement gap is not just a matter of sheer will, because no school serving disadvantaged kids – not even the beacons that pundits point to – has put their kids on par with the kids of the elite. It is then that you learn that education reform is, at best, a stopgap for a much larger problem – one that requires kids to have health care, preschool, and decent housing, and for their parents to have prenatal care and good jobs. So you learn that telling your kids that they can be whatever they want if they just try hard enough is a lie in which you are now complicit by virtue of saying it every day.

You know that people give you more credit than you deserve just because you have elite credentials, and this continues well after you've left the classroom. You learn this when you switch lesson plans with an older teacher the assistant principal is giving a hard time; while she said the same lesson plans were “exceptional” when they were yours, she tells your colleague that they are “under-developed and need more work.” You learn this when your colleague receives only an “acceptable” rating on her observations – though you’ve sat in her class and are certain her kids are learning more science than they ever have - but you receive an "excellent" rating though the assistant principal has never even stepped inside of your classroom.

So perhaps you also learn that your colleagues’ skepticism about your presence was not hostility to reform, but the outcome of thirty years of slights they’ve unfairly received. And when you do leave after stopping in for a couple years, perhaps you also learn that their chidings that you’d never stay were not personal insults, but accurate predictions – the wisdom earned through a career of watching bright-eyed upstarts like you come and go, and then go again.

This week: School Choice - Trick or Treat?

25 days to contemplate your getup and resist your daughter's pleas to rock a Bratz costume. With Halloween on the horizon, we have a facile metaphor through which to consider the issue of school choice. So in this three-part series, I will consider the evidence. What you'll see, I think, is that school choice really is a grab bag of tricks and treats.





Tuesday - The Evander Childs Turnaround: A NYC Small School Trick?

Wednesday - Do Families Choose School Quality or School Racial/Class Composition?

Thursday - Can School Choice Close the Achievement Gap? Evidence from Chicago