Leonie Haimson does exactly what one would expect a class size advocate to do – make a forceful argument in favor of the benefits of smaller classes, emphasizing the studies that support her position, and minimizing those that do not. I’m not interested in a point-by-point sparring match with her; suffice it to say that I think that her broad claims for the benefits of class size reduction are not well-supported by the relevant social science evidence.
Part of what is at issue is what kinds of evidence are persuasive. There are two main considerations: (a) a study’s internal validity, and (b) a study’s external validity.
Internal validity matters. Internal validity refers to the credibility of cause-and-effect claims within a given study. The STAR study was a randomized experiment, and the design gives us a lot of confidence that the more favorable outcomes observed in the small class treatment was due to being in a class of 13 to 17 students, rather than a class of 22 to 25 students. Lots of other studies are not experiments. Some of them show more positive outcomes in small classes and others do not; but the challenge is justifying the claim that the association between class size and student outcomes is caused by being in a smaller class, rather than by other forces. In some studies, these alternative forces are known as selection bias—the nonrandom sorting of students into smaller and larger classes on the basis of student characteristics that might be associated with student outcomes. For example, we would not want to conclude that the reason that academic performance is higher in small honors classes than larger regular classes is because of class size. eduwonkette constantly reminds us of the dangers of selection bias. In others, such as the AIR/RAND study of California, other policy and assessment changes were occurring at the same time as class size reduction, and the researchers were unable to rule out the possibility that changes in student performance during the class size reduction period were due to these other changes. Like many social scientists, I give more weight to studies that have high internal validity. Economist Alan Krueger agrees: “[class size] studies are of varying quality and often examine very different outcomes for different populations. Should the amalgamation of such studies be trusted? Personally, I think one learns more about the effect of class size from understanding the specifications, data, methods and sensitivity of results in the few best studies than from summarising the entire literature.”
External validity matters. External validity refers to the extent to which claims that are derived from a particular study apply more broadly to other settings. Studies are conducted in a particular time and place, with a particular target population. I’ve suggested that a study of class size reduction in kindergarten and first grade might not tell us much about class size reduction in high school, because the settings are so different. I’ve also suggested that how class size reduction is implemented matters. Are the teachers qualified? Is there adequate space? The issue of external validity is also at the heart of economist Edward Lazear’s model that eduwonkette summarized last week: reducing class size in classes of well-behaved students may not tell us much about reducing class size in classes of poorly-behaved students, and vice versa. Economist Steve Barnett makes this point in discussing the three best-known early childhood education studies: Perry Preschool, Abecedarian, and Chicago Child-Parent Center. “No one should expect any public program to produce the same results as any one of the studies,” he writes. “To borrow a phrase from the US Environmental Protection Agency, for any particular public [early childhood education] program ‘your mileage may vary’. In general, variations in the population served, program design, and the neighborhood and broader social context can be expected to affect costs and benefits.” The same is surely true for class size reduction as well.
I thought that my posts last week were a balanced but positive endorsement for class size reduction, based on a reasonable reading of the research evidence. Leonie Haimson disagrees. My main crime seems to be that I don’t agree with what “actually” happened in STAR, in California, etc., which, miraculously, always favors smaller classes. Always! But don’t take my word for the “conventional wisdom” on class size reduction. Here are some quotes from “Class Size: Counting Students Can Count," a publication of the American Educational Research Association researched by the lead authors of the STAR study, Jeremy Finn and Charles Achilles, and reviewed by David Berliner, Eric Hanushek, and Larry Hedges. Read these quotes and judge for yourself if I’m distorting what leading researchers think about class size.
“The most dramatic impact seems to be achieved by reaching students early. Ideally, students should experience small classes of 13 to 17 students when entering school, in either kindergarten or first grade. While there is strong evidence of academic improvement during the first two years spent in a small class, there is more ambiguity about the value of additional years. It is not certain that there are added gains during second- and third-grade small classes.”
“There is no experimental research suggesting that any benefits are realized by subtracting only a few children from a larger class — for example, transitioning from 28 to 25 students. Even a class of 20 students may be too large.”
“In California, a lightning-quick ramp-up of statewide class-size reduction policy created many complications. Many new classrooms had to be found or built, and thousands of new teachers were hired within several months of the 1996 launch. Teaching quality suffered. While test scores have gone up in California since the small-class initiative started, researchers have been unable to determine how much, if any, of the improvement resulted from class-size reductions, as opposed to several other initiatives that were launched at around the same time.”
“[Small classes] are not a cure-all for low academic achievement, and they may not always be the best use of scarce resources. In weighing the pros and cons of a class-size reduction plan, policymakers will want to measure the costs of class-size reduction against other possible uses of the same funds.”
Let me close with a couple of suggestions to proponents of smaller classes. First, the hearts and minds of policymakers will not be won over by hitting them over the head with an increasingly thick sheaf of studies touting the effects of class size reduction. This is not a topic on which research is the most important determinant of policy outcomes. (In fact, there are very few policy issues that hinge most directly on what the research has to say.) One need only look at the literature on high-quality preschool programs to see that research hasn’t carried the day. The cost-benefit analyses of the Perry Preschool Project conducted by economist Clive Belfield and his colleagues have shown substantial benefits to the general public through age 40, to the tune of almost $13 for every $1 invested, across categories such as tax contributions, criminal activity, and welfare receipt. (There are benefits to individual participants as well.) But we’re still awaiting adequate funding for high-quality early childhood education programs for all children who can benefit from them. Why should it be different for class size reduction?
Second, I think a more successful strategy would be to focus on the development of a small, well-designed class size reduction initiative that can generate some “small wins” that can then be used to leverage expansion. Look, I have no love for the weasels that populate the senior management of the NYC Department of Education. (They’re not all weasels, of course, but let’s just say that the NYC Department of Health would shut down Tweed Courthouse faster than a KFC/Taco Bell in Greenwich Village.) But successful policy reform involves finding ways to work with managers and policymakers to craft actual class size reduction initiatives. All of the research in the world is not an actual class size reduction initiative. That’s what’s going to make a difference, and that’s where I’d direct my energies if I were a class size advocate.
7 comments:
I'd agree that skoolboy's original discussion of the research on class size was pretty balanced. However, from a policy making perspective (I'm a local school board member...) there's something frustratingly circular about a lot of the discussion.
Say your district has class sizes of 30 elementary grades. The STAR study looks at the difference between classes of 13-17 compared with classes of 22-25, so in the narrow sense it tells you little about whether trying to reduce class size from 30 to 25 would be a good thing. But does it really (as some people claim) tell you NOTHING?
My sense is that one of the biggest impediments to real educational reform is that too many people are looking for simple, unambiguous answers. And preferably, simple, unambiguous answers that involve no additional spending. The sort of answers that could be the centerpiece of some presidential candidate's education platform...
But reducing the scope and stakes a little: for policy makers working on a smaller scale, what reforms, besides reducing class size, should be getting our attention?
Rachel, that's a hard question to answer in the abstract, because every district has a different mix of problem conditions and values, and resources available to address the problem conditions. I wouldn't start with reforms; rather, I'd start with diagnosing problem conditions that are amenable to policy action. Different stakeholders will have diverse and sometimes conflicting ideas about which problems are salient and most important. A trick espoused by Eugene Bardach is to try to define problems in terms of deficits or excesses, e.g., "too many new teachers in our district leave within the first three years," or "the demand for our summer enrichment program exceeds the supply of seats." I would argue that a formulation such as "too many kindergarten students are in classes larger than 20 students" is not a good problem statement, because it builds the solution (class size reduction) into the problem. A better formulation might be "too few kindergarten students are active participants in learning activities," which allows for a broader range of potential policy solutions than just class size reduction.
Wonderful debate! Had you two continued, I would have continued to agree mostly with the person who I had read last.
Discussion is key. A decade ago, when we were more of an inner ring suburban school and I was teaching freshmen, I saw how learning decreased after we had more than 33 or 34 students. Today in our much poorer school, the dropoff occurs closer to twenty. Back then I had a class of around 70 for a month The biggest apparent problem was that every square inch was taken and big football players accidently elbowed their neighbors as they wrote, and I only had about three square feet and I'm always moving. There was no discipline problems but of course the weaker students had the biggest dropoffs in learning.
Last year I had forty-plus seniors in classes and often students in one side of the class had cousins who had killed cousins of students on the other side of class, and vica versa. But they behaved well in class and worked. My intuition says they learned less than the huge class of a decade before.
I had classes of 70 sophomores last year for a couple of weeks and after a Quarter they were reduced to 35. Those classes never recovered. I predict they will have a dropout rate of 85% or more.
This is a long way of agreeing with both of you. Class size reduction should be a main priority, certainly over the gimmicks that have grown with NCLB. But as long as poor districts resort to the most outrageous methods of balancing their books (in my case assigning 240 students to me, and 200 plus to many other core teachers while reducing class size in tested subjects to 15 and while spending millions on on-line tutorials and the infrastructure for testing.)we will need to discuss the best practical ways of investing in personalized settings.
John Thompson
I'm sorry but skoolboy's arguments betray a fatal lack of familiarity with either children, classrooms or the reality of the Bloomberg administration's tactics. As parents we are used to the patronizing and condescending attitudes of the administration but it's a bit much to be lectured here on how to advocate for what has come to be expected in every other learning environment in the state, public or private.
If I were an academic I would direct my energies to finding positions I felt strongly enough to defend without hiding behind anonymous facades.
Patrick,
You're free to disagree with me, but attacking me personally violates the spirit of eduwonkette's blog, and we're both guests here. Back in October, she posted on attacking the ideas, not the person.
Do you think the current advocacy approach has been successful in reducing class sizes in New York City? I don't believe that it has. That's why I am advocating a different approach.
Under mayoral control, only two men have any real say in what happens in the schools, the mayor and chancellor. Given this governance system and the particular tactics employed by the administration, I think the class size advocacy approach taken by Class Size Matters has been far more successful than I would have thought possible.
The reality is that the mayor and chancellor absolutely do not want to reduce class sizes yet the pressure on them to do so is intense. They have consented to begin with 75 schools in response to pressure brought by the State under the Contracts for Excellence agreement. Advocacy groups like CSM and other had a large hand in that. As a parent of elementary school kids in classes of 28, I'm not happy with the progress but trying to sell Tweed on a pilot program as you suggest is not a viable strategy.
As for attacking you personally, I think you are being overly sensitive. You are making comments on education advocacy and politics and your qualifications to do so are suspect. I think it is important for the readers here to understand that. Honestly, as an anonymous blogger it is not clear if you even have the qualifications to make the academic arguments you make. Public school teachers who fear retaliation absolutely should employ anonymity but I don't see why academics need to.
As for Eduwonkette's "policy" on comments, it is my kids in the system. The academics, politicians and faceless foundations with no skin in the game are going to hear from me whether they like it or not.
Post a Comment