This week: Class size, the sequel

Last week, skoolboy wrote a series of posts about class size (see here, here, and here.)

I'll be celebrating the new year on Monday and Tuesday, so we'll meet again in 2008.

On Wednesday, Leonie Haimson of NYC's Class Size Matters will respond to skoolboy's posts. On Thursday and Friday, I'll take a look at the recently released NYC class size data.

In the meantime, check out skoolboy's top 5 education stories of 2007 here. Happy New Year, everyone!

Bonus Prize: More Thoughts on Class Size

I'm back, hopped up on Christmas goodies, and ready to comment on class size. Skoolboy, thanks a million for skillfully steering the ship this week.

On Monday, skoolboy pointed out that the evidence on class size effects on learning is mixed. Though it's clear that reduced class size improves learning in experimental settings - see skoolboy's description of the STAR experiment or this paper by Angrist and Lavy - many studies find no effect of class size on achievement (as measured by standardized test scores). Considered alongside data from around the world, where large class sizes yield high achievement, many have concluded that class size doesn't matter much for learning.

I think they are wrong. Here's why: my ability to squeeze learning out of a class is affected by the behavior of the kids sitting next to me. If the kid next to me acts like a knucklehead, I lose out. On the other hand, you can put 35 highly motivated, well-behaved teenagers in a classroom together, and their learning is going to be minimally affected by the large class. For example, selective schools like Bronx Science have very large classes, and so do countries where kids are expected to genuflect to adults.

Edward Lazear, an economist at Stanford Business School, used this idea - that the ideal class size for learning varies by the behavior/attention span of the students - to explain the conflicting findings in class size research. The trouble is that observational studies - i.e. studies using data from the real world - are mixing together the effects of class size on very different kinds of kids. In a very nice theoretical paper, he concludes that class size matters - but that the effect of class size reduction on learning varies by the behavior of the students in it. (That paper, "Educational Production," is available here.)

Two other features of class size reduction are important to note. First, I would expect the effects of class size reduction to be larger on non-cognitive outcomes (like motivation, self-discipline, interpersonal skills, and engagement) than on test scores. And to be honest, I care as much about these outcomes as I do about test scores. Improvement in these social skills can become pathways to future learning. Second, smaller class sizes could help to keep teachers in hard-to-staff schools and in the profession overall.

But as skoolboy notes, class size reduction succeeds or fails on its implementation. Future class size reduction efforts need to learn from California's experience (skoolboy describes this here) where a class size reduction created an immediate need for thousands of new teachers. (You can read about this in Jepsen and Rivkin's report here.) They advised, "A better approach to class size reduction would have been to reduce class sizes in a subset of schools each year, starting with low-performing schools serving high-poverty populations." I agree.

skoolboy's Top 5 Stories of 2007

I’m supposed to be posting about class size this week, and this is my final post. I thought about crunching some data on class size in New York City, to see if class size differences across schools are socially patterned. I still might do that at some point, but it would have taken more than a day to do it reasonably well—so I passed. I also thought about writing about the cost of class size reduction. The post would have gone something like this: class size reduction is expensive. But if it yields substantial long-term benefits, some of which can be quantified, we can think of it as an investment in human capital, with a positive return on each dollar invested. I might have done some of those “back-of-the-envelope” calculations that economists seem to love. (Why do economists always have a handy supply of envelopes on which to do these calculations? Must be to hold those fat consulting checks. Not that I’m bitter.) But that too would have required a lot of time, and I wasn’t inspired.

So I’m taking the easy way out: the top five most intriguing education stories of 2007: a skoolboy’s-eye view. Hope I’m not upstaging you, eduwonkette! Here goes, in no particular order:

The rise of the education blogs. I started paying attention to education blogs this year. The number has grown exponentially, and the proliferation of blogs has created new opportunities for parents, educators, advocates, and policy analysts (and students too!) across the U.S. and the world to engage with one another about education policy and practice. (The blogs that eduwonkette reads are listed on the right.) Two that I’ve found consistently interesting are Bridging Differences, featuring Deborah Meier and Diane Ravitch, and (surprise!) eduwonkette. I’ve admired Deborah and Diane for a long time, but didn’t think they’d have much common ground to talk about. They’ve demonstrated that informed people with strong convictions can listen carefully to and engage with one another. It’s been a wonderful model of civil dialogue that I hope will continue for a long time. As for eduwonkette: how lame is it to flog the blog you’re guest-posting on? eduwonkette emerged a scant four months ago, but has taken the education blogosphere by storm. Okay, I’ll stop fawning like a, well, skoolboy. Keep an eye on EduDiva, who in her first month has made several thoughtful posts from the Gateway City.

The rise of the education “suits”. The trend of hiring leaders and managers drawn from outside of the education system has been accelerating. Two such outsiders drew a great deal of attention in New York City this year: Roland Fryer and Jim Liebman. Fryer is a young economist on the Harvard faculty who was appointed as the “Chief Equality Officer” of the NYC Department of Education. He’s the architect of a controversial plan to use cellphones as an incentive for students to get high scores on standardized tests. I don’t know Fryer, and wish him well; but I think he’s in over his head. One indicator of this is that press reports suggest that Fryer and his students, not independent researchers, will be evaluating the success of the initiative. I do know Liebman, and I do not wish him well. A Columbia law professor on leave at the Department of Education as “Chief Accountability Officer,” he’s designed an accountability system that is unintelligible, narrow, and internally contradictory, yet is being used to justify closing schools. Next year’s version may even be worse! Much of the mess he has created could have been avoided if he had been a better listener.

Michelle Rhee’s appointment as the Chancellor of the Washington, DC public schools. A Teach for America alumna who founded the New Teacher Project, Rhee faces the challenge of reforming a system with an astonishing legacy of inefficiency rooted in the organizational structure and culture of the system. Tinkering at the margins will do little to change this shameful picture. But bold strokes such as closing schools and converting central-office staff to at-will status will evoke strong resistance from elected officials and the public alike. It’s an open question whether Rhee will be able to persuade stakeholders to support radical change in how the District does business.

The aftermath of Hurricane Katrina. Not a sexy topic for policy wonks, but a critically important one that ought not be overlooked. The failure to invest adequately in rebuilding New Orleans is a national disgrace. Earlier this year, Paul Vallas, former head of the Chicago and Philadelphia school systems, was named to lead the Recovery School District, a district created by the state of Louisiana which houses most of the schools in New Orleans. Vallas is a proponent of what he calls a “diverse provider model” of school management which increases the role of the private sector in public education. Sociologist Carl Bankston, writing in the cyberpages of the Teachers College Record, suggests that, whatever Vallas might do, successful educational reform probably hinges on a sustained commitment to rebuilding the city.

The increasing influence of philanthropies in public education. We are witnessing a consolidation of power and influence that is rooted in new alliances among philanthropies, school leaders, and the business community. School leaders, starved for public resources, have allowed philanthropies such as the Gates Foundation to dictate school reform strategies in exchange for new private monies. Some new initiatives are worthy of support and experimentation; others are downright goofy, and school leaders should know better. But here’s the real problem, in my view: the rich, and the people they hire to administer their foundations, are different from you and me. The elite social circles in which they travel are increasingly removed from the day-to-day concerns of public school parents and students, and the educators who serve them. School districts that hire senior executives on the grounds that they know how to talk to these elites and loosen their pocketbooks are creating a divide that is increasingly difficult to cross.

I’m sure eduwonkette would join me in encouraging you to post your own top stories of 2007. So thanks, eduwonkette, for letting me guest-post for a few days, and we now return you to our regularly-scheduled programming.

Class size and "The Paper Chase"

Although I haven’t seen it in a long while, one of my favorite films is The Paper Chase, which stars Oscar-winning John Houseman as the crusty Professor Charles W. Kingsfield Jr., who terrorizes a class of first-year students at Harvard Law School. Kingsfield thunders, “You come in here with a skull full of mush and you leave thinking like a lawyer!” – a line I’ve toyed with paraphrasing for the benefit of my own students (who are not, thankfully, studying to be lawyers). The Paper Chase was fiction, but it accurately portrays how large law school classes are taught using the Socratic Method, in which the instructor engages the class by calling on individual students unexpectedly to respond to questions or to discuss legal cases. The process obliges all students to be well-prepared, and encourages them to compare what they hear other students say in class to what they might have said had they been the ones called upon instead. At the University of Chicago, the 190 entering first-year law students are enrolled in one of two sections of Civil Procedure, for an average class size of 95 students. Using the Socratic Method, an instructor can engage virtually all of the students in a class of 95 or larger.

Of course, elite law schools are highly selective institutions, filled with highly-motivated and talented students. I’m not suggesting that the Socratic Method portrayed so memorably in the Paper Chase (and in Scott Turow’s autobiographical account of his first year at Harvard Law School, One-L) is an instructional panacea. (Indeed, the University of Chicago goes out of its way to dissociate itself from that imagery. Their law school website states, “Perhaps because of its over-the-top portrayal in the 1973 movie The Paper Chase, the very mention of the Socratic Method strikes fear in the hearts of those considering attending law school. John Houseman may have won an Oscar for his impressive performance, but if anyone ever did teach a law school class like his Professor Kingsfield, no one at Chicago does today.” Methinks they doth protest too much!)

No, my point is a simpler one. We cannot understand the effect of class size on student learning without a deep understanding of the instructional strategies and techniques employed by the teacher. Most accounts of the mechanisms by which class size reduction might improve student learning depend on teachers teaching differently in smaller classes than in larger classes. The Socratic Method might work as well in a large class as in a small class; but that won't be true of all instructional approaches.

Spotlight on STAR

Yesterday’s post on class size was a bit like a projective psychological test, in that readers saw what they wanted to see. Pro-class size reduction, anti-class size reduction, it’s all in there.Today I’m spotlighting the Tennessee STAR study, which, along with the High/Scope Perry Preschool Project, is one of the two most influential education research studies of the past quarter-century. All studies have their strengths and weaknesses, and no single study constitutes the last word on what is known on a particular topic, especially one as complex as class size. Still, the strengths of the design of the STAR study have made it a star. Funded by the Tennessee General Assembly, the Student/Teacher Achievement Ratio (STAR) study was a four-year experiment begun in 1985 in which kindergarten students were randomly assigned to one of three treatment conditions: a regular class of 22 to 25 students; a regular class of 22 to 25 students with a full-time teacher’s aide; or a smaller class of 13 to 17 students. Students stayed in the same condition through the third grade. Because both students and teachers were randomly assigned to the three conditions, we can be pretty confident that the three groups were equivalent at the start of the study. Therefore, any differences in student outcomes observed among the three groups can be said to be caused by differences in the conditions.

The results were unequivocal. Students in the small class condition scored significantly higher on reading and math tests in the primary grades than students in the other two groups, and the benefits were especially pronounced for African American students and students attending inner-city schools. Follow-up studies show that these effects persist on a range of behavioral and achievement outcomes as students move through secondary school, with students in the small class condition having a lower risk of dropping out of high school, and a higher likelihood of taking college entrance exams. STAR serves as a kind of “proof of concept”: the study demonstrates persuasively that placing students in small classes in the early elementary grades can have lasting beneficial effects on a range of student outcomes.

The challenge is in applying what we’ve learned from STAR to other settings, which is usually what we are interested in doing. STAR is not a license to say that any old class size reduction initiative will have the same effects. The more closely a particular policy initiative resembles STAR, the more likely we are to observe similar findings. If, for example, you believe that teaching and learning in high school are different than teaching and learning in the early elementary grades, you might be reluctant to make any inferences from STAR about what would happen if we reduced class size in high schools, since STAR didn’t do that. And, since STAR involved a contrast between a class of 22 to 25 and a class of 13 to 17, you might not want to speculate about the consequences of reducing class size from 28 to 21, since that wasn’t actually observed in the STAR study. (In fact, there’s evidence from other studies that it’s not so much that small classes are good as it is that large classes are bad.)

But perhaps even more important is how class size reduction is achieved—the mix of what policy wonks refer to as policy instruments that define how a policy is enacted. California tells a cautionary tale. On the heels of the STAR study, California embarked on an ambitious statewide class size reduction initiative. In 1996, school districts received an additional $650 for each student in a K-3 class of 20 or smaller; the figure rose to $800 per student the following year. This was a powerful incentive that few districts could resist. Studies of the initiative show that California districts scrambled to reduce class sizes by hiring teachers with intern or emergency credentials, and many of these teachers wound up teaching in large urban schools serving poor, minority students who were English language learners. (In contrast, all of the teachers in STAR were fully certified.) Moreover, districts had to cannibalize space for small classes that otherwise would have been used for other purposes, including special education, arts and music, and athletics. Perhaps as a consequence of how class size reduction was implemented in California, researchers were unable to conclude that it had positive effects on student achievement.

I’m persuaded by the STAR study that class size reduction can lead to better student outcomes. The key questions for me are: (a) under what conditions might class size reduction work? and (b) what mix of policy instruments can create those conditions? These questions focus attention on class size reduction in particular contexts, which, in the end, is what most of us care about.

skoolboy on Class Size

skoolboy here. My thanks to Eduwonkette for ceding the bully cyberpulpit for a few days while she takes a much-earned break from blogging. I’ve never done this before, and already I’m exhausted. I know that I have a large pair of high heels to fill (ouch! I think I strained a metaphor!), but I’ll do my best not to embarrass the Caped Crusader. Since I’m a newbie, I thought I’d write about a nice, safe, noncontroversial topic – class size. I know that there are some readers of Eduwonkette who are quite passionate about class size, and may not be delighted with all that I have to say. Feel free to disagree, and, if you wish, to e-mail me personally at skoolboy2 at gmail dot com. But please, please don’t blame Eduwonkette for anything I say. She’s not responsible for my opinions, nor I for hers. And now: Watch, as skoolboy sticks his head in the lion’s mouth! Will he emerge unharmed?

As an undergraduate, I had classes that ranged from 600 students to 3 students. Although the class of 600 was wonderful—a much beloved and engaging lecturer on microeconomics—and the class of 3 was great too—a freshman seminar reading intellectual biographies of Einstein, Darwin, Freud and others—on balance, I liked smaller classes. (They could be too small, because it’s tough to hide in a class of 3 if you haven’t done the reading.)

As a college teacher, I’ve taught classes ranging from 60 students to about 8 students. Everything else being equal, I prefer teaching smaller classes. I feel like I get to know the students better in smaller classes, and there are fewer papers and exams to grade. I’m not alone. Given a choice between a larger class and a smaller class, students, teachers and parents in the U.S. almost always prefer smaller classes. Followers of school reform in New York City know that, despite the NYC Department of Education’s best efforts to disguise it, more parents surveyed chose smaller class size over nine other response options as the one improvement they would most like their school to make.

So what’s the problem? First, reducing class size is perceived as expensive. Second, class size reduction policies are often championed on the grounds that they will improve student achievement, and the evidence on this is not as secure as we’d like. I’ll have more to say about these points over the next few days. But for now, I’d like to suggest that proponents of class size reduction frame their arguments on moral grounds, rather than on what the research evidence has to say. Let’s champion smaller classes because it’s the right thing to do for teachers (e.g., providing adequate working conditions for valued public servants), and for children (e.g., distributing opportunities to learn more equitably across students), not because reducing class size will increase standardized test scores by x%.

Merry Christmas to Eduwonkette’s readers. I’m going to post tomorrow morning, and then observe the holiday in my customary way – a movie and Chinese food. For those of you celebrating Christmas in other ways, the posts will still be here on Wednesday, and I wish you a satisfying and peaceful holiday.

Twelve Days of Christmas

It's my last pre-Christmas blogging day, so I offer you the edu-version of the "Twelve Days of Christmas."

Next week, I'll be taking a blogging break from Monday-Thursday, and my colleague skoolboy will take the reigns and write about class size. I'll be back on Friday to comment on his posts.

Enjoy the holidays, everyone!



On the twelfth day of Christmas
eduwonkette gave to me
12 spinners* humming (NYC DOE)
11 schools* I'm wiping (Jim Liebman)
10 lords a leaping (Ms. Frizzle)
9 bloggers blogging (see list below)
8 classes shrinking (Leonie Haimson)
7 days of swimming (KIPP)
6 years not paying (Michael Rebell)
5 golden rings! (Roland Fryer)
4 fighting words (Whitney Tilson)
3 loaded pens (Elizabeth Green)
2 boxing gloves (Diane Ravitch)
And a session in ther-a-py. (Randi Weingarten and Joel Klein)

KEY:
1) Klein/Weingarten relations have chilled since the announcement of the "Teacher Performance Unit." See her NY Sun op-ed here.
2) After the Kathy Wylde attack, Dave Bellel offered this graphic.
3) Elizabeth Green is bringing investigative reporting back to NYC. You go, girl.
4) Whitney Tilson is angry. I am not sure why.
5) Roland Fryer is gifting ring (tones) to NYC kids.
6) Michael Rebell has been waiting for payment since a 2001 decision.
7) Lots of recent internet chatter over the KIPP trips. Some links here.
8) See Class Size Matters.
9) 9 great NYC edu-blogs: NYC Educator, Edwize, Ednotes, NYC Parents, NYC Students, insideschools, Dave Bellel, Ms. Frizzle, jd2718.
10) Ms. Frizzle is my favorite lady blogger, and per this post, she deserves many leaping lords. (Sidenote: A-Rus is my favorite manly blogger, but he is currently mad at me.)
11) 14 schools are closing, not 11 - but there are not 14 days of Xmas.
12) We actually need 15 days of Xmas here - at the Panel for Education Policy meeting, we learned that the NYC Dept of Ed press office employs 15 spinners. (5 of 15 make $115,000 or more; head spinner David Cantor makes $158,603 a year.

PS - Also check out NYC Parents' remix of "The Night Before Christmas" and edspresso's grinch poem.

Cool people you should know: Marigee Bacolod

Marigee Bacolod is a labor economist who teaches at the University of California - Irvine. She is doing really important work on teacher labor markets. Below, I excerpt three key findings from a recent paper published in Educational Evaluation and Policy Analysis, which you can find here:

• "Work conditions play a relatively more important role in determining where new teachers end up choosing to teach, rather than differences in relative teacher wages. Schools with more poor students attract significantly fewer teachers. This is especially true among female teachers.
  
  
• On the other hand, relative teacher wages play a more important role than work conditions at the occupational entry decision, when male and female new college graduates are deciding to teach.
  
  
• This paper also presents findings on the sorting of teachers across schools by ability. Conditional on choosing to teach, those with higher scholastic aptitude (in terms of SAT scores and college GPA) are significantly less likely to teach in central city schools compared to suburban schools."
  

Frogpond Effects: Age and Kids' Long-Term Academic Outcomes

Redshirting is not just for college sports anymore. As a summer NY Times article explained, affluent parents are holding their kids out of kindergarten for an extra year to give them an edge.

Does a kid's relative age - whether s/he is the big frog in a relatively younger pond, or vice versa - affect his or her long-term outcomes?

To provide insight into this question, economists Liz Cascio and Diane Whitmore Schanzenbach took advantage of the fact that kids of the same biological age were randomly assigned to classrooms via the Tennessee STAR class size experiment. As always, some kids were relatively older (or younger) than their peers. But the advantage of the STAR data is that we can be reasonably sure that parents weren't pulling strings to be sure their kid was among the eldest.

On average, relative age didn't have any effect on a key long-term outcome for kids - taking the ACT/SAT. But disadvantaged kids lost out when they were relatively younger than their peers. Free-lunch recipients who ranked among the youngest 25 percent in their kindergarten classrooms were 8.4 percentage points less likely to take the ACT or SAT.

But this doesn't mean that poor kids would be better off if they started school later. Cascio and Schanzenbach also found that the effect of absolute (biological) age varied by socioeconomic status, writing, "Disadvantaged children who are older at the start of kindergarten are less likely to take the SAT or ACT, while the opposite may be true for children from more advantaged families." They attribute poor kids' disadvantage here to the fact that advantaged parents are able to provide their kids with learning opportunities not unlike kindergarten in these extra years - either at home or in a formal setting.

The policy implications? They summed up:

These findings suggest that efforts to provide disadvantaged children with higher-quality care and education prior to kindergarten, as well as changes to state and local rules governing the age of the youngest kindergartner, could substantially affect socioeconomic gaps in educational attainment.

You can find the paper here.

4 Good Ones on NYC Report Cards

Four useful articles on report cards:

1) Diane Ravitch's NY Sun op-ed from earlier this week.
2) Leonie Haimson's post at NYC Parents, which provides links to youtube footage from the City Council hearings.
3) Norm Fruchter's Ed Week commentary.
4) Sam Freedman's NYT column.

My prior posts on report cards are archived here. Basically, here's what they say: even if your only concern is statistical validity, the NYC report cards are a mess. The system doesn't take into account measurement error, creates unreasonable peer groups, and pays no attention to regression to the mean (the idea that very high or low data points will move toward the mean the next time they are measured - i.e. Staten Island's PS 35, discussed in Randi Weingarten's commentary). And that's just a small sampling of the design problems.

What the report card debate reveals, I think, is that parents and citizens don't view school quality as a unidimensional construct. Nor do all parents want the same things from their schools; or, more accurately, they may want the same things, but put different weights on academic growth, climate, etc. Some value overall performance more than growth. Others care more about the climate and social environment than academics. Still others prioritize safety. In short, assigning one grade conflicts with our competing intuitions about how to value different dimensions of schooling.

If a central goal of the report card system is to provide information to parents, the Dept of Ed should consider assigning multiple grades. First, though, the basic design of the system needs some serious work.

Ithaca is Gorges for Teachers

I just profiled a Finger Lakes region teacher, Robb Munro - you can drive by his school on your way to Ithaca's Winter Recess (February 16-23rd). This is the only festival of which I'm aware that honors pre-K - 12 teachers and gives them a week of discounts and goodies. You can find all of the details here.

I visited Ithaca for the first time in October, so let me play travel agent and suggest some restaurants. Just a Taste, a wine and tapas bar, had delicious food and flights of local wine. And be sure to stop in to Moosewood for lunch.

7 Things You Didn't Want to Know About Me

Loony Hiker at Successful Teaching implicated me in the blog version of the chain letter. Basically, this involves sharing 7 things about yourself and passing the torch to 7 more bloggers. There was no "if you don't pass this along, all of your hair will fall out" clause, but I read way too many of these as a kid.

Though you didn' t ask, here are my 7 facts:

1) When I was a kid, I founded a Patrick Swayze Fan Club.
2) Favorite foods: Ethiopian and chocolate (not together).
3) I like things that hang from the ceiling (like mobiles).
4) I read US Weekly.
5) I love Woody Allen movies.
6) I enter the New Yorker Cartoon Caption contest every week.
7) I think there is no better place to vacation than the Jersey Shore.

I hope they won't hate me, but I am tagging:

Ed Notes Online (Norm Scott) (& response)
Ms. Frizzle (& response)
This Week in Education (Alexander Russo)
Sherman Dorn (& response)
Edudiva (& response)
PREA prez (& response)
Joanne Jacobs (& response)

Here's the deal:
- Link to the person that tagged you and post the rules on your blog.
- Share 7 random and or weird things about yourself.
- Tag 7 random people at the end of your post and include links to their blogs.
- Let each person know that they have been tagged by leaving a comment on their blog.

Lessons on Instructional Time from Mother Nature

Yesterday, USA Today championed more time in school in an editorial. States are getting behind the idea by providing grants for schools to extend instructional time (see Ed Week article here). But opt-in programs are hard to evaluate precisely because schools are choosing to participate. So how might we figure out whether and how much instructional days matter?

Dave Marcotte, an economist at the University of Maryland - Baltimore County, turned to mother nature for guidance. Marcotte reasoned that variation in winter weather made non-trivial differences in the number of instructional days before students took the Maryland School Performance Assessment Program (MSPAP) exams.

So how do kids do when snowfall decreases the number of instructional days before state tests? In Marcotte's paper, "Schooling and Test Scores: A Mother Natural Experiment," published this fall in the Economics of Education Review, we find out that that students who took exams in years with heavy snowfall performed worse than their peers in the same school who took MSPAP exams in other years. A 19.1 inch increase in a school year's snowfall is associated with a 1.2% fewer 3rd graders passing the math test. The effects of instructional time varied by subject - math scores were affected more than reading scores. They also varied by grade level - 3rd grade scores are affected more than 8th grade students' scores.

Teachers - go buy an SUV. Global warming will help increase your school's test scores.

(Image from velverse.com)

The bosses are at it again...

And they're writing very smart blog posts. Check out edwize on the NYC report cards' catch-22 - the grading system provides strong incentives to grant students course credit where credits are not due.

Over at the AFT blog, Ed sets the record straight on Siobhan Sheils' KIPP TEAM/Newark Public Schools comparison. From your friendly neighborhood broken record (i.e. me) - kids that choose into a charter school lottery may be different on their observable characteristics (prior test scores, family structure, disciplinary records, grades, special education classification, etc). With comprehensive data (which administrative data never provides, but let's say it did), we can control for these differences.

Perhaps more importantly, we must assume that those who opt into a charter lottery and those who don't are different on their "unobservable" characteristics (think parental support at home, propensity to benefit from the charter school treatment, aspirations, motivation). Here, unobservable means unobservable to the dataset we're working with, not necessarily the human eye.

Comparing kids who win the lottery with those who don't = good. Comparing charter schools and neighborhood schools = wrong.

Dallas ISD Bonuses Disappoint

$6,000 bonuses don't produce hoards of teachers dying to teach in hard to staff schools, writes the Dallas Morning News. Rick Hanushek explains why:

Eric Hanushek said most teachers won't be enticed into tough jobs for a few thousand dollars. His research determined that, for most teachers, the bonuses would have to equal about 45 percent of their base pay before they would take those jobs. If true, that means the district would need to offer more than $20,000 to draw serious interest from mid-career teachers.

"Teachers need a lot of money to move," said Dr. Hanushek, who is also a senior fellow at the conservative Hoover Institute at Stanford University. "Teachers will move, slightly, for salaries, but it's more for other factors" such as school climate, student behavior and the quality of their principals.

Further, such bonuses – sometimes called "combat pay" – tend to attract the wrong teachers, Dr. Hanushek said. "Your best teachers aren't going to take it because they have options," he said.

Private Schools (and Starbucks) in DC

Percent of K-12 Kids Enrolled in Private School in DC: 2000

Readers wanted a DC map, and soupa-prof delivered. Click to enlarge - and for fun, the Starbucks are represented by green dots.

Does the Threat of Closing Schools "Work?" A book for those who want to know.

Stocking stuffer #1/left over from last week: I just read Rick Mintrop's Schools on Probation: How Accountability Works (and Doesn't Work) and highly recommend it for those wondering how the threat of closure may play out in NYC and elsewhere.

Based on a study of 11 Maryland and Kentucky schools on probation, Mintrop, a Berkeley edu-prof, found:

Advocates of high-stakes accountability hope that the public exposure of low performance and the threat of further sanctions will move educators to increase work effort and schools to get organized and focused on student achievement. This book shows, in a nutshell, that probation had a weak motivational effect on most educators. The case is different for administrators and small groups of highly involved teachers. Teachers modestly strove to increase test scores and overcome probation primarily because of a desire to be rid of the negative label and diffuse commitment to their school, not because they expected a clear reward. Nor did they consider accountability goals as particularly meaningful orientations for their work.

Like Charlie Clotfelter et al's paper, "Do School Accountability Systems Make it More Difficult for Low Performing Schools to Attract and Retain High Quality Teachers?" (answer: yes), Mintrop looks at the ways that the label of probation pushes educators out the door. It's a useful study and a nice, if depressing, read.

Happy Holidays to all eduwonkette readers!

Click here to watch Joel, Mike, and I dance. And then visit elfyourself to turn your family and friends into dancing elves.

Update: Dave Bellel takes the challenge, elfing Jim Liebman, Randi Weingarten, Leo Casey, and Bloomberg.

This week: Stocking stuffers

The semester is over and Xmas is a week away, and I'm planning on catching up on some reading this week. Lots of interesting work came out this semester, so I'll provide summaries of some papers that we haven't heard about in the news yet.

Why NYC Private School Kids Drink Frappuccinos

Edudiva and others asked for more analysis on NYC private school attendance (prior posts here and here). Below is a map displaying the proportion of K-12 kids that go to private school by neighborhood. Thanks to my friend "soupa prof," this map makes it easier to see which NYC neighborhoods send high proportions of K-12 kids to private school. (Click on the map below to enlarge.)

NYC K-12 Private School Enrollment by Census Tract, 2000

In the map below, soupa prof also marked every Starbucks location in Manhattan with a green dot. If we adopt education policy's traditional logic - i.e., correlation equals causation - we can conclude that going to private school starts kids down the crooked path of frappuccino consumption.

Manhattan K-12 Private School Enrollment and Starbucks Locations

Or! Maybe Starbucks locate in affluent neighborhoods - as Ms. Frizzle has noted, peppermint mochas and crack are about the same price. Thanks to skoolboy, here's a graph plotting the percentage of students attending private school in a given census tract against median family income. The graph shows a remarkably strong relationship between the two; the correlation is .85.

In short, the Benjamins explain the frap addictions of our well-heeled NYC kids.

Three to Read While the Weather is Icky

1) Take the New Yorker's Rudy quiz. I'll help you with one question - former schools Chancellor Rudy Crew (and animal enthusiast?) said this about the other Rudy:

He is not bound by the truth. I have studied animal life, and their predator/prey relations are more graceful than his.

2) Two on merit pay: Prea Prez on Bushwick Community High School's rationale for rejecting merit pay and Teaching in the 408's thoughts on how merit pay could work.

3) Malcolm Gladwell's review of James Flynn's new book, What is Intelligence?

Ed Schools Eat Children, Kill Puppies

Big shoutouts to my man A-Rus for his post on Whitney Tilson's bile-laden assault on Linda-Darling Hammond. According to Tilson, "She is about as bad as it gets in terms of education reform." He sums up by writing:

I think that Linda Darling-Hammond is little more than a thinly disguised shill for the teachers unions and that her ideas, if adopted, would likely result in much higher spending and little or no improvement in our schools.

This isn't the first time Tilson has gone after ed schools. Apparently, from a screening of "Two Million Minutes" at Harvard Grad School of Ed., we can generalize about all graduate schools of education: "What a pathetic collection of Mad Hatter's Tea Parties these schools are!" (See Bob Compton's posting and Tilson's commentary here.)

Edu-profs - I guess the cat is out of the bag: Linda Darling-Hammond is heading up your coven. And her wicked rules? All ed school grads have to memorize the Communist Manifesto and stop shaving their legs. At convocation, they sacrifice an investment banker, drink his blood, and then kiss Reg Weaver's ring. And then the real Mad Hatter's Tea Party begins...

In all seriousness, it would be more productive if ed reformers learned to critique and play with a modicum of civility. (See "Attack the ideas, not the idea haver" here.)

If you want to read something more thoughtful about ed schools, try David Labaree's The Trouble with Ed Schools, which attempts to explain why ed schools are everyone's preferred punching bag. But he's part of the coven (Stanford Grad School of Ed), too, so beware.

Just Desserts and School Closings

School closings are the mystery dessert of the ed reform menu. We don't know a lot about it, but it sounds really good. Or at least different.

In looking around for evidence to include in this post, I was astounded by just how little we know about reconstitution.

By reconstitution, I refer to closing schools and reopening them in the same building with the same kids, but potentially different faculty and a different principal. It appears this is the game plan for the elementary and middle schools that will be closed. (This, of course, is different than what's happened in NYC small schools, where an extensive school choice system means that new schools reopen with different kids, different staff, and different principals.)

So what have researchers who've studied reconstituted schools concluded? The best paper that I found is by University of Maryland researchers Betty Malen and colleagues, who studied reconstitution in Maryland. They outline the assumptions of reconstitution plans, i.e.: 1) that reconstitution will bring in a more talented and committed principal and faculty, 2) that changes in the composition of the faculty supports the process of redesigning the school, and 3) that these redesigned schools ultimately improve student achievement. Malen et al. concluded that none of these assumptions held up. In particular, they found that there weren't a cadre of super teachers waiting in the wings to take jobs at the hardest schools.

Studying reconstitution in Chicago, Fred Hess came to a similar conclusion: "Reconstitution did not prove to be a successful school improvement strategy." (Hat tip to Mike Klonsky and George Schmidt for helping me understand reconstitution in Chicago.) The University of Maryland's Jennifer King Rice put it best in her article when she wrote:

Our analysis found school reconstitution to cause far more disruption than meaningful redesign of school policies and practices. Our findings suggest that policymakers should consider carefully the potential negative effects of reconstitution and whether they have sufficient resources to effectively support the policy before relying on this approach as a mechanism for school improvement. A common assumption in many low-performing school communities is that any change is better than no change at all. We would argue that such an assumption is not justified. The reconstitution initiative that we studied is a good example. School reconstitution did not enhance school capacity for reform, but rather depleted resources, leaving troubled schools more troubled than before being reconstituted.

Like jalapeno chocolate mousse, reconstitution may sound good in theory - even tempting. But as the cognoscenti have waxed about Max Brenner, not everything should be made into a dessert.

Tomorrow, I'll talk about another side of the reconstitution issue - that is, does the threat of school closure lead to improvements? - and will then take a look at some of the NYC data next week.

Where NYC's Gossip Girls (and Boys) Are

Earlier, I posted the percentage of NYC K-12 kids attending private school by Community School District, but let's get specific. From which neighborhoods do NYC's private school kids hail? Here's a fun pre-Gossip Girl teaser. Not like I watch it or anything.

You'll want to click on the image below to enlarge; darker blue regions mean that higher proportions of K-12 kids are enrolled in private school. The legend is at the bottom of the map. If you'd like a better copy of this image, email me.

NYC K-12 Private School Enrollment by Census Tract, 2000

The three census tracts in Manhattan with the highest proportions of K-12 kids enrolled in private school can be found at the following addresses. No shocker, but they're all on the Upper East Side:

1) East63rd-70th between 5th and Park : 87.6% of K-12 kids attend private school

2) East 58th-63rd between 5th and Park: 87.5%

3) East 91st-96th between 5th and Park: 86.0% (This is the census tract of the GG school, which is on 93rd, east of 5th Ave.)

(Note: These are the top three census tracts in Manhattan of those that have at least 25 kids enrolled in K-12. There are actually 5 census tracts in Manhattan where 100% of kids are enrolled in private school, but there are only small numbers of kids enrolled in K-12 in these tracts- i.e. less than 25.)

Many thanks to skoolboy for the data and to soupa prof for the map; one of these two edu-boys (skoolboy) will be making a guest blogging appearance in the not-so-distant future.

Gossip Girl Meets GIS: Private School Enrollments in NYC

Based on a series of novels by an Upper East Side private school survivor, "Gossip Girl" is a new teen drama that details the lives of the "rich and the popular" teens on the Upper East Side. It's certainly not high art, but all the hype got me wondering what proportion of NYC K-12 kids go to private school by neighborhood.

The 2000 Census includes data on school enrollment - and with the help of skoolboy (thank you!), who processed these data, here's what I found out:

• In 6 of the 32 community school districts, 25% or more of the K-12 students are enrolled in private school. These districts are marked in green below. (Sorry about the ramshackle map - no time for GIS tonight.) In Manhattan's District 2, a whopping 43% of all K-12 students are enrolled in private school. Other heavy private school districts include Manhattan's District 3 and Brooklyn's districts 14, 20, 21, and 22.
  
  
• In 11 of the 32 community school districts, 15% - 25% of the K-12 students are enrolled in private school. These districts are marked in blue below.
  
  
• For the blue/green colorblind, see the table below which lists the percentage of K-12 students enrolled in private school by district.

Percentage of K-12 students Enrolled in Private School by
Community School District
(green>=25%; blue=15.0-24.99%)

And for anyone who is curious, here is a write-up of "Gossip Girl" in the New Yorker and the trailer:

Closing Time (or, Oops! I Did It Again.)

Yesterday, NYC Schools Chancellor Joel Klein wrote an op-ed in the New York Post ("Closing Time") outlining the rationale for closing schools in NYC. Klein explained:

Starting in 2002, we began phasing out and shutting down schools that had a history of failure. These decisions...were an acknowledgement that the schools weren't remotely educating students - and that they weren't going to get better on their own.

To demonstrate the success of prior school closings, Klein provides us with the example of Bushwick High School in Brooklyn:

Bushwick HS had a graduation rate of just 23 percent. We replaced it with four new small schools, which now make up what we call the Bushwick campus. Last year, the new schools had a combined graduation rate of nearly 60 percent -almost triple what it once was. The students literally paraded through their neighborhood in June, demonstrating the pride that they feel for their schools and their community.

If the intent of school closings is to clear out the students who previously attended the "failing school," replace them with higher performing students, and declare victory, Bushwick is a marked success.

The tables below compare the incoming 9th grade students at Bushwick High School with the incoming 9th graders at the small schools that took Bushwick's place. Bushwick stopped taking 9th graders through the formal admissions process in September of 2002, but continued receiving "over the counter students" (OTCs)- students who have not been placed in any school, who are transferring, or who arrive in the middle of the year - in the 2003-2004 school year as well. Zoned schools like Bushwick represent combinations of the formal admissions process students and OTC students; while the small schools do receive OTCs, the proportion of the student population comprised by these students is much smaller.

How was the old Bushwick different from the schools that replaced it?

• The most notable differences include the ELL population and the percentage of students who come into 9th grade proficient in reading and math. Bushwick 9th graders were 30.6% ELL, while in their first year, the new small schools served between 19.5 and 26% ELL. Even more drastically, 83% of the Bushwick OTC kids were ELLs.
  
  
• On most other indicators listed in Table 1 below, the Bushwick 9th graders were lower performing than the 9th graders attending the new small schools. This is particularly true of the Bushwick OTC students.
  

Table 1. Characteristics of Incoming 9th Graders at Bushwick Campus
(click to enlarge)

• Though the small schools are supposed to serve more ELLs as they grow, Table 2 below demonstrates that they are actually serving fewer ELLs over time. By September 2005, the small school 9th graders were between 14.8 - 17.5% ELL; recall that Bushwick served 30.6%.

• Table 2 below shows that the second year classes (04-05) at the Bushwick small schools are significantly more advantaged than the first year classes. (The more disadvantaged population of the first class is a result of the small schools missing the first round of the admissions process in their first year, leaving them to choose from lower achieving students.)
  
  
• While the last Bushwick 9th graders had fewer than 10% of students proficient in both reading and math, the Harbor School had 20.2% proficient in reading and 45.5% in math in 2004; the Academy for Urban Planning 9th graders had 24.1% proficient in reading and 22% in math. Even when the percentage of proficient incoming students drops in 2005, they are still much more likely to be proficient as 9th graders than the Bushwick students were.
  
  
  Table 2. Characteristics of Incoming 9th Graders at the Bushwick Campus, September 2004 and 2005
  

This is an "oops! I did it again" moment for the Department of Education. As I wrote in a previous post, the DOE already made this mistake by declaring victory at Evander Childs in the Bronx.

Can the DOE out-oops Britney? We'll see.

If only I had my invisibility cloak...

Philissa at insideschools reports that Jim Liebman could have used an invisibility cloak at the City Council Report Card hearings today. Patrick Sullivan posts additional details, and the NYT weighs in here.

Sounds like a youtube sensation in the making. Cloaks and jokes aside, see my prior posts about NYC report cards for more serious insights (archived here).

Do Value-Added Models Add Value? A New Paper Says "Not Yet."

Value-added models are all the rage. Last week, the Gates Foundation donated 4.5 million to Houston to fund value-added measurement of teachers' effectiveness and to award bonuses based on teachers' "value-added." Similarly, NYC is developing a system to provide value-added estimates to principals to aid in tenure decisions; last year, Dept of Ed official Robert Gordon suggested that up to 25% of new teachers should be dismissed based on such estimates. In other words, districts are planning to make significant decisions based on these measures.

A formidable challenge to value-added models is the issue of non-random assignment of students to teachers. (See longer posts about this issue here and here.) If students were randomly assigned to teachers, differences in their students' performance could be cleanly attributed to something the teacher did. Accurately measuring the effects of teachers would require this type of assignment.

Of course, students are not randomly assigned to teachers. Parents do not mindlessly flip a coin and leave their child’s placement with a bad teacher up to chance; we know that principals and guidance counselors often heed parents’ wishes in the teacher placement process. Parents aside, we also know that principals non-randomly assign kids to teachers based on their sense of which teachers are good with certain kinds of kids.

My central point here is this: if assignment is non-random, some teachers will spuriously appear to be doing much better than others. And spuriousness is an ugly problem to have on your hands when teachers' incomes and jobs are in question.

Jesse Rothstein, a Princeton economist whose work I've written about before, wondered just how big of a problem non-random assignment is for value-added models. He had a clever idea that I will call the "back to the future hypothesis." Rothstein reasoned that students' future 5th grade teachers cannot have causal effects on their 4th grade achievement gains. In other words, the future should not be able to predict the past if students are randomly assigned to teachers.

In an elegant new paper, Rothstein finds that 5th grade teachers - teachers in whose classes 4th grade students have never sat - have effects on students' 4th grade gains almost as large as their actual 4th grade teachers. That's pretty good evidence of non-random assignment.

Needless to say, this is very bad news for the folks hoping that value-added models will give them an accurate and reliable measure of individual teachers' performance. Read the whole paper at the link above.

Cool teachers you should know: Toni Molnar-Port

Toni Molnar-Port is a 9th year Biology teacher at Trenton Central High School in Trenton, New Jersey. Born and raised in Trenton, Ms. Molnar-Port's commitment to her hometown led her to take a job at the high school she attended.

Her colleague reports that Ms. Molnar-Port emphasizes experimentation and laboratory work in her classroom. As a result, her students come to see themselves as "scientists in training." One of her favorite units is environmental science, in which she takes students to the local marsh to collect and test specimens of all kinds. During her summers, Ms. Molnar-Port has coordinated and led a science camp for middle school girls from Trenton and Philadelphia.

Ms. Molnar-Port is not only an expert teacher herself, writes her colleague, but "the kind of teacher that you are always grateful to have down the hall. Whether you are pulling your hair out or celebrating a small victory, she's the go-to person for support." She is known at her school for reaching out to new teachers to provide mentoring, and is currently hosting two student teachers.

Her advice to new teachers? Ms. Molnar-Port provided this excerpt from a guide for new teachers that she wrote with a colleague:

1. Build relationships — get to know your students, their families, your colleagues, and your school’s local community. Think of the time spent building these relationships as an investment, one that will pay off for both you and your students. People are experts in their own lives, and their insights may have valuable implications.

2. Search for your students’ strengths—it is very easy to make a list of things people don’t know or can’t do. Find opportunities for all students to be successful. Make sure you provide opportunities for your students to demonstrate to you what they already know and can do. Once you know this, building upon these strengths often comes naturally. Labels can help identify some students for extra support, but all too often they become the lens through which we view someone. Resist this temptation.

3. Examine your own perspectives, and understand those of others—try to look at what you teach and how you teach from someone else’s point of view. What messages are you sending, intentionally and unintentionally?

4. Include students' own experiences as a springboard to learning whenever possible; this keeps the content relevant and allows students to make important contextual connections.

5. Consider that learning to teach is a lifelong process—approach your teaching with questions that relate to your goals. Keep track of the things that you’d like to improve or change, and follow through.

6. And most of all, be kind to yourself. Allow yourself to make mistakes, and ask others to allow you to set things right when you do. Extend this courtesy to your students as well. If they came to us perfect, we’d hardly need to teach them at all.

Keep up the good work, Ms. Molnar-Port!

You can nominate a "cool teacher you should know" by emailing me at eduwonkette (at) gmail (dot) com.

This week: Closing Schools

Last week, the New York City Department of Education announced that it will close 13 schools based on the schools' report card grades. This week, I'll discuss the theory of action behind closing schools, explore the evidence on school reconstitution/closing, and analyze some of the NYC data.

Tuesday: Closing Time (or, Oops! I Did it Again.)

Thursday: The Theory & the Evidence

Coming soon: Fun with data

Friday Afternoon KIPP Caribbean Pile On

There's a blogospheric pile on regarding KIPP's private-public financing of staff retreats to the Caribbean. See A-Rus and NYC Educator for two views.

In an era where public education is struggling to justify its need for more funds, stories like this certainly don't help the cause. Even if it is "just private money," this case (and the many others I've seen like this) raises the pesky and under-explored issue of investment-return ratios in educational philanthropy.

A group that's doing really interesting work in this area is called Give Well. The organization was founded around the idea that "generosity and good intentions are nice - but not enough." They're gathering effectiveness data on many kinds of non-profits, including educational non-profits. You can check out the orgs they are reviewing here. As hybrid private-public arrangements grow in education, we should all start reading their blog - check it out here.

Kudos to Robert Pondiscio

Robert Pondiscio enriches this site with the time and care he takes to comment - so check out his article, "Mr. P Learns His Lesson," in this week's Business Week. Congrats on the new job, Robert!

On Best Practices and Learning from "No Excuses" Schools

Yesterday, someone quipped that I write longer blog posts than Gary Becker. The key difference, of course, is that he is a Nobel prize winner. So here's my attempt to keep it real and keep it short:

1) Can we learn something from "outlier schools?" As I have written in the context of KIPP, looking closely at these schools can reveal potential best practices that warrant further study. This is important and useful, so long as we resist the temptation to make strong causal claims about the impact of a given practice on achievement solely from these cases.

2) Karin Chenoweth, the author of It's Being Done (that got short shrift this week because I am admittedly only half-way through - review to follow in the near future), wrote in yesterday's comments:

Wouldn't it be worth trying to identify those practices and structures and then making sure all children--particularly poor children--have access to them? Then we could have a real discussion of the effects of poverty on learning. Right now it is impossible to separate out the effects of bad schooling on students who grow up in poverty because so many poor children are forced to attend schools that do not use the "practices and structures that increase the odds of success."

I'm all for identifying those practices and making sure that all children have access to them. Holding schools accountable for practices and structures, rather than solely measuring results, is something we should be talking much more about. Robert Pianta has a nice commentary on the need to "measure teaching" in Ed Week. On this point, we agree.

But on your second point, I think we know more about the effects of poverty on student achievement than you are acknowledging. That poverty does not have large effects on kids' achievement and life chances is a deliciously American fantasy - and one that I find frustrating to defend in light of the weight of the social science evidence on this topic. As studies analyzing data from the Early Child Longitudinal Study - Birth Cohort (a study of 14,000 children born in 2001) are beginning to show, these gaps emerge early - well before kids ever walk through the school door. Another way of isolating the impact of social class is to compare poor and advantaged kids' learning rates during the summer - where school cannot be the problem - and it's also clear that poor kids are at a disadvantage.

None of this is to deny that schools can, and must, do a lot. So like Debbie and Diane, we agree, we disagree. Can folks on both sides of this argument bridge differences at some point? We'll see.

Enjoy the weekend, everyone!

Cool People You Should Know: Doug Harris

Doug Harris is an economist who teaches at the University of Wisconsin - Madison's School of Education. He writes about a wide range of ed policy issues, so you should check out his papers and see if anything suits your fancy. He has a terrific website and most of them are available there. You can also read about his cool findings in yesterday's post.

Since Harris has a chapter you should read ("Educational Outcomes of Disadvantaged Students: From Desegregation to Accountability") in the new American Education Finance Association Handbook of Research in Education Finance and Policy, let me blurb that volume here. 40 chapters strong and including just about every education policy topic imaginable, this is the mother of all research handbooks. For those of us who complain that we can't find good literature reviews to use as course materials, the whining stops here. More details and order form at the link above.