Learning from charter school lotteries: new J-PAL policy publication

In the United States, about 2.5 million students attend one of the approximately 6,500 publicly funded charter schools. However, until recently there has been little rigorous evidence on the effectiveness of these alternatives to traditional public schools. A new policy bulletin produced by J-PAL North America reviews results from 17 evaluations in the United States that used admissions lotteries to study the impact of charter schools on student test scores and other outcomes, as well as one study that examined the impact of replicating charter school practices in traditional public schools. In addition to these evaluations, this bulletin draws upon a review by Julia Chabrier (J-PAL North America), Sarah Cohodes (Teachers College, Columbia) and Philip Oreopoulos (University of Toronto) that combined school-level data from eight of these evaluations, covering 113 schools in total.

Charter schools are public schools that are granted flexibility by state law over their operations, including staffing, finances, and curriculum, and they have used this flexibility to adopt a wide variety of educational approaches. Charter schools were originally designed as testing grounds for new and innovative approaches to improving student achievement. By law, charter schools must be open to any student residing in a given school district, region, or state. When more students apply to enter a charter school than the school has seats available, the charter school must admit students by lottery. These lotteries create opportunities for randomized evaluations.

The estimated impacts of charter schools have varied widely. In Massachusetts, lottery winners for charter schools in urban areas generally did substantially better than lottery losers, while lottery winners for charter schools in nonurban areas did about the same or somewhat worse. In general, urban charter schools have had the most positive effects on students who are least advantaged, including black and Hispanic students, students with low baseline test scores, students receiving subsidized lunch, and English language learners.

A common feature of the charter schools with the most positive effects has been the adoption of a “No Excuses” educational approach. However, it is hard to disentangle the influence of locating a charter school in a disadvantaged urban neighborhood from the impact of adopting “No Excuses” practices. Based on evidence from Massachusetts, Houston, and Chicago, one promising strategy for improving urban schools is implementing mandated, intensive tutoring.

Read the bulletin for full results and policy lessons.