The Effect of School Choice on Student Outcomes: Evidence from Randomized Lotteries
- Secondary schools
- Urban population
- Arrests and convictions
- Dropout and graduation
- Student learning
- School choice
School choice programs offer students more opportunities to attend schools with higher achievement and graduation levels. In partnership with the Chicago Public Schools, researchers examined the effects of a school choice program on several traditional student achievement outcomes. They used a randomized lottery to induce school choice and found no measurable difference in traditional education outcomes for students who attended better schools. However, there were some improvements in nonacademic outcomes, such as self-reported disciplinary action and arrests.
School choice programs, which offer students increased opportunities to attend schools with higher achievement and graduation levels outside their neighborhoods, have become a popular strategy for improving academic achievement. Theory suggests that students should fare better in higher achieving schools because of access to more resources, accomplished peers, or programs that better suit their learning needs. However, there is little evidence of a relationship between access to these sought-after schools and student outcomes. Moreover, there is no strong or consistent relationship between variations in school resources and student performance. Evidence suggests that adding resources, without changing the way schools are managed and incentivized, is neither necessary nor sufficient for improving student outcomes. The literature to date shows mixed results with regard to policymakers’ ability to influence educational outcomes by altering the set of inputs to the educational process.
Context of the evaluation
The Chicago Public Schools (CPS) system has one of the most extensive school choice programs available in the United States. CPS use an open enrollment system in which each student is guaranteed admission to an assigned neighborhood school, but can also apply to any other CPS school outside of their neighborhood but within the same school district. In fact, more than half of all high school students in CPS in 2000 and 2001 elected to attend a school other than their assigned school. In order to attend an unassigned school, students must apply to that school in the spring of the preceding school year. When the number of applications for a school exceeds the number of available positions, randomized lotteries are used to allocate students to the school. Lotteries are computerized and ensure that all children have an equal chance of being accepted at schools that do not have admissions requirements.
Details of the intervention
Researchers, together with CPS, examined the effect of school choice on several traditional measures of achievement including standardized test scores, graduation rates, attendance rates, course taking patterns, and credit accumulation. Researchers used the lottery-induced randomization to create and compare two groups of students: students who won the lottery and students who lost. They studied the difference in student outcomes between the groups to determine whether winning the lottery provides any systematic benefits to students. Students were allowed to apply to multiple lotteries and the overall sample included 19,520 applications from 14,434 students at 19 different high schools.
CPS provided detailed administrative data on the randomized lottery applications submitted in spring 2000 and spring 2001. Researchers also obtained standard achievement and attainment measures, as well as student survey responses that covered students’ degree of satisfaction with the school attended, how they were treated by teachers and peers, expectations about college attendance, and self-reported arrest data.
In addition to educational outcomes, researchers looked at the effect of winning the lottery on a set of student attitudes and behaviors such as enjoyment of school, behavior of peers, student–teacher trust, expectations for the future, and self-reported disciplinary incidents.
Results and policy lessons
In general, students who won lotteries to the most select programs did attend what appear to be substantially better high schools; e.g., schools with higher achievement levels (and gains), higher graduation rates, and lower poverty rates. Winning a lottery to a high popularity school raised the share of peers who tested at or above national norms, reduced the fraction of free-lunch-eligible peers (a proxy measure for poverty), and raised the graduation rate of peers. The schools that lottery winners attended were located in higher income and lower crime neighborhoods. The study focused on eighth grade students applying for ninth grade admission, which corresponds to the transition period from primary to secondary school in CPS.
Four-year graduation rates from CPS were lower for the lottery winners by 4.4 percentage points (compared to a baseline rate of 58.3 percent). However, the size of the effect depended on the type of school attended, and there were no significant differences in graduation rates at better ranked schools. Additionally, lottery winners showed no improvement on five of the six measured test outcomes and performed slightly worse on reading scores by 1.3 percentile points. Looking at relative performance, students who won lotteries had worse class ranks by 2 percentile points (or 6.9 percentile points for better ranked schools), likely reflecting a more competitive peer group. Looking at specific race and gender groups, there is no evidence that any subgroup derived any substantial academic benefit from winning a lottery to a choice school.
While the lotteries did not improve traditional measures of academic achievement, they did affect certain nonacademic outcomes among students: winners reported fewer incidents of disciplinary action, fewer arrests, and lower incarceration rates. Students who won lotteries to high-achieving schools reported an 8.7 percentage point drop in disciplinary action. Self-reported arrest rates were reduced by 5.1 percentage points at high-achieving schools, a 60 percent decrease from a base of 9 percent.
Overall, the results of this study align with other recent work on the absence of a positive impact of public school choice and high-quality peers. The results reinforce a large body of prior work showing that school inputs have little impact on student outcomes. The evidence suggests that a student’s relative position among his or her peers may be an important factor in determining academic success.