Leveling the Playing Field for High School Choice: A Field Experiment of Informa
- Students
- Information
- School choice
While high school choice can have long-term implications for students’ futures, access to high quality schools is not evenly distributed among students. Furthermore, disparities in the resources and support available leaves many students having to navigate the complex high school admission process alone. In this study, researchers evaluated the impact of several informational interventions with different modes of delivery and customization levels on high school choice outcomes among middle school students in New York City. All interventions made students less likely to rank a low graduation rate high school as their first choice and less likely to apply to any low graduation rate schools. As a result, the interventions shifted students away from matching to and enrolling in low graduation rate high schools.
Policy issue
The choice of where to go for high school can have long-term effects on students’ futures. However, access to high quality high schools is not evenly distributed among students. Disparities in the resources and support available results in many students having to navigate the complex high school admission process alone. In New York City, low-scoring and low-income students apply to and enroll in schools with lower graduation rates. Providing relevant information and access to resources and technology have the potential to help students and families successfully navigate this process.
Context of the evaluation
All eighth graders in New York City participate in a high school choice process where they submit a high school application consisting of a rank-ordered list of high schools. The New York City Department of Education (NYCDOE) then uses a deferred acceptance algorithm to take into account students’ preferences and school requirements in order to assign schools.
Research has demonstrated that there are many misunderstandings about the process among school counselors, students, and their families. Some students may not take advantage of nearby high graduation rate school options, leading to matches at lower graduation rate schools and inequalities between students with correct information and guidance and students with less information. Additionally, significant differences exist between subsidized lunch recipients, non-English speaking families, and Black and Latino students and their more advantaged peers in terms of choosing and matching to higher graduation rate schools, even when academic achievement and borough are controlled for.
Details of the intervention
Researchers conducted a randomized evaluation to test the impact of informational interventions on high school choice outcomes among middle school students in New York City. The interventions differed based on their mode of delivery as well as their level of customization to students. 473 middle schools were randomly assigned to one of three intervention groups or a comparison group. The groups consisted of the following:
- Fast Facts: 247 middle schools received customized lists of recommended high schools that have graduation rates above 75 percent (the New York City median graduation rate in 2015) and some probability of admission for past students at that middle school. Within this group, schools were randomized to receive digital only delivery or a paper and digital version of the tool.
- App: 78 middle schools received a guided introduction to an online app that generated a list of recommended schools based on a student’s input.
- School Finder: 80 middle schools received a guided introduction to the NYCDOE’s online high school search tool.
- Control: 58 middle schools did not receive access to any materials designed for this study. However, students in these schools did have access to the publicly available App and School Finder.
Rather than having the study team distribute the intervention tools, school personnel (e.g., a guidance counselor) distributed the intervention tools to students in order to mimic how a school might use these tools in practice.
Results and policy lessons
Study results demonstrated that the interventions made students less likely to list a low graduation rate high school as their first choice and less likely to apply to any low graduation rate schools. As a result, all interventions (except for Fast Facts digital) shifted students away from matching to and enrolling in low graduation rate high schools.
Application Behavior
In the comparison group, 14.4 percent of students listed a low graduation rate school that they were guaranteed admission to as their first choice. All interventions reduced this rate by 2.5 to 3.3 percentage points (Fast Facts paper: 11.1 percent; Fast Fact digital: 12.6 percent; App: 11.2 percent; School Finder: 11.9 percent).
In the comparison group, 21.1 percent of students listed low graduation rate schools as their top three choices. Fast Facts paper (18 percent) and the App (18.5 percent) reduced this rate by 3.1 and 2.6 percentage points respectively.
Matched and Enrolled High Schools
38.9 percent of students in the comparison group matched to and enrolled in schools with low graduation rates. Aside from Fast Facts digital, all interventions reduced enrollment in low graduation rate high schools by between 5.1 and 6.1 percentage points (Fast Fact paper: 32.8 percent; App: 32.8 percent; School Finder: 33.8 percent ). The Fast Facts paper intervention was the most effective at shifting enrollment to high graduation rate schools. Students in this group matched to high schools with average graduation rates that were approximately 1.5 percentage points higher than the average graduation rate in the comparison group. Alternatively, Fast Facts digital did not appear to shift enrollment due to low utilization.
English learners had the strongest response to all of the interventions and impacts were the largest for students who spoke Spanish at home. High-scoring students did not tend to respond to the interventions, which suggests these students may have already had school choice plans or were aiming to enroll in exam and screened schools.
This study’s results suggest that different types of informational interventions can be successful at shifting students’ behavior in the school choice process. Policymakers and practitioners interested in using informational tools to improve student outcomes should consider how tools can be designed with different modes of delivery and customization levels to encourage successful engagement and take-up among students and school personnel. However, many students still match to lower graduation rate schools, and school districts should also consider simplifying the high school choice process. Due to the interventions, many students now attend higher graduation rate high schools than they would have without the randomized evaluation. Further research will follow these students over time to determine if those who were nudged into higher graduation rate schools are more likely to succeed and graduate from high school.