Since launching the State and Local Innovation Initiative in 2015, J-PAL North America has partnered with eleven state and local governments. These partners are part of a growing movement among state and local governments to use rigorous evidence to improve the effectiveness of their policies and programs and outcomes for people experiencing poverty. Read more about our partnerships below.
J-PAL North America’s current partners in the initiative are the California Franchise Tax Board, the Minneapolis Public Housing Authority, and the New Mexico Legislative Finance Committee. Read more about our work with these partners here. These three governments join eight state and local governments selected through previous rounds of the State and Local Innovation Initiative: Baltimore, MD; King County, WA; Pennsylvania; Philadelphia, PA; Puerto Rico; Rochester, NY; Santa Clara, CA; and South Carolina.
California Franchise Tax Board
The California Franchise Tax Board is interested in using randomized evaluations to identify effective strategies for lifting low-income Californians out of poverty by encouraging eligible households to claim the state’s Earned Income Tax Credit (CalEITC). Modeled after the federal EITC, one of the nation’s largest anti-poverty programs, the CalEITC provides cash to low-income, working Californians. However, not all families who are eligible claim of the credit. The Franchise Tax Board is interested in improving outreach to eligible filers, for example by sending informational letters or personalized communications. The Franchise Tax Board is also interested in testing behavioral nudges that encourage honesty and accuracy among tax filers, which might help close the annual “tax gap”—the difference between what taxpayers owe and what they actually pay—which is estimated to be over $10 billion in California alone.
Minneapolis Public Housing Authority
The goal of Creating Moves to Opportunity in the Twin Cities is to compare the effectiveness of using tenant-based Section 8 Housing Choice Vouchers with mobility supports to using project-based vouchers to provide opportunities for low-income families with children to move to and persist in opportunity neighborhoods. In contrast to tenant-based vouchers, families awarded project-based vouchers in opportunity areas need not search for housing because the voucher is already assigned to a unit. Families on the waiting lists of partner agencies the Minneapolis Public Housing Authority and the Metropolitan Council Housing and Redevelopment Agency will be invited to participate in this study. Those who choose to participate will be randomly assigned to the tenant-based treatment group, the project-based treatment group, or the control group. Mobility supports provided to both treatment groups will include education, pre- and post-move counseling, marketability coaching, and moving assistance funds. The primary outcomes will be to determine (1) if project-based vouchers are more cost-effective than tenant-based vouchers in increasing the number of families who move to high opportunity neighborhoods, and (2) whether the type of voucher (project-based vs. tenant-based) impacts persistence in opportunity areas.
New Mexico Legislative Finance Committee
The New Mexico Legislative Finance Committee is partnering with J-PAL North America to evaluate the effectiveness of the state’s early college high schools. New Mexico high school graduates enroll in higher education above the national average but are more likely to require remedial coursework, and the state ranks among the lowest in the nation for bachelor’s degree attainment. Early college high schools, which have the mission of improving college preparedness and providing students the opportunity to obtain college credits and degrees while they are enrolled in high school, represent one potential solution to this challenge. When there are more applicants to an early college high school than there are available seats, early college high schools may use lotteries to determine admissions. By comparing outcomes for students who gained admissions through a lottery to those who did not, the Legislative Finance Committee can rigorously measure the impact of early college high schools on a variety of long-term outcome, including postsecondary degree attainment and employment outcomes.
Randomized Evaluations Funded Through the Initiative
Increasing the Take-Up of Cal Grant Awards Through Improved Notification Letters
Principal Investigator: Jesse Rothstein
Each year, over 150,000 California high school students receive letters notifying them that they qualify for Cal Grants, grant aid for college that is assigned based on family income and high school GPA. Less than two-thirds claim their awards. The California Student Aid Commission (CSAC) believes this is in part because some students do not understand their eligibility. This study will investigate whether a simplified letter, in terms of both format and the use of language that behavioral science research suggests would nudge students to take action, will induce higher take-up of the Grants and will consider how different letters impact students’ college choices decision making.
In 2017-18, students were randomly assigned (at the high school level) to receive CSAC’s usual notification letter or one of two variants designed with behavioral science principles in mind. CSAC is also working to modify its IT systems to randomly vary the number of follow-up e-mails students receive. Primary outcomes of interest include whether students create accounts in CSAC’s online portal, enroll in college, and claim their awards, as well as which type of college (community college vs. 4-year) they choose. Analysis of these outcomes will occur beginning in Spring 2018. We are working with CSAC to design higher-touch interventions to be implemented in 2018-19.
Applying Promising Findings: An Experimental Evaluation of Philadelphia WorkReady
Principal Investigator: Sara Heller
Sara Heller (University of Pennsylvania), the City of Philadelphia Mayor’s Office, and the Philadelphia Youth Network are conducting a randomized evaluation to evaluate the impact of WorkReady, a summer jobs program for disadvantaged youth. Recent evidence from random-assignment studies shows that summer jobs programs in New York City and Chicago dramatically reduce violence involvement among participants, but have small, if any, effects on education and employment. The Philadelphia study is intended to 1) assess how generalizable the prior findings are by testing the crime, employment, and school effects of a different summer jobs program in a new setting, and 2) expand tests for program effects to socially-costly correlates of violence that may also be affected: mental health, substance abuse, teen pregnancy, housing instability, and child maltreatment. To study these questions, researchers will allocate about 1,000 of the 8,000 summer WorkReady slots by lottery. Researchers will track youth in a range of administrative data, as well as collect some supplementary qualitative evidence about youth experiences. This project received full evaluation funding in Spring 2017.
Improving and Sustaining Management Practices in Public Schools: Evidence from Puerto Rico
Principal Investigators: Gustavo J. Bobonis, Marco González-Navarro, Daniela Scur
Improvements in management practices have been credited with generating stunning productivity improvement in the private sector. Although their adoption in the education sector has been slow, management-level interventions have been shown to lead to improvements in school quality and students’ academic achievement. However, adoption of these practices in the public sector is often not sustained in the long run. Promoting improvements in school management practices is thus an amenable area for policy intervention, given the need to identify ways to achieve this sustainably and at scale.
Our proposed study is a collaboration with the Puerto Rico Department of Education (PRDE) to experimentally evaluate the impact of a large-scale principal training program among its school directors. We will measure impacts on managerial practices collecting detailed longitudinal data to measure short and long-term adoption, as well as impacts on student achievement using standardized test scores data. The study will help inform whether improvements in managerial practices can be implemented in schools at scale, their long-term sustainability, and their effects on student learning.
Impact Evaluation of Bridges to Success
Principal Investigator: William Evans
Co-Investigators: David Phillips, Javier Espinosa
More than 43 million people in the United States live in poverty. Poverty is particularly pervasive in the city of Rochester, NY where rates are more than twice the national average. The Rochester-Monroe Anti-Poverty Initiative (RMAPI), in partnership with the New York Governor's State Anti-Poverty Task Force, is piloting a targeted program designed to reduce poverty in the Rochester-Monroe region. Extensive community research led RMAPI to pilot an adult mentor/navigator program that addresses the issues identified as pervasive in the city of Rochester: a knowledge gap about available services and how to navigate them, and a need for coordinated services. In order to measure the pilot's success and make decisions about scaling the program up, RMAPI has asked the Wilson Sheehan Lab for Economic Opportunities (LEO) from the University of Notre Dame to run a randomized evaluation of the pilot program, Bridges to Success (BTS). This evaluation will test the hypothesis that providing a professional mentor/navigator program in a targeted area of concentrated poverty will increase economic mobility for program participants, resulting in improved self-sufficiency. Results from this study will inform policy decisions to scale up and expand the program in the city of Rochester, and will be disseminated to policy makers and providers in other regions to inform the design or replication of future anti-poverty initiatives. This project received full evaluation funding in Spring 2017.
Leveraging the Random Assignment of Medicaid Managed Care Plans to Study Plan Choices, Treatment Effects, and Cream Skimming
Principal Investigator: Matthew Notowidigdo
Co-Investigator: Craig Garthwaite
In South Carolina, the Medicaid program is administered through Managed Care Organizations (MCOs), which offer different health care plans to Medicaid beneficiaries. These plans differ in their generosity, network coverage, and other attributes, and they are ranked by the state using a system of “star ratings.” The system of MCOs offers choices to health care consumers and allows plans to compete for consumers. In South Carolina, when consumers do not make an active plan choice, the state uses an algorithm to assign plans to consumers automatically. Starting earlier this year (in 2017), this auto-assignment is now being made using an explicitly random process. We propose to use this randomized assignment feature to study the effect of plan assignment on patient outcomes such as health care utilization and health care expenditures (both overall and by category). This prospective analysis will be complemented with a retrospective analysis that takes advantage of the state’s historical quasi-random round-robin assignment procedure to allocate households to plans. Additionally, we propose to combine the analysis of the randomly assigned population with the population that made active choices to try to distinguish between treatment and selection in accounting for which plans perform better.