Evidence Wrap-Up: Evaluations in North America
J-PAL North America’s “Evidence Wrap-Up” summarizes research by J-PAL affiliates in the region, in sectors ranging from crime prevention to health care delivery. J-PAL affiliated researchers have conducted more than 100 ongoing or completed randomized evaluations in North America to answer critical policy questions.
Compiled August 2015
J-PAL has launched a new Crime Program to gain a better understanding of crime and criminal justice in the United States and abroad. The program will promote the use of randomized evaluations to study interventions to both prevent crimes and improve the criminal justice system in the most cost-effective and humane means possible. Cognitive behavioral therapy programs implemented in schools and a juvenile detention center, studied by the University of Chicago Crime Lab, have reduced violent crime arrests, dropout rates, and recidivism. In another study, a re-entry program that provided social services prior to release and subsidized work after release reduced the likelihood of rearrest.
Discrimination and Hiring Practices
Research by J-PAL affiliates has deepened understanding of modern discrimination. In Boston and Chicago, resumes sent with African-American-sounding names received fewer callbacks from employers than those with White-sounding names. In Toronto and Montreal, employers discriminated against resumes with foreign-sounding names. In another resume study, researchers found that resumes with a bachelor’s degree from a for-profit “online” institution were less likely to receive a callback than a similar degree from a non-selective public institution.
Improving the quality of education can meaningfully improve the lives of disadvantaged youth. J-PAL affiliates study interventions aimed at improving student learning, college enrollment and persistence, and adult earnings.
Smaller class sizes, more experienced teachers, and higher quality classroom in early childhood grades can have positive impacts on adult outcomes, such as earnings and college attendance, even though benefits to test scores fade out over time. Access to higher-quality schools has improved student learning and adult outcomes in some contexts. High-performing charter schools in Massachusetts and New York City have been shown to improve test scores and other outcomes of poor urban students, and replicating charter school best practices in traditional Houston public schools increased students’ math scores. In Chicago, students winning school choice lotteries attended higher quality elementary and high schools but did not perform better academically. In a North Carolina school choice lottery, parents with different social-economic statuses had different preferences for types of schools with families of white females most likely to choose academic-oriented schools, leading to higher test scores only for white females and increased college attainment for girls but not for boys.
Financial incentives have been proposed as a possible means to promote student achievement, but they must be carefully designed. Offering incentives to students for achieving math objectives increased math scores for higher-performing students, but decreased reading scores for lower-performing students. Other student incentives have had little to no significant effects. A financial incentives system for teachers in New York City had no effects on student achievement, but another in Chicago found increases in student achievement when money was provided to teachers in advance and withdrawn if teachers did not sufficiently improve student achievement.
In Chicago, providing intensive, individualized math tutoring to disadvantaged high school students improved math grades, raised math test scores, and reduced course failures in both math and non-math classes.
Providing information and personalized assistance to high school and college students can increase knowledge and perceptions about college, enrollment, and persistence. Helping low-income families complete the FAFSA financial aid forms and giving them personalized information about college costs during visits with a tax consultant increased financial aid applications, college attendance, and college persistence. Financial aid in Nebraska increased college enrollment and persistence, while also shifting many applicants from two- to four-year schools. These effects were strongest among demographic groups with historically-low college attendance. In New Hampshire, providing high school students with one-on-one college coaching and a cash incentive for completing the college application process increased college enrollment for women with small to no effects for men. For students in their first year of college in Canada, offering a combination of academic support services and financial incentives improved academic performance for women, but not men. Another individualized student coaching program, for mostly non-traditional college students, increased persistence and graduation rates for both men and women.
Employment and Training
New York City’s summer jobs program increased participants’ earnings during the year of the program but actually decreased earnings in subsequent years. However, the program was effective at helping youth stay out of trouble, reducing criminal involvement and even deaths. A summer jobs program in Chicago similarly decreased violence among program participants.
A subsidized entrepreneurship training program designed and implemented by the U.S. Department of Labor and the U.S. Small Business Association increased business ownership and employment in the short-run, but the effect faded over time. Evidence from an online labor market suggests that helping individuals obtain a first job can improve their future employment prospects.
J-PAL affiliates are using randomized evaluations to study the most effective means of reducing energy consumption. For example, home energy reports persuaded households to reduce energy consumption in the short and long-run. In another study, consumers who were charged real-time pricing for electricity conserved energy during peak hours when prices were higher. A separate study in Michigan of the Weatherization Assistance Program found that households were reluctant to weatherize homes, even with no monetary costs and that engineering estimates substantially overestimated the energy savings that would results from weatherizing homes.
Health Care Delivery
J-PAL North America’s Health Care Delivery Initiative tests how to make health care delivery in the United States more efficient and effective. Researchers studying a randomized lottery which expanded Medicaid to uninsured individuals in Oregon found that Medicaid increased health care use, including both preventive care and emergency room visits. It reduced measures of financial hardship, reduced rates of depression and improved self-reported health. Medicaid coverage did not have statistically significant effects on physical health measures, employment, or earnings. In another study, providing personalized information to seniors about different Medicare Part D options caused patients to cut costs by switching plans. An evaluation in a large hospital found that the costs for patients randomly assigned to teams of physicians from a higher-ranked medical school were lower than the costs of those assigned to teams from lower-ranked schools, while health outcomes were similar between both groups.
Individuals often misunderstand or are misinformed about how to manage debt and increase savings. Improving financial literacy could potentially help people improve their financial decisions, reduce their vulnerability to financial emergencies, and better prepare for retirement. In one study, straightforward information about the dollar costs of loans decreased payday borrowing.
A significant share of low and middle-income American families have inadequate savings for retirement. When employees were incentivized to attend a workplace benefits fair, individuals increased their participation in Tax Deferred Account retirement plans. Savings also increased among their colleagues who did not attend the fair. Giving older adults information about key Social Security provisions, through an informational mailing and web-based tutorial, caused participants to delay retirement.
Housing Mobility and Neighborhood Effects
The Moving to Opportunity (MTO) project, a major randomized housing mobility experiment, demonstrated that neighborhoods matter to the health and well-being of residents. Through the MTO project, the U.S. Department of Housing and Urban Development provided families living in public housing in disadvantaged neighborhoods housing vouchers to move to low-poverty neighborhoods. These vouchers succeeded in helping families live in lower-poverty, safer neighborhoods. Children under 13 at the time of the move were more likely to attend college and earned more income as adults. Older children had slightly negative outcomes, possibly because the disruption of moving outweighs the benefits of a few years in a lower-poverty neighborhood for older children. Adults saw large benefits to their physical and mental health, but moving had no effect on employment or income.
In another housing voucher program in Chicago, receiving a voucher caused families to move to less distressed neighborhoods and decreased mortality rates for girls. Receiving a voucher increased mortality rates for boys, and in particular homicides and accidental deaths, consistent with the medium-term MTO finding that boys were more likely to be injured or engage in other problem behaviors after moving. Unlike the vouchers in MTO, which were given to families living in public housing, the Chicago housing vouchers were primarily given to those already living in private housing, so vouchers increased both their income and their marginal tax rates. Among working-age, able-bodied adults, using a voucher reduced labor force participation and earnings.
For decades, voter turnout has been far lower in the United States than in other developed democracies. Both partisan campaigns and non-partisan organizations use many strategies to attempt to engage voters. In a large randomized evaluation in New Haven, Connecticut, personal canvassing had a much greater effect on increasing turnout than phone calls or direct mail. A different study of an election in Michigan found that direct mail can increase turnout by about as much as canvassing when the mailings apply social pressure through making neighborhood voting history public.
Contact: J-PAL North America Policy Associate Graham Simpson.