Government partnerships for the design of evidence-based policies in education in Latin America
Despite large advances in school enrollment over the last decade, approximately 262 million children, adolescents, and youth between the ages of 6 and 17 are still out of school, according to UNESCO. In Latin America and elsewhere, school dropout is a key obstacle to universal school attendance: in the region, 15.5 percent of students dropped out during their time in primary school in 2016.
How do policymakers begin to tackle such a large and complex issue? And, if policymakers do find a possible solution, how do they know it can successfully and cost-effectively work when implemented at scale?
Long-term partnerships between governments and researchers, based on the use and institutionalization of rigorous evidence, may be the answer.
On December 3, 2018, education policymakers and researchers from the Dominican Republic, Chile, Colombia, Peru, and the United States joined representatives from the Inter-American Development Bank, Teach for All, and Inicia Educación to discuss strategies for establishing these types of partnerships. Co-sponsored by J-PAL’s Latin America & the Caribbean (LAC) office, together with the Dominican Institute for Educational Quality Research and Evaluation (IDEICE) and Princeton University, the event highlighted the successes of education partnerships between governments and researchers throughout Latin America. Successful examples included the evaluation and scale-up of information dissemination policies in education, as well as the strengthening of government data centers and impact evaluation labs in Chile, the Dominican Republic, and Peru.
Implementing evidence-based education policies at scale
National-level education programs that aim to reduce school dropout by providing information to students about the returns to staying in school have proved promising in some contexts.
In the Dominican Republic, J-PAL affiliated researchers James Berry (University of Delaware) and Christopher Neilson (Princeton University), together with researchers Lucas Coffman (Harvard University) and Daniel Morales (IDEICE), evaluated a program that transmits information about the returns to education to middle-school students through the use of different forms of media, including telenovela-style videos, posters, and tablet-based information campaigns.
Preliminary results suggest the program reduced school dropout rates by between 2.5 and 3 percentage points after a year—resulting in approximately 6,500 more students staying in school. In addition, for children who received the information, it improved schooling outcomes in national examination scores by between 0.05 and 0.12 standard deviations.
Similarly, in Peru, J-PAL affiliates Francisco Gallego (Pontificia Universidad Católica de Chile), and Neilson, together with Oswaldo Molina (Universidad del Pacífico), collaborated with Innovations for Poverty Action (IPA) and the Ministry of Education’s MineduLAB to study the effects of the “Choosing a Better Future (DFM)” program. DFM also spread information about returns to schooling through videos, in addition to including more intensive, in-person information delivery to students and their parents.
Consistent with the preliminary results from the Dominican Republic, DFM decreased school dropout rates by 18.8 percent in urban areas and corrected students’ and parents’ perceptions about the returns to education. Furthermore, the intervention reduced child labor by 15 percent for girls in urban areas.
Both programs were found to be cost-effective—the cost of videos per students in Peru was only US$0.05, and increases in national examination scores in the Dominican Republic cost between US$78.5 and US$124.4 per standard deviation. Given their impact and cost-effectiveness, both programs are respectively being scaled-up by the Peruvian and Dominican governments to ensure that all public schools are reached.
Institutionalizing evidence use
Beyond these policies, partnerships between government and academia have resulted in the promotion and institutionalization of administrative data use within governments.
During the event, Juan Ariel Jimenez (Vice Minister of Development Policy of the Ministry of the Presidency, Dominican Republic), Juan Pablo Silva (former Vice Minister of the Ministry of Education in Peru), Ryan Cooper (DIPRES Impact Evaluation Coordinator, Chile), and Elianny Medina (Cabinet for Social Policy Coordination, Dominican Republic) discussed ongoing efforts throughout the region to embed innovation and impact evaluation labs and administrative data centers within governments. These units include Peru’s MineduLAB—which pilots and evaluates the effectiveness of a number of innovations in education policy—as well as the Gabinete de Coordinación de Politícas Sociales (GCPS) evidence lab and a Data Center of the State in the Dominican Republic, which harmonize administrative data across government bodies and generate evidence for social policies.
The gathering in Santo Domingo was a concrete reminder of how long-term partnerships between government and researchers can yield substantial results; either in the form of cost-effective and scalable policies that are proven to be effective in a local context or in the increased use of administrative data to ensure that policies are grounded in evidence. Such data-oriented initiatives are examples of J-PAL LAC’s long-term efforts to forge and build evidence-based government partnerships to improve the effectiveness of programs in education and more, across the continent.
Authored By
It’s easy to talk about increasing the use of evidence in policy—but what does it take to do this in practice? What are some first steps governments can take to build a culture of using data and evidence in decision-making? What role can researchers and practitioners play in supporting these efforts?
Our new J-PAL report, “Creating a Culture of Evidence Use: Lessons from J-PAL’s Government Partnerships in Latin America,” tackles these and other pressing questions to shed light on the many paths from evidence to improved policy in government.
Since J-PAL was founded in 2003, we have built long-term partnerships with government agencies in more than fifteen countries to generate and use evidence from randomized evaluations in policy decisions. Our regional office in Latin America and the Caribbean (J-PAL LAC) has been at the forefront of these efforts, working with policymakers throughout the region to improve the effectiveness of their programs.
“Creating a Culture of Evidence Use,” funded by J-PAL’s Government Partnership Initiative, shares examples of J-PAL LAC’s government partnerships and features insights from interviews with forty officials in fifteen of our partner agencies across the region, as well as staff from organizations like Innovations for Poverty Action (IPA) and the Climate Policy Initiative (CPI) who have led some of these partnerships with us.
It details the challenges our government partners have faced in making the case for greater data and evidence use and how we have tried to help them overcome some of these challenges through our Evidence to Policy partnerships.
Figure 1: The Learning Cycle
All of J-PAL LAC’s partnerships are grounded in a conceptual model called the Learning Cycle, which has three phases: diagnosis, evaluation, and evidence use and learning. Underlying the Learning Cycle are three core competencies that enable governments to use data and evidence: technical capacity, institutional processes, and administrative data.
Building on our experiences and insights from our government partners, we provide practical tips for evidence-to-policy organizations on how to start new government partnerships and sustain them. We also highlight what we see as the most promising areas for governments to invest in to make evidence-informed policymaking more common throughout the region:
| Key Insights for Governments | Key Insights for Evidence to Policy Organizations |
| 1. It is important to allocate resources to evidence use, and to make it someone’s job to apply evidence in policy design. | 1. Proactive support from a senior government official is crucial. |
| 2. Similarly, creating dedicated spaces, like innovation labs or funds, where evidence use is encouraged and rewarded can help build a culture of evidence-informed innovation. | 2. Increasing data and evidence use is an incremental process. It is important to meet governments where they are in their data and evidence use practices. |
| 3. Greater investment in administrative data collection, management, and inter-agency data sharing can help build a foundation for long-term, systems-wide evidence use. | 3. Different types of government agencies may require different approaches depending on whether they implement, finance, and/or evaluate social programs. Be ready to adapt. |
| 4. Collaborating with evidence-to-policy organizations and researchers can provide access to reliable, customized technical support and guidance. | 4. Respond quickly to opportunities and policy windows when there is interest and authority to use evidence.1 |
| 5. Fostering broader, institution-wide support beyond key champions through on-demand data analysis or capacity building is crucial for sustainability. | |
| 6. Investing in and formalizing long-term partnerships can build greater trust and understanding, create repeated opportunities for learning, and make it easier to spot and respond to policy windows. |
Governments are some of the most important actors in reducing poverty and inequality worldwide, providing vital public services from education to health care and social assistance. For this reason, finding ways to forge partnerships that can strengthen government capacity to deliver better public services is critical for solving the world’s most pressing challenges.
The examples of government partnerships featured in “Creating a Culture of Evidence Use” just scratch the surface of a much larger movement among governments in Latin America and around the world to use data and evidence in decision-making. We hope that sharing our experiences will inspire more governments to move in this direction, and more researchers and practitioners to collaborate with governments to improve social policy.
View and download the report >>
We are always looking for new partners in these efforts. If you would like to learn more about J-PAL’s government partnerships, or if you would like to share feedback on this report, please email [email protected].
1Kingdon, J. W. 1995. Agendas, alternatives and public policies. HarperCollins College Publishers, 1995. Andrews, Matthew, Jesse McConnell, and Alison O. Wescott. 2010. "Development as Leadership-led Change-A Report for the Global Leadership Initiative and the World Bank Institute (WBI)."
One reason children from disadvantaged backgrounds receive less schooling and join the labor force at younger ages with fewer skills may be that they and their families lack crucial information needed to make the right long-run investment decisions regarding their human capital. In Peru, IPA and J-PAL worked with researchers and the Ministry of Education to evaluate at scale two low-cost ways of providing relevant information to help students and their families make more informed decisions. Using a series of telenovela-style videos screened as part of the curriculum in schools as well as through an interactive tablet app, the research project evaluated how information provided at different ages could shape human capital decisions. Results suggest that the programs were effective at changing educational plans and lowering dropout rates, while significant effects on child labor were mixed. The policy has now been adopted by the government and scaled up to 100 percent of public schools with full class days.
Policy issue
In many countries, students drop out of school and enter the labor force as children at persistently high rates. One reason for this may be that students and their parents are making their educational decisions without the appropriate information. They may underestimate the long-term benefits of continuing their education. They may also not understand the differences in types of education tracks or fields of study. Importantly, they may not be aware of financial aid programs that would make continuing education feasible for them. This can lead to a series of mistaken decisions about questions like how long to stay in school, how hard to try, and what types of subjects to focus on. These mistakes may accumulate and compound over time, making exiting the system and working with less skills more likely and reproducing poverty. In contexts where students and parents are less informed, previous research suggests that providing students and their parents with accurate information can shift their education decisions (see, for example, evaluations in Chile and the Dominican Republic). However, less is known about whether cost-effective ways to share relevant information as a policy implemented by the government can work at scale.
Context of the evaluation
In Peru, despite recent improvements in the coverage and quality of the education system, high school dropout rates are a significant problem. At the national level, 12 percent of children leave school before age 13, and 17 percent do not finish secondary school.1 Child labor in Peru is common as well. Figures from the ILO and the Ministry of Labor show that 21.8 percent of children aged 5-17 in Peru were working in 2015. In rural areas, the child labor rate was almost four times higher than in urban areas.
The initial survey conducted for this study found that students and parents underestimated the earnings of graduates of different fields of study in both technical and college tracks. For example, the median child participating in the survey expected a university graduate to earn 31 percent less than the real average wages for a university graduate in 2015. Students and parents were also often unaware of important financial aid programs such as Beca18, which provides scholarships for poor students to attend universities.
Watch the below videos for an overview of the evaluation and student and parent testimonials.
Details of the intervention
Researchers partnered with Peru’s Ministry of Education to conduct a randomized evaluation measuring the impact of delivering information on the returns to education through video and an interactive tablet app on school dropout and child labor rates. The information campaigns provided information about the financial and social benefits of education to improve knowledge about the returns to education.
This study involved two different interventions:
- Video series: This program comprised a telenovela-style video series whose plot conveyed messages about the social value of education, real earnings information for different education levels and fields, and options for financing higher education. Students watched these videos in their schools as part of the curriculum.
- Tablet application (“app”)-based intervention: This program delivered similar messaging through a more intensive, tablet-based information campaign, built into an app-based survey which used infographics, interactive activities, and in-depth presentations to present information to students and parents. Some students interacted with the tablets in their homes, and others at their schools.
In the first year of the intervention, the evaluation design differed slightly between urban and rural areas:
- In urban areas, researchers randomly assigned 2,611 schools to either receive the video campaign or serve as part of the comparison group. The sample who received the program included half of all the urban schools in Peru. Within the schools in urban Lima that were part of this sample, researchers randomly selected 3,334 students and 1,816 parents to either receive the application-based intervention or serve as part of the comparison group.
- In rural areas, researchers randomly assigned 249 schools to either receive the policy pilot video campaign (125 schools) or serve as part of the comparison group (124 schools). In these groups, researchers randomly selected 3,000 primary students and 993 parents to either receive the app-based intervention or serve as part of the comparison group.
In the second year of the evaluation, both interventions were scaled back due to budget constraints and to focus on improving implementation, and researchers focused on in-depth follow-up to identify mechanisms underlying the evaluation’s results.
Results and policy lessons
Overall, results suggest that the information provision policy implemented by the government at scale via videos affected human capital accumulation of students, and seemed to help reduce school dropout rates in particular. The impact on child labor was mixed, with some groups such as women in urban areas lowering the amount of work provided at home.
Providing information to students increased perceptions of the returns to education: Students in urban areas who received the video series intervention expected returns to university education that were 8 percent higher than those of students in the comparison group, closer to the actual benefits to study. However, in rural areas, the policy pilot did not have significant effects. Both students and parents who received the app-based intervention had increased perceptions.
The treatment increased the perceived feasibility of pursuing higher education and families updated their long-term education plans: The perceived feasibility of achieving higher education increased for both parents and students in urban areas right after the app information was delivered. Treated households changed long-run educational plans: in addition to updating their beliefs about the long run educational plans, children and parents were 10 percent more likely to improve their educational plans right after receiving new information through the app in both urban and rural areas. As students updated their beliefs about the returns to higher education, they began to consider finishing higher levels of education, and parents tended to be more willing to support their children in this pursuit.
Dropout rates fell as a result of the video series in both rural and urban areas: In urban areas, the video series led to a decrease in the two-year dropout rate of 1.8 percentage points—equivalent to a decrease in dropout rates of 18.8 percent. In both urban and rural areas, the effect was driven largely by the behavior of boys. The one-year effects were largest for fifth- and sixth-graders, and the two-year effects were larger for younger children than older ones.
Effects on child labor from both the video series and app were mixed: Neither intervention reduced child labor overall. However, the video series reduced child labor for girls in urban areas by 15 percent. For children in urban areas who worked at the beginning of the intervention, the video series increased average work hours by 2.2 hours per day. In rural areas, the policy pilot reduced work hours for certain groups—particularly boys and sixth graders. Additionally, the application-based intervention reduced child labor for 6th graders in rural areas by 7 percent, but not for the overall sample.
Ministry of Education, Peru. 2015. SAIGIE 2015.
International Labour Organization and Ministry of Labor, Peru. 2015. Magnitud y Características del Trabajo Infantil en el Perú, Informe 2015.
How important are the returns to education in determining schooling decisions? Do students have accurate information about these returns when they choose whether to continue schooling? In partnership with the Ministry of Education in the Dominican Republic, researchers are evaluating the impact of informational videos about the benefits of education on the decision of students to invest in additional schooling. Preliminary results suggest that exposure to the videos lead to a decrease in dropout for 8th grade students.
Policy issue
How important are the returns to education in determining schooling decisions? Do students have accurate information about these returns when they choose whether to continue schooling?
Many people make the decision to go to school based on what they perceive to be the monetary (e.g., higher income) and nonmonetary (e.g., health, quality of life) returns to education. However, these perceptions may be inaccurate, causing people to under-invest in education. This could be particularly true for the poorest families, who may have fewer well-educated acquaintances to serve as examples for the returns to schooling. If people are in fact underestimating the returns to education, then it may be possible to increase their demand for schooling by simply informing them of the actual increase in earnings and other benefits they could see with additional years of education.
Previous research in the Dominican Republic suggests that providing this type of information is a cost-effective means of increasing investments in schooling. The main objective of this project is to further develop this model and evaluate a scalable, cost-effective means of delivering information on education to students in the Dominican Republic in order to help diminish the high dropout rates among students.
Context of the evaluation
While more than 90 percent of children complete primary school in the Dominican Republic, only about half of students who graduate from eighth grade will go on to reach the end of secondary school. This contrasts significantly with the more than 80 percent of students in the study sample who indicate that they plan to go to college. Moreover, a 2015 survey by the Central Bank of the Dominican Republic shows that the long-term benefits of education are quite high: secondary school graduates earn more than 30 percent higher wages than their counterparts who only finished primary school, and university graduate earn more than double than their counterparts with only a secondary degree. While it is possible that the costs of schooling are high enough to outweigh even these benefits, boys in the sample tend to underestimate the average earnings of high school and college graduates, suggesting that information on earnings could have a large impact on this sample.
Details of the intervention
In partnership with the Ministry of Education in the Dominican Republic, researchers are evaluating the impact of an informational video campaign about the benefits of education on the decision of 7th and 8th grade students to continue schooling. In early 2015, researchers randomly assigned 600 public schools with 43,000 students to the following three groups:
- “Persuasive” videos (200 schools): Students were shown four 15-minute videos with 8th grade characters considering future education choice and discussing the qualitative benefits to having more education, as well as the opportunities to apply for education scholarship funding.
- “Informative/Statistical” videos (200 schools): Students were shown the same four videos, with additional segments that provided detailed statistical information on wages at different levels of schooling.
- Comparison (200 schools): Students received no information or videos.
Students who were shown videos also had posters displayed in their classrooms that illustrated the same types of information shown in the videos. In order to examine parental decisions and how the information is perceived when presented in more interactive and individualized way, a randomly selected subset of the students and their parents, researchers provided more detailed information on wages and education using tablet computers.
Researchers will collect enrollment data from the Ministry of Education to measure effects on schooling decisions, as well as conduct an in depth survey of students, parents, teachers, and school principals to analyze intermediate outcomes such as barriers to and feasibility of different education options, beliefs about the returns to schooling, and students’ education plans. If proven effective and cost-effective, the project will be extended nation-wide to all public schools in the Dominican Republic.
Results and policy lessons
Project ongoing, results forthcoming.