GPI Funded Projects

PDF version

Projects Funded During the Sixth Round (Q2 2017)

Data Science for Policy Impact Initiative at J-PAL South Asia

Researchers/Office: J-PAL South Asia
Government Partner: Government of Tamil Nadu
Location: India
Type: Institutionalizing use of evidence (Type 3)

GPI provided seed funding to set up and build an interdisciplinary Data Science for Policy Impact initiative within J-PAL South Asia. The overarching goal of this initiative is to support strategic government partners in systematically analyzing large volumes of administrative and survey data (and increasingly using new digital technologies) to inform policy research and decision-making, and in the process improve development outcomes. A multi-pronged approach will be used to achieve this goal. First, the initiative plans to help launch proof-of-concept analytics projects and randomized evaluations to generate high-quality, actionable information for government partners. Second, the initiative will allow J-PAL staff and affiliates to engage more deeply with government partners to help them institutionalize a data-driven approach to decision-making. Finally, the initiative will leverage J-PAL expertise to build state capacity to generate, maintain, and use high-quality data, and to run iterative analytics on an ongoing basis.

Continued Partnership for the Use of Evidence to Inform Youth Employment and Skills Development Programs in Côte d’Ivoire

Researchers/Office: Bruno Crépon, Patrick Premand, IPA Côte d’Ivoire  
Government Partner: 
Youth Employment and Skills Development Project (PEJEDEC), Office for the Coordination of Youth Employment Programs (BCPE)
Location: 
Côte d’Ivoire
Type: 
Institutionalizing use of evidence (Type 3)

Youth employment and skills development is a key concern for Côte d’Ivoire in its race to achieve emergence by 2030. Over the last five years, the Government of Côte d’Ivoire, the World Bank, and independent researchers have maintained a close partnership aimed at developing innovative programs to tackle youth unemployment and building rigorous evidence to improve them. Two evaluations have been conducted so far and a third one launched recently. The goal is now to ensure the effective translation of this evidence into policy decisions. The government is eager to build the evaluation results into their decision-making process. GPI funding will be used to provide technical assistance and capacity-building to increase the potential policy impact of this research, including fostering the discussion of specific potential policy actions, optimizing the use of the extensive data collected through the evaluations and the government administrative data, and conducting a cost-effectiveness comparison of similar programs implemented in the region.

Taxing with Evidence: Capacity Building and Pilot Experiments with the Kenya Revenue Authority

Researchers/Office: Lorenzo CasaburiDina Pomeranz, IPA Kenya, J-PAL Africa  
Government Partner: 
Kenya Revenue Authority
Location: 
Kenya
Type: 
Institutionalizing use of evidence (Type 3)

Increasing tax collection has become a key priority of developing countries aiming to increase economic development. In Kenya, “broadening the tax base through an enhanced taxpayer recruitment effort” constitutes the first pillar of the Kenya Revenue Authority (KRA)’s Strategic Plan. In order to support this ambitious goal, researchers are working with IPA Kenya, J-PAL Africa, and KRA in a two-pronged project. First, they will work to build KRA’s capacity to use existing evidence and rigorously test alternative policy interventions. Second, they will design and pilot three experimental interventions to investigate whether improving the match between the cash flow and the frequency of tax payments increases compliance among small firms. The most promising of these will be studied in a full-scale randomized evaluation at a later stage. These pilot experiments will complement the internal capacity building efforts and serve as hands-on case studies for KRA’s research team.

Direct Benefit Transfers for Electricity in Punjab

Researchers/Office: Nick Ryan, Anant Sudarshan, J-PAL South Asia  
Government Partner: 
Punjab Department of Agriculture, Punjab Department of Power, Punjab Department of Finance and Planning
Location: 
India
Type: 
Research, Pilot (Type 1)

Agricultural electricity subsidies in India were created as a lifeline to farmers, allowing the spread of irrigation using electric pumps to extract groundwater. Instead, they have locked rural India into a destructive cycle of groundwater depletion and bad power supply. Farmers, given electricity for free, use too much, draining groundwater, worsening poverty, and bankrupting electricity distributors. Given the over-use caused by pricing power below cost, both farmers and the distribution companies could be made better off if subsidies were given as a lump-sum transfer, rather than being tied to power use. In collaboration with the Government of Punjab’s Departments of Agriculture and Power, the researchers will collect data and design a welfare-improving reform. This project responds to a request from the Government of Punjab for innovative ideas in the power sector and is one of the first evaluations to come out of the MoU between the Government of Punjab and J-PAL South Asia

Institutionalizing Evidence at the Department of Education, Philippines

Researchers/Office: Dean Yang, Emily Beam, IPA Philippines, J-PAL Southeast Asia  
Government Partner: Philippines Department of Education, Policy Research and Development Division
Location: The Philippines  
Type: Institutionalizing use of evidence (Type 3)

The Philippines Department of Education (DepEd) created the Policy Research and Development Division (PRD) in January 2015 to promote evidence-based decision-making and develop evaluation standards. DepEd has requested support from IPA and J-PAL to build the capacity of PRD to institutionalize monitoring and evaluation within DepEd and to develop a large-scale evaluation of the senior high school curriculum under the K-12 Basic Education Program. This evaluation has the potential to inform the Philippine national education policy, reaching over 2 million students each year. This partnership, which builds on earlier travel support from GPI, will help build the foundations for a strong impact evaluation unit within PRD to continuously improve education policies and programs. 

Innovation and Evaluation Hub: A Model for Sub-National Governments in Argentina

Researchers/Office: J-PAL LAC, Alejandro Ganimian
Government Partner: Government of Salta
Location: Salta, Argentina   
Type: Institutionalizing use of evidence (Type 3)

This partnership will create an Innovation and Evaluation Hub in the Ministry of Education of the Province of Salta, Argentina. The hub will have two main goals: (a) to leverage the recent adoption of national student assessments at the end of primary and secondary school to conduct randomized evaluations of education interventions every year; and (b) to provide technical assistance to the Ministry to collect student and school performance data and link these data to the datasets from the national assessments and other sub-national ministries. The team plans to carry out a demonstration randomized evaluation in the fall of 2017 to establish the proof of concept. They will then support the hub in developing its research agenda in early 2018 and pursue it during the rest of the year. Finally, they will assist the ministry in launching its first request for proposals in 2019. The team hopes the hub can serve as a model for other Argentine provinces.

Training Local Leaders to Prevent and Reduce Domestic Violence in their Communities

Researchers/Office: Erica Field  
Government Partner: Ministry of Women and Vulnerable Populations of Peru
Location: Peru
Type: Research, Full RCT (Type 1)

Gender-based violence (GBV) is a pervasive issue that challenges policymakers around the globe. Despite this, there is an alarmingly limited body of rigorous research capable of informing policy. This project will bring new experimental evidence on an innovative and theoretically driven intervention from the Ministry of Women and Vulnerable Populations of Peru that aims to prevent and reduce GBV by training local leaders as community health volunteers to promote change at various levels in their communities. The experimental design will address important shortcomings of the current scientific literature as well as answer policymakers’ priority questions. This project is the result of a previous GPI grant that promoted a partnership between IPA Peru, J-PAL LAC, and the Ministry of Women and Vulnerable Populations to institutionalize the use of evidence on GBV-related policy. To learn more about the partnership with the Ministry of Women and Vulnerable Populations, see this news story from November 2016 and this blog from February 2018.

Projects Funded During the Fifth Round (Q4 2016)

Catch Up Pilot Program (Phase 2)

Researchers/Office: Rachel Glennerster, J-PAL Africa  
Government Partner: Zambia Ministry of General Education
Location: Zambia
Type: Scale-up (Type 2)

Despite almost universal primary school enrollment in Zambia, many students fail to learn basic skills. In 2016, the Zambia Ministry of Education collaborated with J-PAL Africa to launch a pilot of the Zambia Catch Up Program in 80 schools, which was modeled after interventions that had been found effective through several randomized evaluations in India, Ghana, and Kenya. J-PAL Africa, IPA Zambia, and UNICEF conducted an independent process monitoring of the pilot program to ensure that the program could be successfully implemented in accordance with its theory of change. After refining the model that was best for their school system, the Ministry committed to scaling the approach to approximately 1,800 schools across Zambia over three years in August 2017 with funding from USAID’s Development Innovation Ventures and USAID Zambia. This partnership was also supported by IPA, Pratham, UNICEF, VVOB, and Zambia Education Sector Support Technical Assistance (ZESSTA). For more on Teaching at the Right Level in India, see this scale-up story and this December 2017 op-ed in Livemint by J-PAL South Asia Executive Director Shobhini Mukerji and Pratham CEO Rukmini Banerji. For more on the Catch Up program in Zambia, see this blog post from January 2018 and this set of videos produced by Pratham, J-PAL Africa, the Zambian Ministry of General Education.

Online Executive Education Course on Evaluating Social Programs in Portuguese: A Pilot in Partnership with Brazil’s National School of Public Administration (ENAP)

Researchers/Office: Claudio Ferraz, J-PAL LAC 
Government Partner: National School of Public Administration of Brazil (ENAP)
Location: Brazil
Type: Institutionalizing use of evidence (Type 3)

Brazil’s fiscal crisis has increased government interest in evidence-informed policy design. During a time of budget cuts, this project addresses demand for large-scale capacity building on evidence-informed policymaking and impact evaluation within the Brazilian government at a relatively low cost. Together with Brazil’s National School of Public Administration (ENAP), Claudio Ferraz and J-PAL LAC developed and piloted a Portuguese version of J-PAL’s online Executive Education course Evaluating Social Programs. Nearly 5,800 Brazilian civil servants enrolled in the first iteration of this course, which launched in October 2017, and over 1,400 completed it and received a joint certificate from ENAP and J-PAL. The course will be included in ENAP’s regular course catalogue and offered to thousands of civil servants on a yearly basis. ENAP trains approximately 74,000 civil servants per year through its online courses. To learn more about the MOOC and the partnership with ENAP, see this blog post written by Anna Mortara, former consultant to J-PAL LAC who sat within ENAP's office for a year during the development of the course.

Learning2Leap: Systematizing Learnings from Institutional Partnership Schemes in LAC

Researchers/Office: J-PAL LAC 
Government Partner: Ministry of the Environment, Brazil; Citizen Security Institute, Rio de Janeiro, Brazil; Innovation Division, Ministry of the Economy, Chile; National Planning Department, Colombia; Prospera, Program for Social Inclusion, Mexico; Ministry of Social Development, Peru; Ministry of Education, Peru; Ministry of Women and Vulnerable Populations, Peru
Location: Brazil, Chile, Colombia, Mexico, Peru
Type: Institutionalizing use of evidence (Type 3)

J-PAL LAC has collaborated with many governments throughout the region to build demand for and capacity to use evidence in policymaking. Learning2Leap is a qualitative research study to learn from eleven of these partnerships about what encourages governments to become interested in increasing the use of data and evidence in policy, how J-PAL LAC can best support them in these efforts, and what characteristics and activities have made these partnerships most effective at informing policy in the past. J-PAL will use these findings to improve its existing partnerships with governments and the process by which new partnerships are developed. J-PAL also produced a final report to contribute to growing literature on evidence-informed policymaking within government. For early insights about building a culture of data and evidence use in government, see this blog post from February 2018. The report, Creating a Culture of Evidence Use: Lessons from J-PAL's Government Partnerships in Latin America, is available here in English and Spanish. The Executive Summary is also available in English, Portuguese, and Spanish.

Incentives for Immunisations in Sierra Leone

Researchers/Office: J-PAL Africa
Government Partner: Ministry of Health and Sanitation, Sierra Leone
Location: Sierra Leone
Type: Scale-up (Type 2)

Take up of preventive health care has been shown to be highly sensitive to price. Randomized evaluations in India and Pakistan are currently testing how immunization responds to different incentives. Both use mobile payment platforms to provide incentives. However, for the poorest communities, mobile payments are often infeasible. This project tests the feasibility of incorporating immunization incentives into common maternal and infant nutritional support programs. The program will be delivered by a partnership between the World Food Programme (WFP) and the Ministry of Health and Sanitation (MoHS), with technical assistance from J-PAL Africa and IPA on design and monitoring. The team will collect high quality independent data to determine whether the key assumptions in the theory of change hold in this environment. In particular, can nutritional supplements (mainly fortified cereal) be delivered in a reliable and timely way to remote clinics? Can clinic staff effectively incorporate distribution of food supplements into immunization protocols? Does the food reach the intended recipient? After the first year of the program, which is expected to reach 60,000 pregnant and nursing women and 40,000 children aged 6 to 23 months, WFP and MoHS intend to scale it up in additional districts. If the program is successful, WFP is also interested in incorporating the model into its work with governments in other countries.

Projects Funded During the Fourth Round (Q2 2016)

Improving Job Matching through Job Fairs in Egypt

Researchers/Office: Bruno Crepon, Adam Osman
Government Partner: Industrial Training Council, Egypt
Location: Egypt
Type: Research, Pilot (Type 1)

Youth unemployment is a critical challenge in across the world. In Egypt, one sixth of the youth population is unemployed and one third is neither in education, employment or training. Despite high unemployment rates, firms report significant difficulties filling vacancies: in 2012, the private sector in Egypt was unable to fill nearly 600,000 jobs.1 This suggests that the Egyptian economy is characterized by significant labor market distortions and search frictions, hindering economic growth. Job fairs may decrease the matching frictions between firms and job seekers by bringing job seekers in direct contact with firms that are hiring for formal jobs. This project will evaluate the impacts of a) providing information about job fairs and the labor market to job seekers and b) a cash subsidy for travel to the fair on job search behavior, labor market outcomes, and firm performance. For more on J-PAL's work in the Middle East and North Africa, including this project, see this blog post from December 2017.

Scaling up a new funds-flow mechanism for government payments in India (Phase 2)

Researchers/Office: J-PAL South Asia 
Government Partner:  Government of India
Location: India
Type: Scale-up (Type 2)

Low administrative capacity and pervasive corruption constrain the performance of social programs in many developing countries. However, e-governance mechanisms may be a promising way to mitigate this problem by reducing the number of agents acting as intermediaries and making public service delivery more efficient. Evidence from a randomized evaluation of a large public employment program in India, the Mahatma Gandhi National Rural Employment Guarantee Scheme (MGNREGS), found that one such mechanism reduced leakage by decreasing the number of potential bribe-takers in the system, and increased the efficiency of funds distribution by reducing the idle funds in the system. Building on this work and a previous GPI grant, the J-PAL South Asia policy team temporarily placed a staff member within the Ministry of Rural Development to support the government in scaling up the policy lessons from the MGNREGS study nationwide. The staff member provided full-time support to ensure that the policy implementation plan, developed during the first year of this project, was approved and implemented. By November 2017, 21 states and one union territory had already implemented the reform, and three more states had committed to implementing it.

Statistical information and lawsuit settlement in Mexico

Researchers/Office: Enrique Seira, Joyce Sadka, Christopher Woodruff, J-PAL LAC 
Government Partner:  Mexico City Labor Court
Location: Mexico
Type: Scale-up (Type 2)

In order to address a huge backlog of cases and low settlement rates, the President of Mexico City’s Labor Court asked researchers to construct a calculator that provides statistical information to parties and lawyers attending hearings during the course of their lawsuits. In a pilot evaluation funded by GPI, researchers built the calculator using a large database of concluded lawsuits from one subcourt and tested it in the same subcourt. Use of the calculator increased the settlement rate by over 90 percent for lawsuits approached on the same day as the scheduled hearing, and had larger impacts on lawsuits approached at earlier stages. As a result, the court President requested scaling up the calculator to first-time hearings in 5 of the 20 subcourts, as well as to the population of workers seeking legal advice on a firing before they find a lawyer and present a lawsuit. Professors Seira, Sadka, and Woodruff, along with J-PAL staff in Mexico, will provide technical support in the first phase of this scale-up. For more on the scale-up, see this news article from September 2016.

Institutionalizing an evidence-based approach to policy making within the GoTN – Year 3

Researchers/Office: J-PAL South Asia 
Government Partner:  Government of Tamil Nadu
Location: India
Type: Institutionalizing use of evidence (Type 3)

The Government of Tamil Nadu entered into a comprehensive partnership with J-PAL SA at IFMR in 2014 with the stated goal of institutionalizing evidence-based policy making. More than three years into the partnership, the government has supported 15 policy-driven research projects across nine departments, using over US$4 million of its own funding and partnering with more than 34 professors. Using funds from GPI, J-PAL South Asia staff also provide capacity-building support to a Data Analytics Unit set up by the Government to allow them to make real-time decisions based on the administrative data they already collect. Recently, the state issued new guidelines for program evaluation requiring that all programs in sectors of strategic importance with budgets greater than US$22 million undergo impact evaluations. The Planning Department has also created a dedicated fund for program evaluation of US$1.5 million per year. For more on J-PAL South Asia's partnership with the Government of Tamil Nadu, see this news story from May 2016, this November 2017 op-ed by J-PAL Executive Director Iqbal Dhaliwal in Livemint, and this blog post from January 2018.

Projects Funded During the Third Round (Q1 2016)

Leveraging Patients’ Social Networks to Overcome Tuberculosis Under-detection in India

Researchers/Office: J-PAL South Asia, Jessica Goldberg, Mario Macis, Pradeep Chintagunta 
Government Partner:  Delhi State Tuberculosis Department
Location: India
Type: Research, Pilot (Type 1)

In 2015, an estimated 3.5 million people suffered from Tuberculosis (TB) in India. The under-detection of TB represents a key challenge for health officials in developing countries because the success of any treatment program rests crucially on identifying those who have the disease. In collaboration with the Delhi State TB Department, researchers will evaluate a community engagement strategy to increase detection of TB. This pilot randomized evaluation will compare the effects of different types of financial incentives to encourage current TB patients to refer people from within their social networks for TB screening and testing. This project builds upon an earlier project in the private sector, where positive results and ongoing communication led the Delhi State TB Officer to suggest collaboration in the public sector.

Estimating the Impact of Tax Reform Administration Policies: Evidence from Indonesia

Researchers/Office: J-PAL Southeast Asia, Ben Olken, Rema Hanna, Muhamad Chatib Basri 
Government Partner: Directorate General of Taxes
Location: Indonesia
Type: Institutionalizing use of evidence (Type 3)

In 2002, Indonesia’s Directorate General of Taxes (DGT) launched a comprehensive reform of the tax administration. Among other interventions, performance incentives were overhauled, the organizational structure of tax offices was modified, and IT systems were upgraded to streamline data collection and facilitate tax filings. Using quasi­-experimental research designs, Harvard professor Rema Hanna, MIT professor Ben Olken, and former Minister of Finance of Indonesia M. Chatib Basri are collaborating with Indonesia’s Ministry of Finance (MoF) and DGT to evaluate the effectiveness of those various complex reform policies. This partnership aims to not only inform MoF’s ongoing tax reform, but to open opportunities for further testing of tax policies through randomized evaluations and to share results from relevant RCTs to the current administration, institutionalizing the use of evidence and building capacity at MoF and DGT. 

Evidence Use and Generation-A Learning Scheme in the Ministry of Economy

Researchers/Office: J-PAL LAC, Claudia Martinez, Jose Tessada
Government Partner:
Innovation Division, Chilean Ministry of Economy, Development and Tourism
Location: Chile
Type: Institutionalizing use of evidence (Type 3)

J-PAL LAC worked with the Innovation Division of the Chilean Ministry of Economy to develop an evidence-informed strategy for disbursing the funds from its Innovation for Competitiveness Fund (FIC), which finances nearly 70 different innovation efforts each year across various agencies inside the Ministry of Economy. The Innovation Division reached out to J-PAL LAC to get support in building a system for generating and using rigorous evidence within the fund. J-PAL LAC conducted two literature reviews in areas of strategic interest to the Ministry, delivered training courses and workshops on impact evaluation and the use of evidence, and helped develop a Program Design and Evaluation Form that is now required for all applicants to FIC. The system requires applicants to include a theory of change and review of existing evidence from past impact evaluations when seeking funds. FIC also launched a new public bid for impact evaluations of the Innovation Fund’s investments and reserved a portion of program funds for evaluation. Researchers and J-PAL LAC also developed an impact evaluation of a priority program for the Ministry. 

Projects Funded During the Second Round (Q4 2015)

Technical Assistance to Implement a Policy Design and Optimization Lab at the Brazilian Ministry of the Environment

Researchers/Office: J-PAL LAC, Kelsey Jack, Mushfiq Mobarak, Marco Gonzalez Navarro
Government Partner:  Brazilian Ministry of the Environment
Location: Brazil
Type: Institutionalizing use of evidence (Type 3)

Despite its policy and political relevance, the Brazilian Ministry of Environment (MMA) lacks tools to fine-tune and improve its programs. This project will provide MMA with technical support to implement a Policy Design and Optimization Lab (PDOL) and build capacity to generate and use evidence in policymaking. We will train MMA teams to run PDOL as an incubator of randomized evaluations to continuously generate and feed learning into policy design. MMA and J-PAL will implement the project with support of Climate Policy Initiative Brazil (CPI), a non-profit with strong collaboration with MMA and key J-PAL partner in Brazil. PDOL will be strategic to demonstrate how MMA can generate and use rigorous evidence to address pressing environmental challenges.

Institutionalizing the use of evidence to fight crime and reduce police violence

Researchers/Office: Claudio Ferraz, Joana Monteiro 
Government Partner:  Citizen Security Institute of the State Government of Rio de Janeiro (ISP-Rio)
Location: Brazil
Type: Institutionalizing use of evidence (Type 3)

Rio de Janeiro State´s Citizen Security Secretariat and Military Police struggle to fight high rates of crime with a police force that many consider corrupt and violent. Moreover, little is known about the causes of police violence, how police violence affects trust in the state, and what policies can effectively reduce it. Through this partnership, Claudio Ferraz and J-PAL LAC helped Citizen Security Institute of the State Government of Rio de Janeiro (ISP-Rio) lay the groundwork that will allow it to address some of these questions. GPI funds supported a J-PAL Research Associate to sit inside ISP-Rio for one year to organize its data more effectively. Along with organizing data, she also helped ISP-Rio develop ISP-Geo, a platform that allows police officers to view real-time data on crime, police presence, and environmental factors, and to dispatch additional police officers to priority areas. After GPI funds had supported the Research Associate for one year, ISP-Geo decided to hire her as a full-time state employee. For more on the partnership with ISP-Rio, see this news story from August 2016 and this blog post from February 2018.

Supporting the Security Department of the Mayor’s Office of Bogota To Implement Rapid-Fire RCTs and Use Evidence in Decision-Making

Researchers/Office: Chris Blattman, Daniel Ortega, Daniel Mejia, Santiago Tobon
Government Partner: Mayor's Office of Bogota
Location: Colombia
Type: Institutionalizing use of evidence (Type 3)

In Bogotá, one of Latin America’s largest cities, 59 percent of residents report feeling insecure in the city (a sharp increase from 24 percent in 2008). The Mayor-elect has appointed a prominent economist from Universidad de los Andes, Daniel Mejia, to become Secretary of Security. Having spent several years running randomized interventions with police in other Colombian cities, Dr. Mejia intends to run dozens of crime RCTs in Bogota as part of its reform efforts. The Mayor’s office has already planned two rapid-fire RCTs for their first months in office: one of “hotspot” policing tactics in thousands of city blocks; and a second of lighting, clean-up, policing, and infrastructure investments in 200 high-crime parks. Beginning in mid-2016, the Mayor’s office also plans to design larger and more ambitious RCTs around their medium-term projects. The J-PAL Crime & Violence sector and the IPA Colombia office will provide technical assistance in the design, implementation, and scaling of RCTs. This partnership also involved funding and research support from the Impact Evaluation Office of the Corporación Andina de Fomento (CAF) and the Department of Economics and Center for the Study of Security and Drugs at the Universidad de los Andes. For more on the partnership with the Bogota Mayor's Office, see this news story from August 2016 and this evaluation summary.

Institutionalizing Evidence Use in the Sustainable Livelihood Program of the Philippine Department of Social Welfare and Development

Researchers/Office: J-PAL Southeast Asia, IPA Philippines
Government Partner: Philippines Department of Social Welfare and Development
Location: Philippines
Type: Institutionalizing use of evidence (Type 3)

In July 2015, the Philippines government issued a circular mandating evidence-based decisions and proposed ambitious new evaluation standards across all government departments. Motivated in part by these new standards and an impending expansion, the Department of Social Welfare and Development (DSWD) has requested IPA’s support to help its evaluation unit set up a rigorous evaluation of a core program—the Sustainable Livelihood Program (SLP). The SLP is a community-based capacity building program coupled with cash grants that aims to help micro-entrepreneurs and job seekers achieve a sustainable income source. With funding from the Government Partnership Initiative, IPA seconded a Policy Associate to the DSWD to comprehensively train department staff on using and collecting evidence, conduct a process evaluation of the SLP and—with support from the Social Protection Initiative at IPA—develop an impact evaluation of the SLP.

Institutionalizing an Evidence-based Approach to Policy Making within the Government of Tamil Nadu

Researchers/Office: J-PAL South Asia at IFMR, Aparna Krishnan 
Government Partner:  Government of Tamil Nadu
Location: India
Type: Institutionalizing use of evidence (Type 3)
                
The Government of Tamil Nadu entered into a comprehensive partnership with J-PAL SA at IFMR in 2014 with the stated goal of institutionalizing an evidence-based policy making. This requires a holistic approach including a commitment to results, an openness to reviewing failures, strengthening technical capabilities of staff as well as necessary infrastructure to collect, analyze and use relevant and credible data in a timely manner. Recognizing this, the Government of Tamil Nadu has entered into a technical advisory engagement with J-PAL SA to bring about changes in departmental processes, standards and mindsets to adopt data-driven decision making at all levels within the government. This engagement implemented a series of measures aimed at greater institutional links between researchers and policymakers, systematically strengthening the quality of data from evaluations and other forms of measurement, and finally conceptualizing a data analytics initiative that could pool such data for further policy action. For more on J-PAL South Asia's partnership with the Government of Tamil Nadu, see this news story from May 2016, this November 2017 op-ed by J-PAL Executive Director Iqbal Dhaliwal in Livemint, and this blog post from January 2018.

Scaling up training program for informal healthcare providers in West Bengal, India


Researchers/Office: J-PAL South Asia
Government Partner:  Department of Health & Family Welfare (Government of West Bengal)
Location: India
Type: Scale-up (Type 2)   

Informal providers in the private sector with little or no formal medical training represent majority of India’s rural primary healthcare workforce. Given their strong community ties and presence where few alternatives exist, one potential solution to abate harm to the population they serve and improve their quality of care is to provide them with training. Experimental evidence from a RCT of a generalized medical training program found substantial improvements in both knowledge and practice among trained informal health providers. The results from this evaluation have informed the decision of the Government of West Bengal (GoWB) for a state-wide scale-up of the intervention. J-PAL SA will work closely with the GoWB in developing a robust monitoring system for the scale-up and support with process documentation for the first year of the planned scale-up. For more on the scale-up plans, see this news article from October 2016.

Catch Up Pilot Program

Researchers/Office: J-PAL Africa, Rachel Glennerster, Tavneet Suri
Government Partner:  Zambia Ministry of General Education
Location: Zambia
Type: Scale-up (Type 2)   

Despite gains in enrollment, learning levels in Zambia remain low. Large, heterogeneous classrooms mean that students who miss basic skills never have the chance to catch up. Randomized evaluations from researchers in the J-PAL network show that programs which tailor learning to the level of the child help solve this problem. In conjunction with J-PAL Africa and other cooperating partners, the Ministry of General Education began planning to launch a Catch Up Program modelled after interventions proven to be effective through randomized evaluations. The program began with a pilot in May 2016 and, after refining the model that was best for their school system, the Ministry committed to scaling the approach to approximately 1,800 schools across Zambia over three years in August 2017 with funding from USAID’s Development Innovation Ventures and USAID Zambia. For more on Teaching at the Right Level in India, see this scale-up story and this December 2017 op-ed in Livemint by J-PAL South Asia Executive Director Shobhini Mukerji and Pratham CEO Rukmini Banerji. For more on the Catch Up program in Zambia, see this blog post from January 2018 and this set of videos produced by Pratham, J-PAL Africa, the Zambian Ministry of General Education.

Institutionalization of the use and generation of evidence for improving Ministry of Women and Vulnerable Populations interventions

Researchers/Office: J-PAL LAC, IPA Peru
Government Partner: National Program Against Domestic and Sexual Violence (PNCVFS), Office of Monitoring and Policy Evaluation (OMEP), Ministry of Women and vulnerable populations of Peru (MIMP)
Location: Peru    
Type: Institutionalizing use of evidence (Type 3)

The project worked with the Peruvian Ministry of Women and Vulnerable Populations (MIMP) to improve its policies, programs and projects by incorporating and producing rigorous evidence on the impact of its interventions. The project, which was developed in coordination with the National Program against Domestic and Sexual Violence (PNCVFS), the General Office of Monitoring, Policy Evaluation and Decentralized Management (OGMEPGD) and, inside it, the Office of Monitoring and Policy Evaluation (OMEP), sought to institutionalize the use and generation of evidence in the operations of the Ministry. The collaboration included three main components: (i) an evidence repository, which collected evaluation results of interventions that seek to prevent and address violence against women, children and adolescents; (ii) an impact evaluation incubator, aimed to strengthen capabilities on rigorous impact evaluation within the Ministry personnel and to identify innovation and evaluation opportunities; and (iii) an institutionalization proposal to identify concrete steps to durably embed the use and generation of evidence in the Ministry’s operations and ultimately render the policymaking process more effective. To learn more about the partnership with the Ministry of Women and Vulnerable Populations, see this news story from November 2016 and this blog from February 2018.

Projects Funded During the First Round (Q3 2015)

Embedding technical support for system-wide, evidence-based education reform in Delhi 

Researchers/Office: Karthik Muralidharan, Alejandro J. Ganimian
Government Partner: Government of the National Capital Territory of Delhi
Location: India
Type: Institutionalizing use of evidence (Type 3)

The low effort and effectiveness of government teachers in India is now well-documented. A number of randomized evaluations have shown that non-discretionary, systematic monitoring and incentives can raise teacher effort and improve student achievement. However, the question that remains to be addressed by existing research is whether school systems can adopt mechanisms to monitor teacher attendance and effort, and provide feedback to teachers at scale. This project will embed a J-PAL South Asia Research Associate (RA) in the Government of the National Capital Territory of Delhi (GoNCTD) to assist with the preparation of a randomized evaluation of a state-wide rollout of Closed Circuit Television (CCTV) cameras in the 22,000 classrooms under the supervision of the state’s Directorate of Education (DoE). The RA will lend support to other ongoing policy dialogues with GoNCTD, as well as help strengthen its capacity to collect and analyze administrative data.

Supporting Colombia’s Division of Monitoring and Evaluation of Public Policies (DSEPP) at Creating and Using Evidence for Decision-Making

Researchers/Office: J-PAL LAC, IPA Colombia
Government Partner: National Planning Department, Government of Colombia
Location: Colombia
Type: Institutionalizing use of evidence (Type 3)

Colombia’s Division of Monitoring and Evaluation of Public Policies (DSEPP), part of the National Department of Planning (DNP) of the Colombian Government, has the mission of defining evaluation priorities, and coordinating and funding evaluations, to support policymakers in making evidence-based decisions. In 2016, the DSEPP pivoted from being one of many agencies that hire and implement evaluations to being an agency that also coordinates and provides guidance and technical assistance to all government bodies involved in evaluations. As a part of this new strategic focus, DSEPP will ensure that government-funded evaluations answer relevant questions for the policy-making process and meet quality standards, and that evaluation results are effectively communicated to decision-makers. J-PAL and IPA supported the DSEPP by contributing to building a repository of existing and ongoing evaluations, providing technical training to staff, identifying knowledge gaps to define evaluation priorities, and building a strategy to effectively communicate evaluation results to decision-makers. For more on the partnership with DSEPP, see this blog post on the IPA website from April 2016 and this April 2016 press release by the DNP (in Spanish).

Scaling up a new funds-flow mechanism for Centrally Sponsored Schemes in India

Researchers/Office: J-PAL South Asia
Government Partner: Government of India
Location: India
Type: Scale-up (Type 2)

Low administrative capacity and pervasive corruption constrain the performance of social programs in many developing countries. The increasing promise of e-governance for delivering public services makes it possible to design mechanisms with fewer agents intermediating the delivery process. Evidence from a randomized evaluation of a large public employment program in India, the MGNREGS, found that such redesigns reduced leakage by decreasing the number of potential bribe-takers in the system, and increased the efficiency of funds distribution by reducing the funds float in the system. Building on this work, the J-PAL South Asia (J-PAL SA) policy team worked closely with the Government of India (GoI) to develop a roadmap to scale-up the policy lessons from the MGNREGS study and apply the same mechanism for other Centrally Sponsored Schemes of the GoI. Through this grant, J-PAL SA hired a Policy Consultant to provide full-time support to the GoI for this scale-up over the course of one year. For more on the partnership with the Ministry of Rural Development, see this story about the scale-up and this blog post from January 2018.

Federal Court Mediation Study

Researchers/Office: D. James Greiner 
Government Partner: United States District Court for the District of Nevada
Location: Nevada, United States
Type: Research, Full RCT (Type 1)

Despite the centrality of court adjudication to the rule of law, and of court-connected mediation programs to court adjudication, there has never been a well-executed and exhaustive study of mediation’s costs and benefits. This project will complete a randomized control trial (“RCT”) evaluation of a court-connected mediation program. The RCT has taken place in an environment unusually hostile to evidence-based thinking, namely, the United States federal court system. Multiple outcome variables directly relevant to mediation’s alleged justifications will be measured. Intake on the 353-unit study has closed, and the funding will be used to complete cost estimation, research assistant spreadsheeting of outcomes, and nonstandard data analysis. The RCT is critical to a resource allocation dispute ongoing within the federal judiciary, which has indicated its desire for more such evidence, and it will serve as a model for court systems in the United States and abroad.

Evidence use and generation – A learning scheme in the Ministry of Economy

Researchers/Office: Claudia Martinez, José Tessada, and J-PAL LAC
Government Partner: Chilean Ministry of Economy, Development and Tourism 
Location: Chile
Type: Institutionalizing use of evidence (Type 3)

In partnership with the Chilean Ministry of Economy, we will develop a learning scheme inside their Innovation Division, building and strengthening the capacities of their team and identifying evaluation opportunities. The Innovation Division at the Chilean Ministry of Economy, Development and Tourism is in charge of the Innovation for Competitiveness Fund, which annually finances nearly 70 different innovation efforts across various government agencies inside the Ministry of Economy. Due to concerns regarding the way in which such funds should be distributed, and acknowledging the lack of a proper system of evidence generation and usage, the Innovation Division’s team reached out to J-PAL LAC. This project will work with the Innovation Division to identify and learn from relevant evidence, to develop plans for new evaluations, and to build capacity to implement evaluations.

Implementing and Evaluating Immunization Incentives at Scale in Haryana

Researchers/Office: Esther DufloJohn Floretta, Harini Kannan, Maheshwor Shrestha, J-PAL South Asia
Government Partner: National Health Mission, Government of Haryana
Location: India
Type: Research, Full RCT (Type 1) and Scale-up (Type 2)

Immunization is one of the most cost-effective ways to improve child health in developing countries. In India, at most 54 percent of children aged 12 to 24 months are fully immunized. The government has invested heavily in improving vaccine delivery. However, this big push to improve access has had limited impact. In Haryana, official full immunization rates actually fell from 60 percent in 2007-2008 to 52 percent in 2012-2013. This project will provide technical assistance to the Government of Haryana to scale an evidenced-based model to build demand for vaccinations by providing small incentives to parents. The model is based on a seminal J-PAL evaluation in Rajasthan, which covered 134 villages. The main finding that households were very sensitive to small incentives has been replicated in other contexts, but not yet implemented on a large scale. J-PAL will provide support to replicate the program in 1,300 villages. A randomized evaluation will assess the impact of incentives, along with two complementary interventions to build demand through information, to help make the case for further scale up. For more on the program, see this December 2016 article in The Pioneer. For more on the ongoing evaluation, see this evaluation summary.

Expert advice versus statistical information: effects on settlement rates in labor disputes in Mexico

Researchers/Office: Enrique Seira, Bentley MacLeod, Joyce Sadka 
Government partner:  Junta Local de Conciliación y Arbitraje del Distrito Federal (Local Conciliation and Arbitration Court of the Federal District)
Location: Mexico
Type: Research, Pilot (Type 1)

A distinguishing feature of labor litigation is that plaintiffs are rarely repeat players and therefore often have poor information about probable results of the lawsuit. In developing countries, this problem is exacerbated because a large proportion of fired workers are low-skill, low education and severely under-informed. Furthermore, the evidence suggests that they are overconfident about the outcomes of the lawsuit, which hinders settlement. In this context, we will test and compare two policies to reduce misinformation and increase settlement rates. We will build a calculator using outcomes of recently concluded case files, and provide this “statistical information” on a randomized basis to workers right before their conciliation hearing. We will also provide qualitative advice on their lawsuits through labor law experts who are not their legal representatives and have no stakes in the case. We will carry out detailed surveys to elicit information on trust in the legal advice, expectations before and after interventions, expert prediction of outcomes, and the contract terms between the lawyer and the plaintiff. GPI funded a scale-up of the intervention in its fourth funding round; for more on the scale-up, see this news story from September 2016

Improving the Health Worker‐Patient Relationship

Researchers/Office: Seema JayachandranSimone Schaner 
Government Partner: Government of Punjab, Department of Health and Family Welfare
Location: India
Type: Research, Pilot (Type 1)

Despite investments in infrastructure and personnel that have increased access to care, the quality of government health care in India remains low. One important, but understudied, aspect of quality is provider-patient communication. Poor “soft skills” among providers—being disrespectful of patients or lacking a compassionate interpersonal style, for example—may play a central role in inhibiting the exchange of information between providers and patients, and could even drive patients to forgo care or turn to unqualified private providers. Along with the Government of Punjab, we are planning a randomized evaluation of several innovative interventions aimed at making nurses at health subcenters more attentive to patient satisfaction. The interventions include soft skills training, feedback on patient satisfaction, and rewards for having high patient satisfaction. We will pilot the soft skills training and provider feedback interventions before launching the full-scale study.

References

1African Economic Outlook 2012

Proposal to set up a Data Science for Policy Impact Initiative at J-PAL South Asia