Desenvolvendo uma comunidade global para avaliação de impacto: Cursos online do J-PAL

Earlier this year, the Data, Economics, and Development Policy MicroMasters program celebrated its five-year anniversary, and we reflected on our achievements throughout this journey. Today, as we look forward, we are delighted to announce another milestone for the program: the launch of a new Public Policy Track to complement our existing curriculum, which has so far primarily focused on international development. Accompanying this new Public Policy Track is a change in the name of the program to Data, Economics, and Design of Policy. This revised program name maintains the acronym DEDP while capturing the expanded range of subjects covered in the curriculum. Additionally, we will be introducing a new course, Microeconomic Theory and Public Policy, to the program’s offerings.

Despite these exciting changes, the mission of the DEDP program remains steadfast on empowering a community of diverse, exceptional individuals with the technical skills to tackle poverty alleviation using evidence-based approaches. The expansion will allow us to strengthen this mission by broadening the scope of the program, thus allowing learners to specialize in their respective interests.

Our existing curriculum delves into the fundamental questions that are central to understanding the challenges faced by countries grappling with widespread and persistent poverty. This emphasis is evident in our elective courses offered under the International Development Track, which include the Challenges of Global Poverty, Foundations of Development Policy, and Political Economy and Economic Development. Employing real-life, empirical examples from the context of low- and middle-income countries, these courses investigate the key questions posed by scholars studying poverty: What determines the decisions of low-income households in developing countries? Is microfinance invaluable or overrated? Does economic development drive political institutions, do political institutions affect economic development, or is there a third factor affecting both? 

However, it is important to recognize that poverty manifests differently across regions and contexts. In high-income countries, pressing policy concerns possess distinct characteristics. Thus, the new track will concentrate on public policies relevant to areas like international trade, climate change, and minimum wage, primarily drawing examples from the US and other high-income countries. As such, the Public Policy Track builds on the core courses through two electives: Good Economics for Hard Times, taught by Esther Duflo and Abhijit Banerjee, and Microeconomic Theory and Public Policy, taught by David Autor. Specifically, the new course will utilize empirical examples to evaluate theory, focusing on the causal effects of policy interventions on economic outcomes, and explore a range of local topics as well as those that transcend geographic boundaries.

What do these changes mean for our learners in practice? Learners in both tracks will be required to complete the three existing core courses: Microeconomics, Data Analysis for Social Scientists, and Designing and Running Randomized Evaluations. Learners pursuing the International Development Track (the existing track) credential will be required to take two of three electives. Learners pursuing the Public Policy Track credential will be required to take the two electives (Good Economics for Hard Times and Microeconomic Theory and Public Policy). Learners who meet the requirements of both tracks will earn two separate credentials.

Credentials holders will be eligible to apply to a DEDP Master’s program at MIT or pursue an accelerated degree at one of our pathway universities around the world. Learners in the Public Policy Track will have the opportunity to apply to the Master’s program starting in fall 2024. Admitted applicants will be notified in spring 2025 and will matriculate in spring 2026.

The journey of the DEDP MicroMasters program has been marked by growth, reflection, and celebration. As we push forward with these exciting developments, we simultaneously widen our scope on poverty alleviation as well as sharpen our focus on the nuances of evidence-based policy making in varied contexts. In doing so, we hope to continue to welcome a wider, more diverse audience of learners who can contribute their perspectives and who are collectively better poised to implement meaningful change.

This is the second blog in a series illustrating stories of how J-PAL’s training courses have built new policy and research partnerships and strengthened existing ones to advance evidence-informed decision-making. The first blog in the series highlights two meaningful examples from J-PAL’s Evaluating Social Programs course participants, the third illustrates how online course participants applied learnings at their organizations, and the fourth shares how training activities within J-PAL's evaluation incubators helped partners engage more deeply with research design.

Through our custom courses, J-PAL works in close partnership with organizations to tailor our training materials on using and producing rigorous evidence to meet specific interests and learning goals. This can include customizing content to a particular sector, geographic area, or level of technical detail, such as a workshop on strategies to measure gender outcomes for practitioners working on gender-related programs in South Asia. These tailored offerings enable participants to engage with examples that resonate with their experiences and equip them to apply learnings to their work.

Many of our custom course partners include government ministries, non-governmental organizations, and multilateral organizations that hope to establish evaluation strategies for the social programs they implement. Custom courses can offer both theoretical and practical guidance to organizations through critical stages of their programs’ evaluation cycles, from developing a theory of change and measurement strategy to using evidence in program decision-making.

This blog showcases two examples of how custom courses contributed to meaningful research and policy partnerships by building evaluation skills. These represent two out of the hundreds of custom courses J-PAL has held for thousands of participants around the world: with courses delivered in Arabic, English, French, Indonesian, Portuguese, and Spanish, J-PAL’s past custom courses span across forty countries, from Argentina to Vietnam. 

Building data collection and analysis skills for implementation monitoring in Indonesia

Since 2021, J-PAL Southeast Asia has worked closely with the Center of Education Standards and Policy (PSKP) within the Ministry of Education, Culture, Research, and Technology of Indonesia to evaluate the impact of their Empowering Schools Program on learning outcomes. This program encompasses an integrated set of interventions to transform teaching and school management practices. Through this collaboration, J-PAL Southeast Asia learned about the PSKP’s ongoing process monitoring activities. Drawing on J-PAL’s experience collecting and analyzing quantitative data, J-PAL Southeast Asia partnered with the PSKP to develop a custom training series to share best practices and support the PSKP in conducting a survey to monitor the nationwide implementation of the Empowering Schools Program.

The training series, delivered throughout 2022 and 2023, provided an overview of basic monitoring and evaluation concepts, including survey development, sampling and sample size, and data management and analysis. In addition to in-class training, J-PAL research staff served as discussion partners to provide examples and inputs based on real cases faced by the PSKP. For example, in meetings with the PSKP on their survey questions and sampling, M. Thoriq Akbar, Senior Research Associate at J-PAL Southeast Asia, offered practical guidance on how to draw samples based on the PSKP’s list of participating schools and how to rephrase ambiguous survey questions.

Deepening a partnership for evaluating early childhood programs in Egypt

In collaboration with the Egyptian Ministry of Social Solidarity (MoSS), J-PAL Middle East and North Africa (MENA) co-designed a randomized evaluation to test the impact of providing childcare subsidies and employment services on women’s employment and empowerment outcomes, as well as children’s cognitive and socioemotional skills. To ensure the success of this collaboration and strengthen capacities for producing and using evidence, MoSS requested a tailored training for monitoring and evaluation specialists and implementers of the program.

J-PAL MENA organized a two-day training workshop for Ministry staff in January 2022 to provide foundational knowledge on randomized evaluations tailored to the Egyptian context. After covering considerations for why, when, and how to conduct a randomized evaluation through lectures and interactive case studies, the final session offered a deep-dive into the ongoing evaluation of the subsidized access to nurseries program. This created an opportunity for participants to learn more about research design and implementation decisions, contribute new perspectives, and clarify important details of the study. Participants shared positive feedback on the workshop and came away with a deeper understanding of how the evaluation will lead to actionable insights for the program and their work.
 

Strengthening partnerships through custom course collaborations

Through customized training modules and discussions, J-PAL training teams are able to share relevant experiences and provide timely inputs to support partners’ objectives, forging relationships for evidence use beyond a single randomized evaluation. Custom courses support partners in pursuing rigorous monitoring and evaluation, which strengthens the culture of evidence-informed decision-making in their institutions. 

Interested in learning how J-PAL can design a custom course for your organization? Visit our custom courses page to learn more and get in touch with our team.  

This is the first blog in a series illustrating stories of how J-PAL’s training courses have built new policy and research partnerships and strengthened existing ones to advance evidence-informed decision-making. The second showcases how custom courses strengthened government partnerships, the third illustrates how online course participants applied learnings at their organizations, and the fourth shares how training activities within J-PAL's evaluation incubators helped partners engage more deeply with research design.

Since establishing our training group in 2005, J-PAL has delivered hundreds of in-person and online courses to train over 20,000 people around the world on how to generate and use evidence from randomized evaluations. From integrating existing evidence and data into decisions to building research partnerships to conduct impact evaluations, many of our training participants apply learnings to strengthen the culture of evidence use in their organizations.

Growing a community of practice for Evaluating Social Programs

Our flagship course, Evaluating Social Programs, is designed for policymakers, practitioners, and researchers interested in learning how randomized evaluations can help determine whether their programs are achieving their intended impact. Offered in several locations around the world each year, the week-long course combines interactive lectures, real-world case studies, and small group sessions to integrate key concepts with practical applications. Participants build connections with peers, academic researchers, and policy experts and come away from the course with the resources and tools to map out an evaluation strategy for their own programs and policies. 

In this blog, we highlight two notable examples of how Evaluating Social Programs courses led to high-impact research and policy partnerships, while recognizing the impressive community of practice among course alumni around the world. First delivered in Cambridge, USA, and Chennai, India in 2005, the course has trained over 2,750 participants through more than 75 offerings in locations ranging from Argentina to Zambia, as well as live Zoom offerings in response to Covid-19. We also have self-paced, online versions of the course in several languages to expand the reach of the course content to learners worldwide.

Forging a partnership for evaluating biometric enrollment in Liberia

J-PAL’s Digital Identification and Finance Initiative (DigiFI Africa) built a partnership with Liberia’s Ministry of Gender, Children, and Social Protection through J-PAL Africa’s virtual Evaluating Social Programs course in August 2021 when Shadrach Saizia Gbokie, a program manager for Liberia's Social Safety Net Project, attended the training and engaged in group work sessions led by a policy manager from DigiFI. After the training, members of our team and the Ministry discussed their work and their interest and capacity to conduct research. 

During these conversations, DigiFI connected with the National Coordinator for Social Protection at the Ministry, Aurelius Butler. Since then, DigiFI has worked closely with Aurelius and the Ministry to develop a new randomized evaluation to help answer their questions around how to enroll urban communities in their new household registry using biometric data. Researchers Erika Deserranno, Andrea Guariso, and Andreas Stegmann were matched with this opportunity, and Aurelius and Dackermue Dolo from the Ministry later attended J-PAL Africa’s 2022 Evaluating Social Programs course to dive deeper into the research methodology. One of the questions the evaluation will try to answer is whether biometric identification systems help reduce barriers to accessing social programs and achieve accurate program targeting and efficiency. 

The course on Evaluating Social Programs truly presented a space for the Ministry to fully engage with research and understand what we could achieve for the betterment of our citizens. By providing a comprehensive deep dive into research methodology we are now able to better articulate and refine our objectives. The course truly made an immediate impact in our work, as we have already begun using 90% of what we took from the course.

-Aurelius Butler, National Coordinator for Social Protection, Ministry of Gender, Children, and Social Protection, Liberia

Sparking an education technology evaluation in India

An Evaluating Social Programs training in New Delhi in 2015 was instrumental to seeding an evaluation of Mindspark, a personalized adaptive learning platform of the Indian ed tech company, Educational Initiatives (Ei). At the time of the training, Pranav Kothari was Vice President at Ei and was overseeing the development of the Mindspark product as well as deploying the product in Mindspark centers, which sought to bring a Hindi version of the tool to low-income neighborhoods in urban Delhi. He applied to J-PAL South Asia’s training, hosted in partnership with CLEAR South Asia, with a specific goal—to be sure that every child in a Mindspark center is learning—and was keen to undertake a systematic assessment of the program and its implementation to understand whether or not it worked as intended.

The course’s focus on both the theory and practice enabled Pranav to take the first step in designing a potential evaluation of the Mindspark center program. During the training’s small group work sessions, he worked closely with Maya Escueta, former Policy and Training Manager at J-PAL South Asia, to develop a preliminary evaluation plan. Seeing the potential in the program and its evaluation, Maya visited the Mindspark centers, and eventually brought on J-PAL affiliated researchers Karthik Muralidharan, Abhijeet Singh, and Alejandro Ganimian to conduct a full-scale randomized evaluation. Run in close collaboration with Pranav and the Ei team, the evaluation demonstrated that the program increased learning levels across all groups of students. The study has since become one of J-PAL’s landmark studies in education and offers valuable evidence on the use of computer-adapted learning in low-resource settings.

While we had been working on the ground for over two years, we wanted a third-party evaluation to determine whether our intervention was useful or not. This course helped us plan and conduct an RCT, allowing us to identify how well our program worked and where we could refine it. These results have led to newer versions of the program, and have helped us scale Mindspark to reach over 300,000 students, or about 500 times the number of students that were initially studying in the centers. I have also learned many skills which helped Ei in doing impact evaluations of other educational interventions.

-Pranav Kothari, Chief Executive Officer, Educational Initiatives

Training to strengthen evidence-informed decision-making

J-PAL’s Evaluating Social Programs course equips program implementers, policymakers, and researchers with the tools to understand and engage with randomized evaluations to become better producers and users of evidence. This blog shines a light on just a few examples of how past participants through the years have integrated learnings at their organizations to advance evidence-informed decision-making.

Interested in learning how to strengthen impact evaluation at your organization? Explore and apply to one of J-PAL’s upcoming Evaluating Social Programs courses.