Teaching resources on randomized evaluations
Over the last 15 years, J-PAL has offered its Evaluating Social Programs course in a number of different locations worldwide. This five-day program on evaluating social programs provides a thorough understanding of randomized evaluations and pragmatic step-by-step training for conducting one’s own evaluation. In addition to offering the course in person, J-PAL also offers a free, online version of the course. The teaching materials from these and other J-PAL courses are available in this section.
What is Evaluation?
This lecture will provide an introduction to impact evaluation, from the types of questions we can answer to how we can ensure impact evaluations build on our theories of change.
- What is Evaluation? slides (2019)
Measurement: Outcomes, Impacts, and Indicators
This lecture will explore how we define and measure our key outcomes, types of data and where we can find it, along with potential biases in our data.
Measurement slides (2019)
Case studies on developing a theory of change and identifying appropriate outcomes and indicators:
- Cognitive Behavioral Therapy in the US (Case Study)
- Village Financial Services in India (Case Study)
In this lecture we present different impact evaluation methodologies, discuss the advantages of randomized evaluations, as well as which factors influence the choice of one impact evaluation method over another.
- Why Randomize? slides (2019)
Case studies illustrating the benefits of random assignment, compared to other identification strategies:
- Get Out the Vote in the US (Case Study)
- Vocational Training in Colombia (Case Study)
How to Randomize
This lecture illustrates how the research question determines the randomization strategy and shows how multiple research questions can be answered within one study. In addition, this lecture explores how a number of potential issues and pitfalls can be anticipated at the design stage.
- How to Randomize slides (2019)
Case studies illustrating how the randomization design of an evaluation can be tailored based on the research question(s):
- Extra Teacher Program in Kenya (Case Study)
- Labor Displacement from Job Counseling Programs in France (Case Study)
Power and Sample Size
Power and Sample size introduces the concept of statistical power and walks through the factors which influence it. The lecture illustrates how the design choices introduced in How to Randomize, along with the outcomes determined in Measurement: Outcomes, Impacts, and Indicators influence the power of a study.
Exercise on power calculations:
Threats and Analysis
This lecture illustrates some of the challenges that can occur while conducting a randomized evaluation and how researchers handle these challenges.
- Threats and Analysis slides (2019)
Case studies illustrating how to analyze and interpret the results from a randomized evaluation in the presence of various threats to analysis, such as attrition and non-compliance:
- Deworming Program in Kenya (Case Study)
- Training and Wage Subsidies in Jordan (Case Study)
How can results from one context inform policies in another? This lecture will provide a framework for how to apply evidence across contexts.
- Generalizability slides (2019)
Please note that the teaching resources referenced here were curated for specific research and training needs and are made available for informational purposes only. Please email us for more information.
Last updated July 2020.
These resources are a collaborative effort. If you notice a bug or have a suggestion for additional content, please fill out this form.
We thank the Training teams past and present of all J-PAL offices for their assistance in creating these materials. Any errors are our own.