Teaching resources on randomized evaluations
Over the last 15 years, J-PAL has offered its Evaluating Social Programs course in a number of different locations worldwide. This five-day program on evaluating social programs provides a thorough understanding of randomized evaluations and pragmatic step-by-step training for conducting one’s own evaluation. In addition to offering the course in person, J-PAL also offers a free, online version of the course on edX. The teaching materials from these and other J-PAL courses are available in this section.
What is Evaluation?
This lecture will provide an introduction to impact evaluation, from the types of questions we can answer to how we can ensure impact evaluations build on our theories of change.
- What is Evaluation? slides (2019)
Measurement: Outcomes, Impacts, and Indicators
This lecture will explore how we define and measure our key outcomes, types of data and where we can find it, along with potential biases in our data.
In this lecture we present different impact evaluation methodologies, discuss the advantages of randomized evaluations, as well as which factors influence the choice of one impact evaluation method over another.
How to Randomize
This lecture illustrates how the research question determines the randomization strategy and shows how multiple research questions can be answered within one study. In addition, this lecture explores how a number of potential issues and pitfalls can be anticipated at the design stage.
Power and Sample Size
Power and Sample size introduces the concept of statistical power and walks through the factors which influence it. The lecture illustrates how the design choices introduced in How to Randomize, along with the outcomes determined in Measurement: Outcomes, Impacts, and Indicators influence the power of a study.
Exercise on power calculations:
- An exercise that explains the trade-offs in designing a well powered randomized trial, using the EGAP web application.
Threats and Analysis
This lecture illustrates some of the challenges that can occur while conducting a randomized evaluation and how researchers handle these challenges.
- Threats and Analysis slides (2019)
Case studies illustrating how to analyze and interpret the results from a randomized evaluation in the presence of various threats to analysis, such as attrition and non-compliance:
How can results from one context inform policies in another? This lecture will provide a framework for how to apply evidence across contexts.
We thank the Training teams of all offices–past and present–for their assistance in creating these materials. Any errors are our own.