In its most simple sense, randomization is what happens when a coin is flipped, a die is cast, or a name on a piece of paper is drawn blindly from a basket, and the outcome of that flip, cast, or draw determines what happens next. Perhaps, the outcome of the coin flip determines who has to do some chore; the role of the die determines who gets a pile of money; the draw of a name determines who gets to participate in some activity, or a survey. When these tools (the coin, the die, the lottery) are used to make decisions, the outcome can said to be left to chance, or, randomized.
Why do people let chance determine their fate? Sometimes, because they perceive it as fair. Other times, because uncertainty adds an element of excitement. Statisticians use randomization because, when enough people are randomly chosen to participate in a survey, conveniently, the attributes of those chosen individuals are representative of the entire group from which they were chosen. In other words, what is discovered about them is probably true about the larger group. Using a lottery to get a representative sample is known as random sampling or random selection.
When two groups are randomly selected from the same population, they both represent the larger group. They are not only statistically equivalent to the larger group; they are also statistically equivalent to each other. The same logic carries forward if more than two groups are randomly selected. When two or more groups are selected in this way, we can say that individuals have been randomly assigned to groups. This is called random assignment. (Random assignment is also the appropriate term when all individuals from the larger group divided randomly into different groups. As before, all groups represent the larger group and are statistically equivalent to each other.) Random assignment is the key element of randomized evaluation.
What happens next in a simple randomized evaluation (with two groups) is that one group receives the program that is being evaluated and the other does not. If we were to evaluate a water purification program using this method, we would randomly assign individuals to two groups. At the beginning, the two groups would be statistically equivalent (and are expected to have equivalent trajectories going forward). But then we introduce something that makes them different. One group would receive the water purification program and the other would not. Then, after some time, we could measure the relative health of individuals in the two groups. Because they were statistically equivalent at the beginning, any differences seen later on can be attributed to one having been given the water purification program, and the other not.
Why this method is used is covered in the Why Randomize section.
Randomized Evaluations go by many names
- Randomized Controlled Trials
- Social Experiments
- Random Assignment Studies
- Randomized Field Trials
- Randomized Controlled Experiments
Randomized Evaluations are part of a larger set of evaluations called Impact Evaluations. Randomized evaluations are often deemed the gold standard of impact evaluation, because they consistently produce the most accurate results.
Like all impact evaluations, the primary purpose of randomized evaluations is to determine whether a program has an impact, and more specifically, to quantify how large that impact is. Impact evaluations measure program effectiveness typically by comparing outcomes of those (individuals, communities, schools, etc) who participated in the program against those who did not participate. There are many methods of doing this.
What distinguishes randomized evaluations from other non-randomized impact evaluations is that participation (and non-participation) is determined randomly—before the program begins. This random assignment is the method used in clinical trials to determine who gets a drug versus who gets a placebo when testing the effectiveness (and side-effects) of new drugs. As with clinical trials, those in the impact evaluation who were randomly assigned to the “treatment group” are eligible to receive the treatment (i.e. the program). And they are compared to those who were randomly assigned to the “control group” –those who do not receive the program. Because members of the groups (treatment and control) do not differ systematically at the outset of the experiment, any difference that subsequently arises between them can be attributed to the treatment rather than to other factors. Relative to results from non-randomized evaluations, results from randomized evaluations are:
- Less subject to methodological debates
- Easier to convey
- More likely to be convincing to program funders and/or policymakers
Beyond quantifying the intended outcomes caused by a program, randomized evaluations can also quantify the occurrence of unintended side-effects (good or bad). And like other methods of impact evaluation, randomized evaluations can also shed light on why the program has or fails to have the desired impact.
1. Randomization In the Context of “Evaluation”
Randomized evaluations are a type of impact evaluation that use a specific methodology for creating a comparison group—in particular, the methodology of random assignment. Impact evaluations are program evaluations that focus on measuring the final goals or outcomes of a program. There are many types of evaluations that can be relevant to programs—beyond simply measuring effectiveness. (See What is Evaluation?)
2. Methodology of Randomization
To better understand how the methodology works, see How to conduct a randomized evaluation.