Administrative Data and Evaluation Guides

Administrative Data and Evaluation Guides

J-PAL North America has developed a number of resources to make it easier for researchers, policymakers, and practitioners to design randomized evaluations and produce rigorous evidence in the fight against poverty.

Administrative Data

Open drawer of card catalog

Catalog: Administrative Data Sets

A searchable catalog (html) of key US data sets that is designed to assist researchers in screening potential data sources and documents procedures on how to access each data set.

Brief: The Lessons of Administrative Data

A two-page document (pdf) that highlights examples of landmark studies made possible by administrative data.

Guide: Using Administrative Data for Randomized Evaluations

A guide (pdf) that provides practical guidance on how to obtain and use nonpublic administrative data for a randomized evaluation. A poster (pdf) summarizes some of the key takeaways and visuals.

Evaluation Guides

Pen marks off check boxes

Guide: Six Rules of Thumb for Determining Sample Size and Statistical Power

A guide (pdf) for policymakers and practitioners that outlines the main factors that affect statistical power and sample size, and demonstrates how to design a high-powered randomized evaluation.

Guide: Implementing Randomized Evaluations in Government

A guide (pdf) that provides practical guidance for state and local governments on how to identify good opportunities for randomized evaluations, how randomized evaluations can be feasibly embedded into the implementation of a program or policy, and how to overcome some of the common challenges in designing and carrying out randomized evaluations.

Checklist: Administrative Steps for Launching a Randomized Evaluation in the United States

A checklist (pdf) that provides guidance on the logistical and administrative steps that are necessary to launch a randomized evaluation that adheres to legal regulations, follows transparency guidelines required by many academic journals, and complies with security procedures required by regulatory or ethical standards.

Brief: Common Questions and Concerns about Randomized Evaluations

A two-page document (pdf) that addresses concerns potential evaluators may have about the logistical, ethical, and financial implications of running a randomized evaluation.

Brief: The Danger of Underpowered Evaluations

A one-page document (pdf) that highlights the risks associated with running an evaluation that is not designed to detect a meaningful impact of a program.

Brief: Impact Evaluation Methods

A table (pdf) that describes and compares different evaluation methodologies and indicates when each one is valid.

Code: Power Calculations in Stata

A comprehensive tutorial that walks users through how to run parametric and non-parametric power calculations using the statistical software Stata (.zip).

Guide: Real-World Challenges to Randomization and Their Solutions

A guide (pdf) to help researchers and implementing partners develop evaluation designs that fit their program’s context. Using real examples from ongoing and completed randomized evaluations, the document describes multiple research designs that accommodate existing programs, mitigate foreseeable implementation challenges, and demonstrate the flexibility of randomized evaluations across contexts. A poster (pdf) summarizes the key takeaways and visuals.

Brief: Why Randomize

A one-page document (pdf) and video that summarize the rationale behind why randomized evaluations are a powerful way to credibly evaluate the impact of a policy or program.

J-PAL's Research Resources section has a broader set of resources developed both within the J-PAL network and externally. For information and questions, or if you have an idea for a resource you’d like to see, please contact Rohit Naimpally ([email protected]) or Elisabeth O'Toole ([email protected]).