Announcing J-PAL’s newly updated research resources

Posted on:
Authors:
A laptop computer screen displays J-PAL's research resources website.

Today, J-PAL is launching its completely redesigned, expanded, and updated library of research resources

Structured around the lifecycle of a project, these resources now comprise more than 300 pages of new or substantially updated material. They provide practical guidance on conducting RCTs and include templates, checklists, and concrete advice based on J-PAL’s deep experience implementing field research projects. 

These research resources aim to provide rigorous advice for running randomized controlled trials in the social sciences, grounded in many years of practical experience and publicly available for all to use. 

Developing the resources

There are many resources on conducting statistical analyses and basic causal reasoning already available in the form of economics textbooks, course lecture notes, and technical guides. However, easily accessible guidance for research staff, practitioners, and new researchers on the details of designing, funding, and carrying out a field research project from start to finish is much more sparse.

In any given year, J-PAL and our partner Innovations for Poverty Action (IPA) introduce upwards of 100 new research associates as well as many research staff in partner organizations to the ins and outs of empirical research. From newly minted associates to senior directors, almost all of J-PAL’s staff in our seven offices worldwide have carried out field work at one point or will do so in their time at J-PAL; many of our senior staff have worked as PIs (principal investigators) on one or more randomized evaluations.

At any given time, nearly half of J-PAL’s staff are dedicated research staff who are deeply involved in the implementation of randomized evaluations. 

The new research resources distill the collective wisdom accumulated through this work, gathered over many years of experience across hundreds of projects implemented in all regions of the world.

We have sought to create resources that are rigorous and of high quality, and cite all important sources and reference documents. Where authoritative sources are not available, we have drawn on the collective knowledge of experienced J-PAL research staff and internal teaching materials, avoiding sources that are neither “crowd tested” nor peer reviewed. 

A virtual “handbook” for running an RCT

J-PAL’s research resources offer something for everyone, from those new to randomized evaluations to evaluation staff, faculty research assistants, and graduate students in economics and policy programs. The research resources are structured around the lifecycle of a project, creating a “handbook” for running an RCT field research project from start to finish. 

For example, 

  • The resource on surveyor hiring and training includes tips from J-PAL South Asia on keeping surveyors engaged through training;
  • The survey logistics and field team management resources describe best practices on choosing surveyors’ method of transportation, sample protocols to ensure surveyor safety, and considerations for purchasing and tracking survey tablets, compiled by J-PAL South Asia and J-PAL Africa;
  • The resource on randomization provides concrete guidance on programming code to ensure randomization is stable, verifiable, and replicable; 
  • The Institutional Review Board (IRB) resource covers all aspects of writing and submitting an IRB proposal and common concerns of IRBs; and 
  • The data quality checks resource discusses the main considerations J-PAL and IPA staff make when setting up high frequency checks and back checks. 

In addition, these guides are intended to establish a central reference point for J-PAL research policies, best practices, and accumulated knowledge. They complement other excellent resources that have been published in recent years, such as Rachel Glennerster and Kudzai Takavarasha’s Running Randomized Evaluations, the World Bank’s DIME Wiki, EGAP’s Methods Guides, and more.

Many of the resources are based on the lectures and tutorials used to train J-PAL and IPA research staff, and we intend to integrate these resources pages closely with our research staff training going forward.

The overhauled website design reflects the needs of users who would like to gain a quick overview as well as those who are looking for specific guidance, with sidebars to facilitate navigation within and across resources, mouse-over definitions of key terms and footnotes, clearly delineated references and resources for further reading, and, where applicable, sample code snippets that can be pasted directly into the relevant program.   

A collaborative effort

When planning these resources, we have listed authors and contributors, as well as reference sources, following academic standards wherever possible. You can find contributing authors and reviewers on each page on the site; moreover, contributors were invited to credit others who have shared their materials or given feedback. 

In developing this authorship process, our goal was to make these resources citable by others, to achieve the highest quality of contributions, and to give credit to J-PAL’s many knowledgeable and resourceful staff members who typically operate behind the scenes and often wrote these guides while also fulfilling the (more than) full time job of actually conducting randomized evaluations. 

The writing of these resources was a truly collaborative effort: each resource is authored by up to four staff members and reviewed by no fewer than three additional staff members with relevant expertise, with senior research staff providing final input on content. We also thank many J-PAL affiliated researchers who provided valuable input: Shawn Cole, Manashi Deshpande, Rebecca Dizon-Ross, Amy Finkelstein, Ray Kluender, Neale Mahoney, Ben Olken, William Parienté, and Lars Vilhuber.

Looking forward

In addition to creating a process for regular updates, we plan to fully integrate these new research resources with our twice annual Research Staff Trainings in order to make these materials a common point of reference within J-PAL. 

We already have several new resources planned for the coming months, with topics that include remote survey work, monitoring implementation of interventions, and power calculations. Future content will be demand-driven, based on challenges our research staff encounter in the field or those voiced by the research community. 

Every page of the new research resources has a link at the bottom for feedback, and we invite you to send us suggestions or corrections if you find that anything is missing. Please also feel free to send us your thoughts and comments at [email protected].

Visit the resources >>

Posted by Sarah Kopper and Anja Sautmann.