Research Resources

Repository of measurement and survey design resources

Summary

This is a list of resources on measurement and survey design relating to various topics. Topics are organized alphabetically for ease of navigation using the sidebar. We rely on crowd-sourced material to maintain this page and update it regularly. Feedback on the page, or suggestions of resources to add or remove, can be submitted using this form.

General resources

Welcome to the J-PAL Repository of Measurement and Survey Design Resources. The purpose of this repository is to provide an introduction to the measurement and survey design resources available in a particular subject or for a specific type of question. It is a companion piece to our Introduction to measurement and indicators and Survey design pages, compiling resources that discuss and provide guidance on specific issues related to the concepts introduced in those resources. In each section, you will find a list of resources, moving from more general to more specific, that are meant to introduce readers to the measurement tools, difficulties, and solutions in that particular topic. 

These lists do not aspire to comprehensiveness but rather attempt to tackle the main points in measurement within a topic. They cover a variety of media, from blogs and academic papers to books and journal issues. While heavy synthesis is outside the scope of this resource, short descriptions of the papers, as well as introductions to each section written by J-PAL staff with expertise in that area, are provided for the reader’s ease of use.

The General section includes resources that provide an overview of measurement concepts and guidance on survey design. It also includes a few repositories of sample questionnaires and resources on remote surveying in light of the shifts during the Covid-19 pandemic. Some of the following sections are divided by research area, for example, financial inclusion, corruption, and environment & energy. Others, like the forthcoming section on measuring consumption, assets, income, prices, and poverty, cut across several research topics. 

The current repository includes only a portion of the total number of sections, and we will continue to release more sections in phases over the coming months, so please check back for more. 

As with our other research resources, this is meant to be a living resource. If you have feedback on any of the resources in this repository, or if you have recommendations for resources and/or further subjects to add, please fill out this form.

General measurement resources

Practical guides to designing and implementing surveys

Sample questionnaires

This section contains sources for finding sample questionnaires across sectors; topic-specific sample questionnaires can also be found under the relevant topic header. Note that project data is typically stored with the questionnaire and can be found by following the links below.

  • Datahub for Field Experiments in Economics and Public Policy, by J-PAL and IPA; Harvard Dataverse -- Includes survey questionnaires, research results, replication code, and documentation from studies conducted by J-PAL and IPA affiliated researchers. 
  • The World Bank’s Microdata Catalog -- Central cache of questionnaires and data used and produced by the World Bank.
  • Model surveys from the Demographic and Health Surveys Program -- DHS questionnaire modules on household composition, biomarkers, and behavior related to the health of women, men and children.
  • The International Food Policy Research Institute's Database of Household and Community Surveys -- A library of datasets resulting from household and community surveys conducted by IFPRI, many of which are open access and contain the survey questionnaires.
  • International Household Survey Network -- A catalogue of surveys, research data and documentation from the IHSN.

Phone surveys

General guides

Research on response rates and mode effects

Practical survey guides

Covid-specific resources

Corruption in governance and service provision

Overview

Corruption, or when bureaucrats and elected officials misuse their positions or break rules for private gain, is difficult to measure due to its illicit and often secretive nature. Directly observing corrupt activities like bribery by government officials, neglecting official duties, or tax avoidance can be challenging as officials may change or conceal their behavior in response to being monitored. Further, traditional survey approaches are unlikely to elicit truthful responses as officials may be unwilling to confess to corruption due to social desirability bias. Alternative methods of measuring corruption that rely on asking about citizens’ experience with corrupt officials (e.g. “have you paid a bribe for a service before?”) or their perceptions of corruption may be biased, outdated, or incomparable across contexts (Olken 2009). Perception-based indices and rankings may also provide limited insight into the type, causes, or consequences of corruption in a given context (Banerjee, Mullainathan, and Hanna 2012).

Given these challenges, open questions around measuring corruption include: How can researchers measure corruption without distorting public officials’ behavior or eliciting a biased response? What is the best way of measuring social norms around corruption? What types of corruption, if any, can citizen reports shed valuable insights on? Can e-governance reforms that improve the collection of administrative data also improve our ability to measure corruption? More reliable measures of corruption help us better answer policy-relevant questions like the effects of corruption on the efficiency of public service delivery, and the effectiveness of anticorruption policies and programs.

While measuring corruption is difficult, researchers have made remarkable progress in doing so in the past few years, including through the use of a variety of innovative approaches that directly measure corruption and begin tackling some of these questions. This includes:

  • Surveyors accompanying truck drivers on delivery routes, dressed as their assistants, to record bribes paid to police at checkpoints (Olken and Barron 2009)
  • Combining GPS-tracked company vehicle data with administrative data to measure corrupt behavior among bureaucrats of a large public service provider (Schonholzer et al., ongoing)
  • Comparing villagers’ perceptions of corruption to an objective measure (e.g. the difference between government-reported expenditure for a road building project and the estimated cost of actually building the road according to independent engineers; Olken 2009).

The papers that follow include many more examples of methods that can be used to measure corruption in governance and service provision, including through audits, public expenditure tracking surveys, market inference, and more. For a discussion on the different measurement approaches and their applicability, see the MITx Micromasters Course on Political Economy and Economic Development.

- Aimee Barnes, Policy Associate, and Eliza Keller, Senior Policy & Communications Manager, for the J-PAL Political Economy and Governance sector

Datasets

General resources

  • Paper: Eight questions about corruption, by Jakob Svensson (2005) -- Provides a definition of corruption and then discusses the level of corruption in different countries, the different ways to reduce corruption, and the impact of corruption on growth.
  • Paper: Corruption in developing countries, by Benjamin Olken and Rohini Pande (2012) -- A review of the different measurement techniques and the existing evidence. 
  • Paper: Section 4 (measurement), of Corruption, by Abhijit Banerjee, Sendhil Mullainathan, and Rema Hanna (2012) -- A review of different measurement methods and their application in the literature.
  • Book: New advances in experimental research on corruption, edited by Danila Serra, Leonard Wantchekon, R. Mark Isaac, and Douglas A. Norton; Emerald Group Publishing Limited, Vol. 15 (2012) -- Reviews the research on corruption measurement and reduction generated from laboratory and field experiments. [Gated]
  • Paper: Survey techniques to measure and explain corruption, by Ritva Reinikka and Jakob Svensson (2003) -- Reviews the use of Public Expenditure Tracking Surveys (PETS), provider surveys, and enterprise surveys for measuring corruption in education, health, and private businesses.
  • Book: Are you being served? New tools for measuring services delivery, edited by Samia Amin, Jishnu Das, and Markus Goldstein; the World Bank, Vol. 1  (2008) -- Examples of using different methods and tools for measuring public service delivery.
  • Book: Advances in experimental political science, edited by James N. Druckman and Donald P. Green; Cambridge University Press, Vol. 1 (2021) - A comprehensive guide to the next experimental methods, data collection, analysis, and challenges. [Gated]
  • Book: Corruption: What everyone needs to know, by Ray Fisman and Miriam A. Golden; Oxford University Press (2017) - An overview of corruption and its causes and consequences with examples from around the world. [Gated]

International indicators of corruption and governance

Specific approaches to measuring corruption

Through perception
  • Paper: Corruption perception vs corruption reality, by Benjamin Olken (2009) --  Examines the reliability of villagers’ perception of corruption in a project by comparing it with the “missing expenditure.” Missing expenditure is the difference between the actual cost of the project and the individual’s perception. [Gated published version]
  • Paper: Parochial politics: Ethnic preferences and politician corruption, by Abhijit Banerjee and Rohini Pande (2009) -- Uses expert surveys to measure perceptions about how corrupt a candidate is. They report a high correlation between journalist’s perception about the candidate with actual data. 
Through survey estimates of bribes
Through direct observation
By comparing estimated and actual expenditure
From market inference
Using audits
Through other methods

Energy and environment

Overview

Research in environment, energy, and climate change encompasses a range of topics and with them challenges and opportunities in measurement unique to the sector. Some of the topics included in the sector include greenhouse gas emission reductions; measures to help people cope and live with the effects of climate change; pollution reduction and sustainable natural resource management; and access to affordable, reliable, and clean energy sources.

Generating evidence in environment, energy, and climate change is becoming increasingly urgent as emissions increase, global warming progresses, and communities start to feel the effects of a changing climate. Climate change is highly inequitable, with low-income communities being hardest hit by climate and weather shocks, while at the same time having the fewest resources to adapt. To better understand the impacts of programs, technologies, and policies, researchers are exploring new ways of combining different sources of data. Combining remote sensing data, satellite data, or administrative data collected by governments and utility companies with ground-truth and survey data has the potential to unlock insights about the efficacy of climate solutions. Technological innovations in, for example, sensor technology can produce more granular data on air quality and pollution (Khanna, 2000; O’Neill et al., 2003), allowing for new and innovative combinations with data on welfare losses and health. Policies and research on environment and climate change often face challenges in using available data to predict and measure the impacts of environmental shocks - a challenge that is being met with developments in predictive modeling to inform policy and humanitarian interventions.

Lastly, understanding human behavior and household-level measures to face environmental and energy challenges as well as mitigate and adapt to climate change opens up more questions. Researchers tackle these questions by studying incentive structures (Jayachandran et al., 2017; Hanna et al., 2016), effective regulation enforcement and monitoring (Duflo et al., 2013; Ghanem and Zhang, 2014), and energy consumption and conservation behavior (Burgess et al., 2020; Lee et al., 2020).

- Maike Pfeiffer, Policy Associate, and Andrea Cristina Ruiz, Policy Manager, for the J-PAL Environment, Energy, and Climate Change sector

Environment: General resources

The reading lists for MIT’s Environmental policy and economics (Allcott, Spring 2011) and Environmental economics and government responses to market failure (Greenstone, Spring 2005) are good primers for measurement issues in environmental economics. Though the measurement issues are more focused on cost benefit analysis, the Energy economics (Joskow, Spring 2007) course also has a reading list that may be helpful.

  • Book section: Environment modules, by Dale Whittington, in Designing household survey questionnaires for developing countries; World Bank, Volume 2, 5-30 (2000)  -- An overview of measurement issues surrounding indicators relevant for environmental policy, including sections on contingent valuation, measuring resource use, and capturing environmental priorities. Has a particular focus on LSMS methods and includes example questionnaire modules.

Environment: Measuring benefits

General resources
  • Paper: Nonmarket valuation of environmental resources: An interpretive appraisal, by V. Kerry Smith (1993) -- A literature review of the pros and cons of various methods of nonmarket valuation of environmental resources; covers both indirect (e.g., revealed preference) and direct (e.g., WTP surveys) methods. [Gated]
  • Book: The measurement of environmental and resource values, by A. Myrick Freeman III, Joseph A. Herriges, and Catherine L. Kling; RFF Press, Vol. 3 (2014) [direct download link] -- A graduate-school level textbook that covers the valuation of environmental resources. Includes chapters on topics including, but not limited to, hedonic pricing, environmental quality as a factor input, and valuing longevity and health.
Hedonic method

Environment: Measuring bads and costs

Measuring pollution using air quality sensors
Measuring pollution using audits
Measuring deforestation using satellite imagery
Anthropometric measurement of costs

Energy: General resources

  • Book section: Environment modules by Dale Whittington, in Designing household survey questionnaires for developing countries; World Bank, Volume 2, 5-30 (2000)  -- An overview of measurement issues surrounding indicators relevant for environmental policy, including sections on contingent valuation, measuring resource use, and capturing environmental priorities. Has a particular focus on LSMS methods and includes example questionnaire modules.
  • Paper: Does household electrification supercharge economic development?, by Kenneth Lee, Edward Miguel, and Catherine Wolfram (2020) -- Challenges to measuring electrification (and combining measures of electrification).

Energy: Access and use

Energy demand

Financial inclusion

Overview

Financial inclusion, or access by individuals and businesses to quality and affordable financial products and services that meet their needs, is an increasingly common goal among policymakers. Measuring financial inclusion can help policymakers assess the current state of financial inclusion, set goals, and monitor progress towards achieving them. Moreover, research on these topics helps build our understanding of how interventions can support financial inclusion efforts. Data on financial inclusion has grown and improved in quality over the past decade. 

To measure financial inclusion, researchers usually collect data from users (demand-side) or providers (supply-side). On the demand-side, surveys can capture an individual’s, household’s, or businesses' access to and use of financial services. One challenge with surveys is that it relies on self-reported data: respondents may not remember some financial decisions and may not want to provide accurate responses given the sensitivities of talking about one’s finances. On the supply-side, researchers can use administrative data from financial service providers or regulators. However, one limitation is that data from a single financial institution does not provide a full picture of an individual’s financial behavior, particularly if they use multiple institutions or do not use one at all, a case which is particularly common for low-income individuals. Finally, there are efforts to develop indices that capture the many dimensions of financial inclusion and aggregate measures of financial inclusion at a macro-level. 

This section discusses various approaches to measuring financial inclusion, including by using  survey data, using administrative or non-survey data, constructing indices of financial inclusion, and by using aggregate/macroeconomic measures of financial inclusion. 

- Mikaela Rabb and Sam Carter, former Senior Policy Associates for the J-PAL Finance sector

General resources

Using survey/administrative data

Using survey data
Using administrative/non-survey data

Constructing indices or using aggregate measures

Constructing indices
Aggregate/macroeconomic measures

Health

Overview

Field experiments in health economics help to answer a variety of questions related to the take-up and delivery of health products and services. From helping to better determine the factors that motivate individuals to adopt healthy behaviors to identifying innovations that improve the delivery of health services, this type of research is an important input to strengthening health systems and improving health outcomes around the world. Accurately measuring baseline, intermediate, and final health outcomes is a critical component of determining whether a given policy or program was effective. Some metrics, such as HIV prevalence, can be measured through relatively straightforward tests. But other outcomes are trickier to measure. For instance, child malnutrition is a key predictor of mortality. What is the best measure of malnutrition rates? Height-for-age, weight-for-height, mid-arm circumference, iron deficiency anemia, and more can all be appropriate in certain situations. Which one should a researcher choose given the context and their research questions? Use of modern contraceptives is an important measure of fertility, but respondents may be tempted to report regular use, even if this is untrue, if they feel they should be using them. How can researchers avoid this type of desirability bias?

This section, categorized according to outcomes and health conditions, compiles resources to guide researchers through these and other health measurement challenges. Produced by experts including the World Health Organization, UNICEF, and pioneering researchers in the field, these resources range from survey design guides to best practices for measuring tricky outcomes. In instances where multiple metrics may be appropriate, they also provide suggestions on how to help determine the best indicator(s).

- Anupama Dathan, Policy Manager for the J-PAL Health sector

General resources

  • Questionnaire: Model surveys from the Demographic and Health Surveys program -- Provides a high-level overview of the DHS’s four main questionnaires (Man, Woman, Household, and Biomarker), and provides links to current and past modules.
  • Book section: Health modules by Paul Gertler, Elaina Rose, and Paul Glewwe, in Designing household survey questionnaires for developing countries; World Bank, Volume 1: 177-216 (2000) -- An overview of indicators relevant for health policy, a discussion of survey methods used to capture those indicators, and annotated example questionnaire modules. Has a particular focus on LSMS methods.
  • Online course: J-PAL’s Measuring health outcomes in field surveys course -- Contains lectures and interactive material on all aspects of measuring health outcomes in field surveys: measuring individual and population health, selecting health indicators and measurement tools, questionnaire development, and practical and ethical issues for data collection.
  • Paper: The impact of recall periods on reported morbidity and health seeking behavior, by Jishnu Das, Jeffrey Hammer, and Carolina Sánchez-Paramo (2012). -- An experimental comparison of different recall periods on different reported health outcomes, including morbidity, doctor visits, time spent sick, and use of self-medication. Includes an exploration of the effects among different subgroups of the sample. [Gated published version]
  • Blog post: Quantifying the Hawthorne effect, by Jed Friedman and Brinda Gokul (2014) -- A compilation and short literature review of papers attempting to quantify the Hawthorne effect in health studies.

Health indicators

Conventional indicators
Composite measures
Anthropometric data
Early childhood development (general)
Early childhood development (cognitive)
  • Journal issue: ScienceDirect’s collection of articles on the Bayley-III Scale -- A collection of journal articles on the Bayley-III Scale, an instrument designed to assess the developmental functioning of infants, toddlers, and young children aged between 1 and 42 months; contains articles on the individual scales that make up the Bayley-III, as well as an international review of research that employs it and reviews of similar developmental tools.
  • Questionnaire: Tools from the U.S. Bureau of Labor Statistics:
    • The Peabody Picture Vocabulary Test -- An overview of the Peabody Picture Vocabulary Test, which measures verbal ability and scholastic aptitude for individuals 2.5-40 years of age. Includes links to the instrument itself, its technical report, and similar cognitive development tools. 
    • The Home Observation for Measurement of the Environment (HOME) -- An overview of the Home Observation for Measurement of the Environment module, which measures the quality of a child’s home environment. Includes links to the instrument itself, its technical report, and similar tools from the BLS.
Early childhood development (physical)
Nutrition
Sexual and Reproductive Health

Healthcare quality

Healthcare quality/patient satisfaction
Using audits and mystery shoppers

Housing stability and homelessness

Overview

Housing instability is both a function of and a catalyst for poverty. Maintaining stable housing is a necessary prerequisite in many cases for health, employment, education, and a host of other fundamental needs. The scope and complexity of housing instability and homelessness highlight the need for rigorous evidence on the effectiveness of strategies to prevent and reduce homelessness. A first step in generating this evidence is defining and measuring homelessness and housing instability adequately. 

Unfortunately, the measurement of housing instability is complicated by the existence of a variety of definitions and no widely established measurement system of it. For instance, in the United States, children who share housing with others (living “doubled up”) qualify for assistance under some programs, but not others. Moreover, the scope of people experiencing homelessness can vary by orders of magnitude depending on which definition one uses; including children who are living in doubled up conditions increases estimates of the number of children experiencing homelessness by a factor of 10 from the standard “point in time” (PIT) count. An emerging literature looks at how to measure housing stability using techniques for reaching mobile populations and consumer reference data (e.g. Phillips (2020)Kalton (2001))

Further challenges to measurement come with measuring people who are unsheltered (those sleeping outside or in places not meant for human habitation), typically part of the PIT count in the United States; some studies have found that PIT counts can understate the rate of unsheltered homelessness by as much as 50 percent (e.g. Evans, Phillips, and Ruffini (2019))

The resources included below cover survey and administrative data methods for counting people experiencing housing instability and homelessness, covering topics from oversampling to ensure adequate representation of minority groups through methods for including hard-to-reach subpopulations. A reflection of J-PAL’s internal expertise, the resources below center around housing instability and homelessness in the United States; we welcome suggestions for additional resources to include, particularly those based in or relevant to other countries.

- Rohit Naimpally, Senior Research and Policy Manager, for J-PAL's Reducing and Preventing Homelessness Initiative

General resources

Using administrative data

Survey methods

Capture-recapture methods (plant capture, mark-recapture etc.)

Last updated August 2021. These resources are a collaborative effort. If you notice a bug or have a suggestion for additional content, please fill out this form.

Acknowledgments

We thank Aimee Barnes, Sarah BaumSam CarterAnupama DathanMaya Duru, Sarah Gault, Nilmini Herath, Eliza KellerTithee Mukhopadhyay, Rohit NaimpallyWilliam Pariente, Maike PfeifferMikaela Rabb, Andrea Cristina Ruiz, and Caroline Tangoren for helpful review and comments, and Manvi Govil and Ximena Mercado Garcia for their help copy-editing the resource. Any errors are our own.

In this resource