Updates on Evidence-Based Policymaking in the United States
In September 2017, the US Commission on Evidence-Based Policymaking released its final report on improving the use of data to evaluate the effectiveness of government programs. The bipartisan Commission—supported by Republican House Speaker Paul Ryan and Democratic Senator Patty Murray and signed into law by President Obama in March 2016—makes promising recommendations to improve access to secure data and move toward a future in which rigorous evidence is generated as a routine part of government operations. If acted on, the Commission’s report could translate to new opportunities for academic researchers to conduct randomized controlled trials using government data.
The Commission held public hearings across the country, gathering testimony and public comments. J-PAL North America submitted a public comment, and former J-PAL North America Executive Director Quentin Palfrey and current Executive Director Mary Ann Bates both testified before the Commission (view their statements here and here).
Several of the Commission’s recommendations are particularly encouraging for academic researchers. First, the Commission emphasizes that evaluation should be embedded into program design. Programs, policies, and regulations are typically designed without input from academics and evaluators. As a result, data crucial to evaluating the impact of a policy may not be collected or accessible and programs are often rolled out in a way that makes rigorous evaluation difficult. Considering evaluation and data needs in program design can ensure that data are collected and analyzed in a way that enhances a program’s ability to implement day-to-day operations and achieve its intended goals in a cost-effective manner.
In particular, the Commission recommends establishing sufficient legislative authority or flexibility to enable impact evaluations, using random assignment when appropriate. This section of the report specifically cites input from J-PAL North America, which noted the importance of identifying opportunities for randomized evaluation, which generates high-quality evidence that identifies the causal impacts of programs or policies. For example, when there aren’t enough resources to serve everyone, scarce resources can be allocated by lottery. Randomly assigning program slots is often the fairest and most transparent way to select individuals off a waitlist (rather than first come, first served, which can favor those with the most resources), and creates an opportunity to rigorously evaluate the program’s impact.
Second, the Commission calls on Congress and the president to establish a novel National Secure Data Service (NSDS), charged with facilitating data access for evidence building while ensuring transparency, security, and privacy. One of the NSDS’ primary responsibilities would be to enable secure, temporary linkages across government datasets for use by qualified researchers in specific, pre-approved projects. Potential datasets to be offered through NSDS include wage data from Unemployment Insurance records and data from programs administered by states with substantial federal investment, such as the Supplemental Nutrition Assistance Program (SNAP) and the Temporary Assistance for Needy Families program (TANF). Importantly, the NSDS will operate as a service, not a clearinghouse with permanent open access to data. This approach is designed to strike a balance between improved access to data and strong privacy protections.
The Commission also recommends developing a uniform process for external researchers to apply and qualify for secure access to confidential government data for evidence-building purposes. They further call on the NSDS to make more information available about the government’s current data inventories and to supply documentation to help researchers inside and outside government better understand the existing data, allowing them to identify and request data needed for a particular evaluation.
As the members of the Commission rightly point out, “Generating and using evidence to inform government policymaking and program administration is not a partisan issue.” Speaker Ryan and Senator Murray have already signaled their intent to translate the report into meaningful legislation. The onus of implementation will also fall on the executive branch, particularly the Office of Management and Budget. Meanwhile, momentum for evidence-based policymaking continues to build among localities and states (which collectively account for 40 percent of total government spending in the United States), with more governments demonstrating interest each year in our State and Local Innovation Initiative, which supports governments as they develop randomized evaluations to inform critical policy questions.
The members of the Commission will be holding a number of events to take questions and hear from stakeholders and the public regarding their report this fall. J-PAL North America staff plan to attend several. If you have a question you would like us to ask the Commission, please email Hannah Myers ([email protected]) and Emma Rackstraw ([email protected]).