Leveraging Evaluation and Evidence for Equitable Recovery (LEVER) Evaluation Incubator

Overview

J-PAL North America's LEVER Evaluation Incubator supports state and local government agencies seeking to generate rigorous evidence on their programs or policies via randomized evaluations. Through the LEVER Evaluation Incubator, governments and service providers apply for pro-bono training and technical assistance, flexible funding, and connections to expert researchers to develop their evaluations. 


Applications across all policy areas are welcome with a focus on economic mobility or racial equity. This year, we especially encourage applications from state and local government agencies that are currently working closely with community organizations, leaders, or stakeholders, or are seeking to collaborate with community members and groups on an evaluation. See the FAQs below for more information on how we define community engagement in research and evaluation. 

How does the Incubator work?

Selected state and local agencies work with J-PAL North America staff to design and determine the feasibility of a randomized evaluation and develop a partnership with an academic researcher to later conduct this evaluation. 

Selected agencies may receive:

  • Up to four months of technical support from J-PAL North America and Results for America staff of policy and research experts to design a randomized evaluation. J-PAL North America and Results for America staff will provide technical assistance including project management, research support, and training as needed to help partners develop feasible and policy-relevant opportunities for randomized evaluations.

  • Flexible project development funding of up to $50,000 to help design an evaluation. Examples of costs that may be covered include personnel costs and travel costs to visit implementation sites or meet with stakeholders. Funds may not be spent on the implementation of the program/policy or randomized evaluation itself. 

  • Training on program evaluation, evidence use, and data.

  • Connections with J-PAL’s network of leading academic researchers. J-PAL staff will make an ongoing effort to provide introductions to researchers to explore potential research partnerships. The goal is to identify a researcher in our network who can lead the evaluation.

  • A peer convening that supports relationship building with other leading jurisdictions that are committed to using data and evidence. 

  • For jurisdictions who are well positioned, J-PAL North America will equip state and local government agencies to develop community engagement strategies as part of their randomized evaluation plan. This could include connecting projects with researchers who have community-engagement expertise.

Application

Submit your letter of interest today »

The Incubator application process is rolling and designed for us to be in conversation with you as you determine whether this program is the right fit for your team. Review the LOI instructions at the bottom of this webpage, submit your letter of interest early and be in touch with us often! 

 Once you submit your letter of interest, our team will reach out to schedule a coaching call to:

  • Discuss your submission and viability for randomized evaluation;

  • Preview the full proposal materials;

  • Determine any next steps or additional support you may need in submitting the full proposal. 

Applicants have until April 15, 2024 to submit their letters of interest. We encourage you to submit as soon as possible to allow for ample coaching and preparation in submitting your full proposal. Feel free to reach out to [email protected] at any time. We appreciate hearing from applicants and welcome any questions! 

Evaluation Incubator Timeline

Phase 1

February 6: Call for Letters of Interest (LOIs) opens

February - April: Once you submit your LOI, our team will schedule a coaching call with you to prepare for the full proposal submission. We will share full proposal materials with you at this time. 

April 15: Deadline for Letters of Interest 

Phase 2

July 1: Deadline to submit full proposals 

Beginning of August: Incubator partners are selected

September - December: Selected partners receive technical assistance, connections to researchers, and funding (if awarded) to advance their evaluation ideas

Image removed.Submit your letter of interest today »

FAQs

Who is eligible?

The LEVER Evaluation Incubator welcomes applications from US state and local government agencies interested in evaluating programs and policies that advance economic mobility and racial equity in their communities. See a list of past and current J-PAL North America State and Local Evaluation Incubator partners here

Applicants must be open to working with J-PAL North America to look for ways to include random assignment in the implementation of the program or policy they are interested in evaluating. For example, slots for a program with a waitlist might be allocated by a lottery; eligible beneficiaries of an undersubscribed program may be randomly encouraged to sign up; multiple versions of a program may be randomly assigned to individuals to test them against one another, and pilot programs or programs being expanded to new locations could provide opportunities to randomly select who receives the program first. 

How do I know if the Incubator is the right fit for my team?

While each engagement looks different, we find that agencies with the following are typically the most prepared for the Incubator: 

  • A specific policy or program you are interested in evaluating. Programs and policies that tend to be conducive to randomized evaluation include pilot programs, new programs before scale-up, programs introducing a new component, and programs expanding to a new population; new policies being implemented; and relatively new policy initiatives where data are needed to assess impact. 
  • An interest in using randomized evaluation to assess the impact of a program or policy
  • A general idea of how randomization could work for your program and inform policy or program improvement. 
  • An initial understanding of the priority outcomes your team wishes to evaluate. 
  • The number of people targeted by a policy, or number of participants eligible for the program, including those who do and do not currently receive the program. 
  • Motivation to build on established relationships to plan an evaluation that will substantively engage community residents, groups, and/or organizations.

What type of assistance does J-PAL North America provide for applicants?

After you submit your letter of interest, we will schedule a coaching call to discuss your submission in more detail and identify areas of support in preparation for submitting your full proposal. In addition to this support, you can:

Who will review my application?

In phase 1, Letters of Interest will be reviewed by J-PAL North America staff. In phase 2, a Review Board composed of J-PAL affiliated researchers Bruce Sacerdote and Day Manoli, as well as Vincent Quan, co-executive director of J-PAL North America, will review full proposals.

How are LEVER's Evaluation Incubators different from LEVER's Training Sprints?

While LEVER’s Evaluation Incubators empower agencies to take action on a specific evaluation idea, Training Sprints will equip agencies to build internal capacity, culture, and processes to develop sustainable evaluation practices, paving the way for future evidence use and creation.

During the spring 2024 Training Sprint, 10-15 teams from different jurisdictions around the country will be selected to learn together in weekly virtual sessions across ten weeks. The Training Sprint will help state and local government leaders incorporate best practices for evidence-based decision-making into their current processes so that the recovery funding evidence and evaluation standards for policy solutions become a long term practice. ​​Participants will:

  • Develop their own policy for how evidence, evaluation and equity are prioritized in their jurisdictions. 

  • Build a decision-making framework that makes their policy actionable and sustainable so that they consistently incorporate equity, evidence use and evaluations into their practices and initiatives. 

Why randomized evaluation?

J-PAL North America specializes in randomized evaluation, the most rigorous type of impact evaluation. Randomized evaluation can determine the causal impact of a program by comparing the outcomes of a group that receives a program (individuals, communities,  schools, etc.) against a group that does not. Because individuals are randomly assigned to either the program or comparison group, we can be confident that the individuals do not differ systematically at the beginning of the study. We can therefore confidently attribute any difference in outcomes after the program to the intervention itself.


Treasury’s Economic Recovery learning agenda has identified the importance of understanding  short-term and long-term impact on households, organizations, communities, and governments from specific American Rescue Plan State and Local Fiscal Recovery Funds (SLFRF) projects in priority policy areas such as affordable housing, workforce, and public safety. Randomized evaluation is a critical tool that states and localities can use to advance knowledge and inform future decision-making in these areas.

What does it mean to incorporate community engagement into my planned evaluation?

Community engagement in research and evaluation encompasses a range of approaches for working with and involving residents, leaders, groups/organizations, and stakeholders along the research continuum, from conceptualization to dissemination and application. Engaged research is designed to ensure projects reflect the perspectives, strengths, and priorities of community members. Ideally, projects are built on mutually beneficial and reciprocal relationships that center community leadership in decision-making, therefore providing a structure for inclusive and equitable assessments, analyses, and policy recommendations. Given the complexity and dynamic nature of communities, we are looking for projects that intentionally define what constitutes ‘community’ and ‘engagement’ in the context of an evaluation project.

What are the benefits of incorporating community engagement into research and evaluation?

By nurturing mutually beneficial, reciprocal relationships grounded in trust, a community engaged approach can contribute to the reliability of research processes and data collection. This, in turn, enhances internal and external validity, leading to more robust research with increased community buy-in and equity.

By collaborating with community members on evaluation projects, states and localities can:

  • Ensure equitable representation of people on the frontlines who are directly impacted by programs and policy decisions. 

  • Bring rigor to the research by developing questions, analyses, and data collection strategies that accurately reflect community dynamics and are responsive to community priorities. 

For additional guidance, please reference the following: J-PAL blog post on the value of centering lived experience in the research process, Urban Institute’s Community Engagement Resource Center, and Durham University’s guide to ethical principles and practices in Community Based Participatory Research.

Letter of Interest Instructions

Page Content