State and local policy responses to COVID-19: Lessons from evidence
To learn more about the evidence below or to discuss how to apply evidence from J-PAL North America’s randomized evaluations in your context, contact Rohit Naimpally.
To download this note in pdf form, click here.
The COVID-19 pandemic has caused enormous health, social, and economic challenges. J-PAL North America has curated a set of randomized evaluations that tested specific policy options related to the social safety net, health, and education. Our goal is to provide actionable evidence to inform state and local leaders’ policy responses to challenges during the COVID-19 pandemic. Please note that we do not have research evidence specifically for responding to a pandemic. We generalize from existing evidence in different contexts to provide policy recommendations and considerations for state and local governments responding to this situation.
Via our State and Local Innovation Initiative, we are able to provide pro bono technical assistance to state, local, or tribal leaders who would like to tailor and apply evidence in their local context. Our team can help you analyze whether evidence is likely to be relevant in your context and provide implementation details from the original studies. Please contact Rohit Naimpally to learn more.
Increasing access to the social safety net
The federal government expanded social insurance programs in response to the COVID-19 pandemic, but expansion of these programs is not enough to ensure eligible individuals claim these benefits. State and local governments can implement evidence-based policies to ease barriers to access important social insurance programs like the Supplemental Nutrition Assistance Program (SNAP), Earned Income Tax Credit (EITC), and Temporary Assistance for Needy Families (TANF). This section provides lessons from evidence on increasing access and take-up of social benefit programs.
Provide salient information and reminders to increase the take-up of critical social safety net programs.
Many individuals may not access the critical social benefits, such as the EITC, that they are eligible for during this time. The 2019 tax filing deadline extension from April 15th to July 15th due to COVID-19 gives states an opportunity to encourage individuals who could be eligible for the EITC to file their taxes.
A randomized evaluation from Day Manoli and Saurabh Bhargava, in partnership with the US Internal Revenue Service tested the effectiveness of different messages to taxpayers who were eligible for, but did not claim, the Earned Income Tax Credit and found that targeted messages increased take-up of the EITC.
Social benefit programs like SNAP and TANF are also crucial for vulnerable families and individuals, particularly during economic downturns. Yet millions of SNAP-eligible households do not enroll and miss out on assistance that can help address food insecurity. A randomized evaluation from researchers Amy Finkelstein and Matt Notowididgo, in partnership with Benefits Data Trust, found that sending eligible households informational mailings nearly doubled SNAP enrollment, while combining these informational mailings with application assistance tripled enrollment.
It is worth noting that sometimes text messages and/or letters alone are not enough to increase take-up of social insurance programs and tax credits. A recent evaluation by the California Policy Lab found that text messages and letters alone were not enough to increase the take-up of the state and federal EITCs. If possible, states should simplify enrollment processes while also providing targeted outreach to increase take-up of social insurance and tax credits.
- Recommendation: State and local governments can implement mailings or text messages that provide relevant program information for those who may be eligible for benefits, provide assistance, and simplify application processes to increase the likelihood of eligible participants claiming benefits.
- For consideration: States can streamline enrollment for programs like SNAP, TANF, and Unemployment Insurance by implementing presumptive enrollment, or the ability to enroll those who appear eligible, rather than requiring a full application. This mechanism can ease administrative barriers and provide expedited support to those who need it.
For more on this topic, please see J-PAL North America Scientific Director Amy Finkelstein and J-PAL North America Work of the Future Initiative co-chair Matt Notowidigdo’s op-ed on easing access to the safety net, or contact Rohit Naimpally.
Increasing access to health insurance and care during a pandemic
Without a tested vaccine or identified cure, preventive measures and access to medical treatment are critical to combat the spread of COVID-19. Public health authorities looking to ensure residents are covered by health insurance and able to seek medical care can look to the following evidence-based recommendations for guidance.
Reducing barriers to information and enrollment can increase insurance take-up.
In response to the coronavirus pandemic, several states have extended or re-opened the enrollment period for health insurance on state health insurance exchanges. However, despite insurance being available, many eligible Americans remain uninsured. People who are uninsured will likely face barriers to accessing testing and treatment for COVID-19 symptoms. In an effort to mitigate the costs of seeking COVID-19 testing and treatment, the CARES Act expanded the number of tests and services insurers must cover at no cost. However, there are gaps in these protections that may expose patients to unexpected medical bills, and insurance can help mitigate these costs. Experts found that informational interventions improved take-up of plans.
- Recommendation: Letters advertising enrollment deadlines and personalized information such as projected subsidies and plan ratings can increase take-up of insurance, even among healthier individuals. Increased take-up of health insurance will not only allow more Americans to access and pay for healthcare, but also decrease insurance costs in the long-run. The evidence also suggests that simplifying enrollment processes—such as through reminder emails with easy links to enrollment or auto-enrollment—could increase enrollment, more than through subsidies alone.
- For consideration: Under the National Emergencies Act, states have been granted the option to expand Medicaid coverage and reduce barriers to enrollment by applying for Medicaid Disaster Relief State Plan Amendments. One strategy to streamline enrollment is to implement presumptive eligibility, thus allowing hospitals, clinics, and other entities to screen individuals for Medicaid eligibility, and to temporarily enroll those who appear eligible.
Simple nudges and reminders can induce people to take up Medicaid.
Lack of insurance coverage during a global pandemic will cause even greater financial strain on low-income populations. Despite access to free or significantly subsidized health insurance through the Medicaid program, many eligible individuals and families remain unenrolled. A study of low-cost improvements to outreach efforts to Medicaid-eligible residents found these nudges to be particularly effective among hard-to-reach vulnerable populations. It was especially beneficial if outreach efforts were coordinated for times of the month when low-income populations were likely to be responsive.
- Recommendation: Behaviorally-informed nudges with specific next steps, such as postcards, emails, and automated calls reminding people of upcoming deadlines, can increase enrollment in the Medicaid program. Given that many states have authorized shelter-in-place and other stay-at-home orders, letters and other forms of electronic or phone communication will be critical to remind residents of key deadlines and key steps for securing Medicaid coverage.
For more on this topic, please see J-PAL’s note on “Increasing Adherence to COVID-19 Guidelines: Lessons from Existing Evidence.” Policymakers interested in learning more on the evidence presented on health insurance take-up and access are encouraged to contact Hannah Reuter.
Improving virtual and online learning while schools are closed
The physical closing of schools is among the unprecedented COVID-19 disruptions across the United States. Many districts and schools have rapidly adapted by virtualizing learning for their students. It is important to note that the use of many education technology solutions to deliver lessons hinges upon students’ access to reliable internet and internet-enabled devices. While internet access alone is not sufficient to improve learning outcomes, it is in many cases necessary to utilize effective computer-assisted learning programs. If these students are unable to access this necessary technology, the achievement gap will likely widen.
In this section we describe evidence-based approaches that can support learning via education technology informed by a J-PAL North America review of 126 randomized evaluations examining the effectiveness of many different kinds of education technology interventions. Recommendations on tutoring programs are drawn from a forthcoming J-PAL review of 96 randomized evaluations of K-12 tutoring programs.
Educational software, or computer-assisted learning (CAL) programs, shows promise in improving learning outcomes, particularly in math.
Under shelter-in-place and other orders to stay home, students will likely receive classroom instruction and any supplemental exercises online. Effective CAL programs that have been rigorously evaluated share a few common features. Students can advance through exercises at their own pace, allowing them to work incrementally until they have mastered the material. Evidence suggests that simply watching videos is insufficient, no matter how stimulating the content. School leaders should provide educators with guidance on monitoring and rewarding progress for advancing through instructional videos.
- Recommendation: Educational software that adapts instruction to meet student needs or that offers timely feedback to students and reports data on student performance to teachers can support teacher instruction at this time. For example, ASSISTments, a free online math homework platform that offers students feedback as they solve problems, has been evaluated through multiple randomized evaluations and shown to have positive results even when students used the program for less than an average of ten minutes per night, three to four nights per week.
Online learning is more productive when platforms create interactive lessons and offer incentives to learn.
Students in online only courses tend to perform worse than students in in-person-only courses. Studies of Massive Open Online Courses (MOOCs) show that persistence is low and few students get past the first few lessons. It is possible that students taking online courses may struggle with the lack of accountability or miss out on motivating relationships with instructors and peers.
- Recommendation: Engage students in video and online learning by creating assignments that ask students to engage with material and share their reactions through online discussions to increase the likelihood that students process the material.
Tutoring programs have shown promise in improving learning outcomes for disadvantaged students.
Evidence-based education non-profits, including Saga Education, are now delivering their tutoring model virtually to support at-home learning. Saga Education partners with public school districts to provide personalized tutoring during the school day for students who are falling behind. Randomized evaluations of Saga’s model have shown that the program results in large academic gains for students. Saga is offering virtual tutoring services to students in New York City while schools are closed. While this online implementation has not been evaluated, our assessments of SAGA’s approach and evaluations of tutoring-style programs delivered virtually suggest that these supports could be valuable to students who are struggling academically.
- Recommendation: Tutoring programs offering two-to-one support can help students get back on track when schools reopen and will help fight the widening of the achievement gap that is likely to result from school closures.
- For consideration: Evaluations of summer youth employment programs have found these programs offer considerable promise to alleviate the social costs of youth violence. Several cities have been forced to cancel these programs due to COVID-19. While no formal evaluation has been done on the topic, districts might consider partnering with summer youth employment programs to offer youth jobs as virtual two-to-one tutors for younger students needing remedial support during the summer months. For more on summer youth employment programs, see J-PAL North America’s Policy Bulletin: Stopping a Bullet with A Summer Job.
Messaging platforms and behavioral nudges can support parents and students.
Evidence suggests that with short, actionable directions and suggestions for engaging activities, parents are better equipped to support learning at home. Evidence-based family engagement apps are a proven method of providing parents with these behavioral nudges. There is also evidence that suggests timely and specific reminders, like text message reminders about tasks required for matriculation can help students enroll in college at higher rates. Behavioral nudges delivered through text messages may be an inexpensive way for districts to support both parents and students during school closures.
- Recommendation: Districts can adapt learnings from behavioral science and develop timely, actionable text messages for parents to encourage and support their students while learning at home.
For more on this topic, see J-PAL North America Education Technology and Opportunity Initiative Co-Chair Phil Oreopoulos’ op-ed “Stumped by How to Best Serve Students with At-Home Learning? Follow the Evidence” or contact Kimberly Dadisman.
Resources for further reading
Increasing adherence to COVID-19 guidelines: Lessons from existing evidence (J-PAL): Resource compiled by the J-PAL Health Sector to provide recommendations for responding to the COVID-19 pandemic. This note summarizes evidence generated by J-PAL affiliates and is not a summary of all the rigorous evidence on the discussed topics.
How Data Governments Already Have Can Guide the Pandemic Response (Governing): Op-ed on how state and local governments can leverage data collected for operational purposes (administrative data) to help them deal with the economic devastation of the coronavirus.
Easing Access to the Safety Net, and Why We Need to Do It Now (Governing): Op-ed in Governing by J-PAL North America Scientific Directory Amy Finkelstein and J-PAL North America Work of the Future Initiative co-chair Matt Notowidigdo outlining how state and local governments can leverage administrative data to guide policy responses to COVID-19.
Challenges of Equitable Rapid Response Cash Payments (Econofact): Econofact brief by J-PAL affiliated researcher Lisa Gennetian about the need for infrastructure to ensure delivery of economic impact payments or other cash stimulus.
How You Can Protect Your Community, Not Just Your Own Health (The New York Times): Op-ed in the New York Times by J-PAL affiliated researchers Kate Baicker and Oeindrila Dube on lessons learned from the Ebola crisis on garnering trust in communities to spur behavioral change to manage the pandemic.
Recommendations for a Metropolitan COVID-19 Response (Johns Hopkins School of Public Health): Special report for city leaders on public health recommendations for managing the coronavirus pandemic in a metropolitan context.
COVID-19 Resources (Results for America): Suite of resources for state and local governments, including resources from Bloomberg Philanthropies and the National League of Cities.
Coronavirus: What You Need to Know (The National Governors Association): Suite of resources for governors and states including memos on public health interventions, the federal response to the pandemic, and links to the CDC and other public health authorities.
J-PAL health sector co-chairs and staff discuss some lessons from the evidence for increasing adherence to Covid-19 guidelines
Please note: This note was prepared by the J-PAL Health Sector in 2020 to provide recommendations for responding to the COVID-19 pandemic. It is not an exhaustive review of all the rigorous evidence on the discussed topics.
To download this note in pdf form, click here.
As of May 2020, the novel coronavirus 2019 (COVID-19) has spread to every region of the world, infecting millions of people and killing hundreds of thousands.1 With no cure yet identified, prevention is essential. The World Health Organization (WHO), national health agencies, and other experts have issued guidance on best practices, from practicing social distancing to frequent handwashing.
Following this guidance is not necessarily easy for everyone. For instance, individuals living in overcrowded conditions may not be able to practice social distancing, those whose livelihoods depend on frequent and close interactions with others may not be able to afford it, and those without access to clean water and soap would not be able to wash their hands frequently. Other individuals may simply not be aware of the guidelines, may not understand the specific steps to follow, or they may not be convinced of the need to practice these behaviors.
In instances where lack of clear messaging or inadequate understanding of the need to practice recommended behaviors are the primary barriers to guideline adherence, evidence from rigorous research can help to inform government actions. This note provides some general lessons on increasing uptake of healthy behaviors and on improving the delivery of health products and services. It does not tailor recommendations to specific contexts or provide details on implementation. However, given the importance of tailoring interventions to suit local contexts, we encourage policymakers to reach out to Anupama Dathan, J-PAL Health Sector Manager, for follow-up conversations on incorporating the evidence into policy decisions.
Designing information campaigns to increase uptake of preventive behaviors
Many governments are advising residents to stay home, practice social distancing, regularly wash hands, and to take various other actions to prevent COVID-19. Many individuals will find it impossible to regularly practice the recommended behaviors due to financial insecurity, lack of access to clean water or soap, and various other constraints. For those who are physically able to take up these behaviors, what information is shared and through whom may influence adherence. It will also be important to avoid spreading misinformation when doing so.
Information is often impactful in changing behavior when it is specific and actionable.
Simply urging people to change behavior usually does not work. This insight comes from a number of studies from around the world, focused on a number of health topics.2 For instance, providing health insurance consumers in the United States with specific and personalized price information on the cost of prescriptions increased the likelihood of consumers switching plans and decreased annual consumer costs (Kling et al. 2012).
- Recommendation: Specific and actionable information campaigns to encourage key preventive behaviors for COVID-19 may be particularly effective. Experts may find it beneficial to continue issuing specific information on what behaviors to take up and why these actions are important. For example, advice to systematically wash hands upon returning from the grocery store or to set an alarm to remember to wash hands every 2 hours may be more effective than more general exhortation to “wash hands regularly.” Likewise, encouragement to replace handshakes with head nods or elbow bumps may be more actionable than simply telling individuals to avoid handshakes.
It is important to leverage the influence of peer networks such as friends and neighbors when asking individuals to take up new behaviors.
While top-down messaging that is framed in a specific and actionable way can change behavior, leveraging the community is also important for sustained behavior change.3 For example, a study in Nepal found that high school girls whose friends received a menstrual cup were more likely to adopt them as well (Thornton and Oster, 2008). Two experiments from India demonstrated that highly central individuals in a community may be particularly effective at spreading important information and that community members are able to identify such individuals quite reliably (Banerjee et al. 2019). In rural Peru, a large-scale handwashing intervention in 2010 involving a mass media campaign (radio, posters, and promotional events) alongside a community-based training improved knowledge and modified behaviors, while mass media alone had no impact. The campaign led to more households correctly identifying that water and soap was the best way of washing hands and increased the number of people washing their hands before eating (Galiani et al 2015; Galiani et al. 2012).
- Recommendation: Directly leveraging the influence of well-connected community members and peers (while avoiding face-to-face interactions) may help to spread accurate information on new recommendations. While guidelines on social distancing make in-person interactions difficult, online and mobile platforms may be good ways to leverage this influence.
Edutainment may be a promising delivery method for key public health messages and increasing take-up of health products.
Television, radio, or other entertainment platforms can be used to change attitudes and behaviors by embedding educational messages in a bigger storyline.4 In Nigeria, for example, the edutainment television series MTV Shuga improved knowledge and attitudes towards HIV and risky sexual behavior and increased the likelihood of getting tested for HIV (Banerjee et al. 2019). Effects were stronger for viewers who reported being more involved with the story or identified with the characters.
- Recommendation: Where possible, incorporating key public health messages on social distancing and hand washing into new or existing entertainment media could help improve adherence to recommended behaviors. The increased time spent at home due to COVID-19 lockdown orders and the ability to identify with characters facing similar changes to their personal life may further underscore edutainment’s potential to influence individual behavior. Edutainment may be difficult to produce under social distancing practices; however, animation or other art could potentially ease some of these concerns.
Engaging celebrities in sharing key health messages can increase the reach of these messages.
In Indonesia, a study leveraging Twitter found that celebrity endorsement significantly increased the likelihood that a tweet promoting immunization was liked or retweeted relative to similar tweets without celebrity endorsement. The effect was particularly large when celebrities were seen as speaking in their own voice, rather than citing external sources. By contrast, explicitly citing sources in the tweets actually reduced diffusion (Alatas et al 2020).
- Recommendation: Educating celebrities early on in an epidemic and leveraging their voices can help to avoid the spread of misinformation or the undermining of public health campaigns. Depending on the content and accuracy of their message, all individuals with a media following can have a positive or negative impact on public opinion and behavior. Celebrity influence may continue to be important for raising the profile of COVID-19 in the medium-term, when the current level of awareness has subsided but preventive measures are still important. Note that, to date, this recommendation is based on a single study rather than a broader body of research.
Evidence on information provision demonstrates that receiving messages from those with whom one feels a connection—community members, peers, characters on TV shows and other forms of entertainment, celebrities, etc.—can be important for uptake of recommended behaviors. Messaging this information with specific and actionable steps can be additionally impactful. It is also important to avoid spreading misinformation through these platforms.
Leveraging cash transfers to increase uptake of preventive behaviors
Cash transfers can act as both income support and encourage uptake of healthy behaviors.
Since the COVID-19 pandemic is a health crisis accompanied by a severe economic one, cash transfers can provide income support and potentially increase uptake of healthy behaviors.
A review of the literature shows that offering conditional cash transfers (CCTs) that require households to meet certain conditions on healthy behavior generally increases adoption of targeted behaviors.5 In some cases, making households aware of the importance of such behaviors and implying that the transfer should be used for them, without strong enforcement of conditions, also contributed to the increase in health service utilization.
Providing cash with no conditions, or unconditional cash transfers (UCTs), increases spending on household priorities and often improves overall household well-being. However, improved health outcomes are difficult to detect unless the cash transfer is very large.
- Recommendation: Cash transfers can be leveraged to increase uptake of healthy behaviors. Both UCTs and CCTs have costs associated with determining eligibility, targeting the transfer to intended households, and delivering the cash. Additionally, the conditions in a CCT can be soft (unenforced) or hard (enforced), or the transfer can also be unconditional but accompanied by specific, actionable information. When determining which type of transfer to implement, policymakers may consider the outcome(s) they are hoping to achieve, as well as the relative costs and benefits of each type of transfer across all target outcomes. A CCT with hard conditions has additional costs associated with monitoring which households are adhering to the conditions, which vary based on the design and strictness of the conditions, as well as on local capacity for monitoring and enforcing the conditions. If implementing a CCT, it will be important to determine what desired behaviors would be conditioned and whether monitoring take-up of these behaviors is feasible, or whether to rely on soft conditions or information provision instead. For instance, a hard condition may be possible if focused on testing for COVID-19, but a soft condition or information provision may be preferred if the priority is ensuring adherence to a lockdown or shelter-in-place policy.
Strategies to improve the delivery of health products and services
The practices described above can help motivate individuals and households to take up healthy behaviors. As governments and other actors think through distribution of essential commodities once they are developed, research yields insights on how to deliver them to maximize uptake.
Increased trust in the health system may be critical for more community cooperation with health guidelines.
A study in Sierra Leone found during the 2014-16 Ebola crisis that community monitoring of government-run health clinics and status awards for clinic staff (both implemented roughly two years prior to the Ebola health crisis) improved the community's perception of health care quality, utilization of health services, and Ebola-related health outcomes. During the crisis, the interventions increased Ebola testing rates and reduced mortality among patients, driven by improvements in the community monitoring intervention (Christensen et al. 2020). A study from the United States similarly showed the importance of trust in the health care system: black men, typically more mistrustful of the health care system, were more likely to trust providers of the same race. Seeing doctors they trusted increased their preventive health care utilization rate, including receiving the seasonal flu vaccine (Alsan et al. 2019).
- Recommendation: Programs or policies that increase trust in the health system could improve reporting and cooperation with health guidelines, which in turn could help increase testing, reduce the spread of the disease, and reduce mortality. In the study from Sierra Leone, the social accountability intervention had been implemented prior to the Ebola health crisis. Still, interventions that increase trust may be most impactful in areas with low baseline utilization of health care services or amongst populations with low levels of trust in the health system (including marginalized groups that may include migrants, LGBTQ individuals, indigenous communities, or racial minorities depending on the context). Additional trust-building policies, such as ensuring patient confidentiality with regard to immigration status, could also help address wariness of the formal health system. Such measures would most likely improve the resiliency of the health systems over time; they may also possibly improve outcomes if implemented during or right before a major health shock, though this has not been studied to date. Note that, to date, this recommendation is based on two studies rather than a broader body of research.
Providing recommended preventive health products such as COVID-19 tests, masks, an eventual vaccine, and others for free (once supply-side shortages are addressed) can help to ensure mass uptake.
Uptake of preventive health products, such as vaccines, is highly sensitive to price. A large body of evidence shows that take-up reduces dramatically even with small price increases, and especially so for products with large social externalities.6 For example, when a program in Kenya moved from free provision of deworming tablets to charging US$0.30 per child, take-up fell from 75 percent to 18 percent (Kremer and Miguel 2007). Furthermore, preventive products distributed for free have generally been put to good use.
- Recommendation: When feasible, subsidizing recommended preventive health products and eliminating user fees can increase uptake. There is currently a severe shortage of preventive products such as protective masks and COVID-19 tests, and a COVID-19 vaccine is not yet available. Once these supply-side issues have been addressed, providing these recommended products for free to the target population can ensure immediate, mass uptake.
This note highlights some general lessons on how policymakers may be able to increase adherence to COVID-19 guidelines and improve the delivery of key health products once they are readily available. It does not intend to provide details on implementation. Policymakers interested in learning more about the evidence presented here are encouraged to reach out to Anupama Dathan, J-PAL Health Sector Manager, for follow-up conversations on incorporating the evidence into policy decisions.
For more information, see this webinar by J-PAL Health Sector co-chairs, Karen Macours and Pascaline Dupas, on "Insights from behavioral economics for adherence to COVID-19 recommendations and improving service delivery during the current crisis."
1. World Health Organization. 2020. “Coronavirus disease 2019 (COVID-19): Situation Report 107.”
2. Studies from J-PAL affiliates on specific and actionable information provision include, Galiani et al. 2015; Bennear et al. 2013; Dupas 2011; Dupas et al. 2018; Maughan-Brown et al. 2015; Meredith et al. 2013; Kling et al. 2012
3. Studies from J-PAL affiliates on peer effects and social networks for health include Dupas 2014; Oster and Thornton 2012; Kremer and Miguel 2007; Goldberg et al. 2019; Banerjee et al. 2019
4. Studies from J-PAL affiliates on edutainment include Banerjee et al. 2019; Banerjee et al. 2016; Green et al. 2018
5. J-PAL's policy insight on the topic is available here.
6. J-PAL’s policy insight on the topic is available here.