Evaluations

PDF version

The following is a list of projects supported by the U.S. Health Care Delivery Initiative.

Full Projects

Health Care Hotspotting: A Randomized Controlled Trial
Researchers: Jeffrey Brenner, Joseph Doyle, Amy Finkelstein, Sarah Taubman, R. Annetta Zhou
Location: Camden, New Jersey

The Camden Coalition of Healthcare Providers’ Care Management Program, Link2Care, targets “super-utilizers” of the health care system – specifically adults with two or more hospitalizations in the prior six months and multiple chronic conditions – with intensive care-management services in the months following hospital discharge. A team of nurses, social workers, community health workers, and health coaches, supported by real-time data of health care use, perform home visits, accompany patients to doctor visits, and help patients enroll in social-service programs. This approach aims to improve the self-sufficiency of patients in navigating the health care and social-service systems and has the potential to reduce hospital re-admissions and improve patient well-being. This project will rigorously investigate the impacts of the program on subsequent health care utilization and the use of social services through a randomized controlled trial.

Improving the Outcomes of SSI Children through Information
Researchers: Manasi Deshpande, Rebecca Dizon-Ross
Location: United States

Supplemental Security Income (SSI) provides cash payments to the families of 1.3 million low-income disabled children who are at risk for poor life outcomes. Available evidence indicates that many SSI families are unaware of the likelihood that children will be removed from SSI at age 18, and may therefore underinvest in the human capital of their children. In this randomized evaluation, researchers will collaborate with state vocational rehabilitation agencies to estimate the effects of providing SSI youth and their families with information about the likelihood of removal from SSI at age 18. Researchers will estimate the effects of written information and information-driven counseling about SSI removal on the educational, health, employment, and justice-involvement outcomes of the SSI youth.

In it to Quit: Commitment Contracts for Smoking Cessation
Researchers: Daren Anderson, Dean Karlan, Amanda Kowalski, Jody Sindelar, Jonathan Zinman
Location: Connecticut

This study aims to examine whether a combination of positive and negative commitment devices is effective at inducing long-term smoking cessation in smokers from a low-to-moderate income background. Researchers will randomly assign patients of a multi-site clinic in Connecticut to receive one or both of a succession of two commitment contracts. First, a two-month reward contract will pay participants for (a) engaging in clinic and web-based activities that support smoking cessation and (b) quitting smoking, as measured at period end. Second, a four-month deposit contract will invite participants to commit funds which they forfeit unless they abstain from smoking over the entire period. A random subset of participants will also have the option to pre-commit to the deposit contract at study enrollment. Reward payout and deposit recovery will be conditional on biochemical verification of self-reported abstinence. Researchers will conduct follow-up biochemical verification at 12 months to observe whether abstinence persists after removal of incentives. The primary outcome of interest will be continuous abstinence from smoking between months 2 and 12, i.e., biochemically verified abstinence at all three measurements. A short baseline and endline survey will also be administered to measure characteristics and outcomes related to smoking behavior.

Intervening with Consumers to Improve Choices on Health Insurance Marketplaces
ResearchersKeith Ericson, Jon Kingsdale, Timothy Layton, Adam Sacarny
Location: Colorado

The Affordable Care Act dramatically expanded the use of regulated marketplaces to provide individuals with health insurance coverage. The efficiency of these marketplaces depends on the ability of consumers to choose plans that reflect their preferences, but inattention and information frictions may inhibit optimal choices. In this project, researchers will conduct a randomized intervention to test the effect of sending information about the potential to switch plans and save on insurance premiums. Researchers will provide one random subset of re-enrolling individuals in Colorado's health insurance marketplace, Connect for Health Colorado, with personalized information about potential savings in two letters and two e-mails. They will send another random subset of re-enrolling individuals generic information about potential savings in two letters and two e-mails. Researchers will test the effect of these interventions on whether an individual switches her plan, the amount of her plan premium, and the breadth of her plan provider network. Researchers will also estimate the effect of the information on elasticities with respect to plan premiums and provider networks. The results of this study will shed light on behavioral frictions in health insurance and economic parameters central to market design. The study will also provide evidence on inexpensive, effective strategies for marketplaces and policymakers to improve consumers’ decision-making in selecting health plans.

Randomized Evaluation of the Nurse-Family Partnership
ResearchersKatherine BaickerMary Ann BatesMichelle Woodford, Annetta Zhou
Location: South Carolina

There is enormous policy interest in expanding programs that move beyond traditional health care walls into the community to improve health outcomes – and holding those programs financially responsible for doing so. Billions of public dollars are devoted to “home visiting” programs that seek to improve birth and long-term outcomes for low-income mothers and children.  The Nurse-Family Partnership (NFP) program provides regular nurse home visits to low-income, first-time mothers through pregnancy and up to the first two years postpartum in order to improve the outcomes of both mothers and their children. South Carolina obtained a Medicaid waiver and generous philanthropic support to fund a landmark pay-for-success initiative that expanded NFP’s services across the state.  This randomized evaluation will assess a number of “success indicators” relevant to the pay-for-success initiative, such as the program’s effectiveness in reducing injuries among newborns and toddlers, as well as study the potentially wide-ranging effects of NFP on the health and well-being of mothers and children for many years to come. The study aims to yield insights into the effectiveness of home visiting programs, the sustainability of more flexible public insurance benefits, and the potential role that evidence-based payments can play in driving improved outcomes.

Reducing Inappropriate Prescribing of Controlled Substances in the United States
Researchers: Adam Sacarny, David Yokum, Amy Finkelstein, Shantanu Agrawal
Location: United States

Inappropriate prescriptions are a rising threat to the health of patients, particularly senior citizens. Overprescription can also increase health care expenditures due to the direct cost of the drugs and the resulting health care use caused by adverse outcomes. This randomized evaluation studies a low-cost, light-touch intervention aimed at reducing the inappropriate provision of Schedule II controlled substances, which carry particularly large risks for patients, in the Medicare Part D program. Potential overprescribers were sent a letter explaining that their practice patterns were highly unlike those of their peers. Using rich administrative data to measure outcomes, researchers found no evidence that the letters had an impact on prescribing rates. The results of this study and further learning about the potential of light-touch interventions will help produce a better toolkit for policymakers to improve the value and safety of health care.

The Determinants of Health Insurance Take-up and Plan Choice in Covered California
Researcher: Wesley Yin
Location: California

While the Affordable Care Act aimed to expand insurance coverage in the United States, insurance take-up rates remain fairly low, and many individuals purchase plans that are sub-optimal for their needs. The success of the health insurance exchanges created by the ACA largely depends on the choices consumers make, but the underlying factors contributing to those choices are not well understood. This project will conduct a randomized evaluation in California to study the key determinants of health insurance take-up and plan choice. The study will evaluate how providing information about the benefits of purchasing a plan, premium subsidies, penalties of remaining uninsured, and making plans more easily comparable, affects insurance take-up and choice of plans.

The Effect of Informative Letters on the Prescription and Receipt of Seroquel
Researchers: Adam Sacarny, David Yokum, Shantanu Agrawal
Location: United States

Abusive prescribing exposes patients to unnecessary health risks and results in wasteful public expenditures. This study will evaluate an innovative approach to fighting abusive prescription: sending letters to suspected inappropriate prescribers warning them that they are outliers compared to their peers and have been flagged for review. The study will target high prescribers of Seroquel (Quetiapine), a commonly prescribed antipsychotic. Using claims data, the researchers will assess the effect of the letters on prescribing of Seroquel, receipt of Seroquel by patients, substitution behavior by prescribers and patients, and health outcomes of patients.

The Impact of Employee Wellness Programs: A Randomized Controlled Trial
Researchers: Katherine Baicker, Zirui Song
Location: United States

There is great public and private interest in the use of employee wellness programs to improve health and lower health care costs, with such programs promoted in the Affordable Care Act and adopted by half of large firms. However, little rigorous evidence exists on the effects of such programs. Using a randomized controlled design, the researchers propose to evaluate a year-long wellness intervention at BJ’s Wholesale Club, a large multi-state U.S. business with 201 sites (clubs) and over 25,000 employees. The wellness intervention will consist of several components, including team-based wellness challenges, nutrition counseling, stress reduction, and physical activity. The intervention has been designed with and will be fielded through Wellness Workdays, an experienced and large-scale vendor. Randomizing this intervention across a set of treatment clubs and control clubs, researchers will evaluate its impact on five categories of outcomes gathered from both primary and administrative data sources: (1) self-reported health and well-being, (2) biometrics (such as body mass index, blood pressure, blood sugar, etc.), (3) worker absenteeism and turnover, (4) health care spending and utilization, and (5) the implied return on investment. Better information about the effectiveness of such programs should inform both employer investment and public policy.

Worksite Wellness: A Field Experiment on Participation Incentives and Selection into Wellness Programs
Researchers: Damon Jones, David Molitor, and Julian Reif
Location: Illinois

Workplace wellness programs have become a $6 billion industry and are widely touted as a way to improve employee well-being, reduce health care costs by promoting prevention, and increase workplace productivity. Yet, there is little rigorous evidence available to support these claims, partly because the voluntary nature of these programs mean that participants may differ from nonparticipants for reasons unrelated to the causal effects of the wellness program. Researchers will implement a randomized control trial with the aim of answering three questions: (1) how do incentives affect the level of participation in wellness programs; (2) what types of workers select into wellness programs, and how do incentives affect that composition; and (3) what is the causal impact of worksite wellness participation on health, health care costs, and productivity?

Pilot Studies

Contraceptive counseling for postpartum moms during pediatric visits
Researchers: Jessica Cohen and Margaret McConnell
Location: Kansas

Researchers are conducting a pilot study in collaboration with University of Kansas Pediatric Clinic to prepare for a large-scale trial examining the impact of contraceptive counseling for moms at their infants’ pediatric visits on postpartum contraceptive use. Over half of all pregnancies in the United States are unplanned, with a substantial share occurring after insufficient spacing between pregnancies, contributing to morbidity and mortality risk for mothers and babies. The researchers hypothesize that many postpartum women have insufficient contact with providers who traditionally counsel on family planning, but that these moms are in frequent contact with the medical system through their infants’ pediatrician visits, which provide repeat opportunities for contraceptive counseling and follow-up. The objectives of the pilot study are to collect baseline data to identify a key target population and conduct a sample size calculation, assess the acceptability of the proposed intervention, develop protocols and data collections systems, and to identify potential complementary interventions to increase the implementation and impact of the program. The findings of this pilot will be used to design a full-scale effectiveness trial assessing program impact on postpartum contraceptive use.

Designing Incentives for Impatient Diabetics: a Randomized Experiment 
Researcher: Rebecca Dizon-Ross
Location: United States

Poor disease management among diabetics is a major driver of recent U.S. health care cost growth. One potential barrier to proper disease management is impatience, since the costs of managing the disease are borne today but the benefits not realized until the future. A promising approach for increasing compliance is thus to offer incentive payments for healthy behaviors. However, it is not well understood how to design optimal incentives for impatient agents. Two key aspects of the incentive design (the lag between incentivized behavior and payment, and whether the contract is additively separable across days) should theoretically interact with time preferences, causing the optimal design to depend not just on average patience levels but also on the shape of time preferences. This project will conduct a randomized evaluation of different incentive schemes for diabetics, varying lag length and additive separability, to evaluate what incentive scheme works best and how incentive effectiveness varies by individual time preferences. The project will also assess whether input or outcomes incentives are more effective. Effects will be measured on health outcomes like blood sugar control and BMI, and the persistence of the effects will be assessed.  

Do Informed Physicians Make Better Referrals?
Researchers: Zack Cooper, Michael Chernew, Fiona Scott Morton
Location: United States

Health care providers’ prices vary substantially within geographies and there is little evidence that higher priced providers deliver higher quality care. With more than 43% of total health care spending estimated to be ‘shoppable’, the savings from improving how patients navigate the health system could be substantial. Indeed, using claims data from a large commercial insurer, the researchers have identified that, if every patient who received an MRI in 2014 at a provider with prices above the 25th percentile had gone to a provider at the 25th percentile price, it would have resulted in a 41% decrease in MRI spending. The researchers will use a clustered RCT to test the effect of providing referring physicians – patients’ agents – with pricing information on where they send their patients for care and financial incentives to steer their patients towards more efficient providers. This project will test whether equipping providers with information on the price of their referrals during the care process leads to reductions in the price of imaging services received by their patients.

Financial Incentives to Increase Health System Engagement Among Those with Complex Needs
Researchers: Marika Cabral, Rebecca Dizon-Ross, Caterina Hill, Zirui Song
Location:
Massachusetts

Financial incentives to encourage patient engagement have been found to be useful in other settings, but have not been adequately tested with complex needs populations. In collaboration with the Commonwealth Care Alliance, researchers will pilot a randomized controlled trial to test whether small financial incentives will increase beneficiary engagement in primary care amongst individuals dually eligible for Medicare and Medicaid. The pilot will randomize financial incentives for engagement in primary care among newly enrolled CCA beneficiaries. In a later, full RCT, researchers plan to investigate the impact of financial incentives for increased primary care engagement on additional outcomes, such as the receipt of clinically effective prevention services and utilization.

Mobile-izing Medicine: The Effect of Accessible Clinical Practice Guidelines on the Quality of Hospital Care
Researchers: Zarek Brot-Goldberg, Benjamin Handel, Jonathan Kolstad, Michael Whinston
Location: Maine

Lack of adherence to clinical practice guidelines is an important source of preventable medical errors. Nevertheless, lack of adherence to such guidelines, and clinical error more generally, is widespread in health care delivery today. One potential barrier to adherence is the fact that these guidelines are typically inaccessible at the point of care, precisely when they are most needed. The expansion of IT systems (e.g. electronic health records) in health care settings, as well as the propagation of mobile devices, has created a plausible way to overcome this barrier. In this study, researchers assess the impact of electronic access to practice guidelines to improve care, using the case of a mobile-based application called AgileMD. To do so, researchers assess adoption of the tool, use of the tool and, ultimately, clinical outcomes. Using detailed data and quasi-random variation, results from these analyses will lay the groundwork for a randomized controlled trial of the AgileMD tool.

Nutritional Supports for At‐Risk Patients
Researchers: Seth Berkowitz, Joseph Doyle, Amy Finkelstein
Location: Massachusetts

Cardiovascular diseases are by far the leading causes of death in the U.S., with over 800,000 deaths annually. Approximately 1 in 4 Americans with cardiometabolic conditions such as hypertension, hyperlipidemia, and diabetes experience food insecurity, which is suspected to be a major, modifiable risk factor for these health conditions. This pilot study seeks to determine the feasibility of a randomized evaluation of two interventions targeting the diets of low-income patients at high risk of cardiovascular disease: (1) helping patients enroll in the Supplemental Nutrition Assistance Program (SNAP) and (2) providing medically‐tailored meals directly to patients.

Overcoming Financial Barriers to Caring for Preterm Infants
Researchers: Guenther Fink, Margaret McConnell
Location: Massachusetts

This pilot randomized controlled trial, conducted in collaboration with Tufts Medical Center, aims to determine whether financial support can increase caregivers’ ability to breastfeed and provide skin-to-skin care to preterm infants. The study team's central hypothesis is that low-income households cannot afford to spend extended periods of time in the hospital after preterm delivery, which makes both exclusive breastfeeding and continued skin-to-skin care – two highly effective practices to improve preterm health – very difficult. The objective of the pilot study is to develop a protocol for a financial support system targeted to low-income mothers with babies in neonatal intensive care units (NICUs). The primary outcomes of the pilot trial will be breastfeeding initiation and continuation and the number of hours of skin-to-skin care provided to the infant. Estimates of the impact of financial transfers on these outcomes will be of critical importance to power a larger subsequent effectiveness trial assessing program impact on key child health outcomes.

Pain Management Options for Opioid-Tolerant Patients: A Randomized Controlled Trial
Researchers: Joseph Doyle, Padma Gulur, Mireille Jacobson
Locations: California, North Carolina

Pain is a fundamental symptom that drives healthcare use, including hospital readmissions and patients who are on high levels of opioids have particularly high utilization patterns, including hospital admissions and emergency department visits. Pain specialists may be able to improve these patients’ health and reduce healthcare costs. While pain management is actively studied for acute episodes, and for chronic-pain patients, sub-acute patients leaving the hospital after surgery have received relatively little prior research. Researchers are conducting a randomized evaluation of a new clinical pathway developed to provide pain management to these patients throughout the acute and into the sub-acute phases. The pathway includes a consultation in the hospital by a team of pain management specialists. Patients will have access to these specialists during the sub-acute phase after discharge. Improved pain management holds the potential to break the cycle of readmissions and escalating opioid use among these high-risk, high resource utilizing patients. In addition, the study provides a model of how to roll out care re-designs via random assignment to study their effectiveness in a rigorous way.