The LA Homelessness Evaluation Network: Lessons learned from supporting organizations build evidence and evaluation capacity

The growing rate of homelessness and housing instability is a pressing issue for many areas across the United States, including Los Angeles (LA) County. As of 2023, Los Angeles City and County had the largest number of individuals experiencing homelessness of any Continuum of Care and the largest number of unsheltered people in the country. While there are some existing evidence-informed models to address homelessness and housing instability, many open and testable questions remain on how to best address this complex and widespread issue. 

In March 2023, J-PAL North America announced a new project to increase the capacity of LA County-based homeless service providers and government agencies to generate and use evidence to inform decision making—the LA Homelessness Evaluation (LAHE) Network. 

LAHE Network overview

The LAHE Network was created in partnership with the Conrad N. Hilton Foundation, who share in J-PAL North America’s goal of increasing evidence-informed solutions to homelessness in LA County. Our aim was for Network participants to gain a basic understanding of rigorous evaluation methods, knowledge to build evidence and evaluation capacity, and the opportunity to access tailored support to potentially develop their own randomized evaluations.

In reflecting on why this engagement was important, Seyron Foo, Senior Program Officer at the Conrad N. Hilton Foundation, noted that, 

“We must have evidence-based policies and interventions, such as Housing First, that successfully connect an unhoused person to the safety and dignity of a permanent home. This collaboration between J-PAL North America, nonprofit organizations, governmental agencies, and the Hilton Foundation helps build the fundamentals for effective future research collaborations grounded in rigorous methodologies.”

The LAHE Network launched in August 2023 with eleven LA County-based homeless service providers and government agencies who work with a variety of communities including youth, families, people of color, transgender and intersex individuals, people involved in the criminal legal system, and people with serious mental and physical health needs. Over the course of seven months, Network members attended monthly trainings on impact evaluations, theories of change and measurement, comparing impact methodologies, ethics, and common randomization designs. They also heard from two J-PAL North America partners, King County and Larkin Street Youth Services, who have hands-on experience running randomized evaluations in the homelessness and housing space. Their presentations showcased that, while evaluations in this sector can come with challenges, they can also be both achievable and invaluable. 

Participants also engaged in “office hour” sessions where they received small group support from J-PAL North America staff to facilitate more peer learning. During the sessions, participants applied evaluation concepts introduced in the trainings to their own programs and contexts using a customized workbook. 

The engagement culminated in an in-person convening in Los Angeles in February 2024 where Network members celebrated their hard work and accomplishments over the past seven months, shared potential randomized evaluation ideas, and discussed opportunities for the future.

Lessons learned and future possibilities

For some organizations, the LAHE Network engagement marked the beginning of a journey to take a more evidence-driven approach to tackling the pervasive challenge of homelessness in LA County. For others, it was an opportunity to deepen existing evaluation skills to help them serve their clients and community more effectively.

At the convening, some Network members shared that while they were skeptical of randomized evaluations at the start of this engagement, they now understand the value of them and believe rigorous evaluation is a crucial component of their work that they want to prioritize moving forward. 

Chrismen Oliver, a member of the LAHE Network and the Associate Director of Interim Housing at HOPICS, shared that, 

"My experience with the LAHE Network was incredibly valuable. Even though the time commitment could be a bit challenging, the trainings and office hours strengthened my understanding of randomized evaluations and how they can be applied in the homelessness and housing space. I’ve also found myself thinking more critically about data and evaluation results that I now come across in other settings." 

Across the engagement, a common theme emerged: running good, rigorous evaluations is a marathon, not a sprint. Participants noted that there can be myriad evaluation-readiness roadblocks, including staff capacity constraints, data access challenges, insufficient time or funds to design studies and interpret data, lack of institutional buy-in, and inaccessible information and expertise.

Through the Network, we began to chip away at one small but essential component of evaluation readiness: access to information and expertise. The lessons learned through the LAHE Network demonstrated the value of providing trainings and office hours, offering resources on building out research questions and theories of change, and preparing organizations to think critically about evaluation. As we close out this engagement, J-PAL North America is working to identify additional ways to address these remaining roadblocks, and we can’t wait to see what evidence these partners may generate in the future.

As J-PAL North America continues to support homeless service provider organizations in strengthening their evaluation capacity, we welcome the opportunity to collaborate. If you would like to explore a partnership or collaboration, please contact Laina Sonterblum.