Evaluation of SAMASOURCE Training Program
- Recruitment and hiring
Access to quality jobs is a pressing concern in sub-Saharan Africa. In Kenya, researchers partnered with the technology company Sama and Innovations for Poverty Action to conduct a randomized evaluation of Sama’s training and job referral programs. The training program provided youth from underserved communities with digital training, while the referral program provided trainees with a job referral to work with the company. Providing both training and a job referral increased earnings by 37 percent and reduced unemployment rates by 10 percentage points.
Access to quality jobs is a pressing concern in Sub-Saharan Africa. More than one third of employed workers in the region still lived on less than $1.90 a day as of 2018, and 60 percent of all salaried jobs were in the informal sector in 2016. 1 2 Opportunities for well-paying formal-sector jobs are in high demand as population growth has exceeded formal sector job growth in the region. Barriers to quality employment are particularly large for youth, who often lack the skills and information to access good jobs. Compared to employed adult workers, employed youth in the region faced a poverty rate (living on less than $1.90 a day) twice as large in 2018. 3 For young women with childcare responsibilities or young people who are spatially disconnected from job opportunities, these barriers can be even larger.
An increasing number of low-and-middle income countries see the Information and Communication Technology (ICT) sector as having the potential to generate a large expansion in well-paying formal-sector jobs and rapid economic growth. Opportunities in this sector might be well-suited for young jobseekers in Sub-Saharan Africa given the significant number of young computer-literate individuals fluent in several global languages. However, there is little evidence on both how to equip these young workers with the skills required for digital work opportunities and on how this type of work might impact their livelihoods.
Context of the evaluation
Informal jobs, which often put workers at a higher risk of vulnerability and precariousness, are predominant in Kenya: in 2018, informal sector jobs accounted for 83.6 percent of total employment. 4 5 In addition to facing a low availability of formal jobs, youth in Kenya are disproportionately affected by unemployment. In the third quarter of 2019, the unemployment rate amongst 20-24 year old jobseekers in the country was 16 percent, compared to 5.3 percent in the overall population. 6 These challenges have been exacerbated by the impact of the Covid-19 pandemic on the Kenyan and world economy, which particularly struck informal workers’ wages in the country (decreasing them by 32 percent between February 2020 and November 2020, compared to a 3 percent decline in formal workers’ wages). 7
The partner for this evaluation, Sama–a data annotation company for machine learning algorithms–provides training and job opportunities to low-income youth in Nairobi with little to no formal employment experience. The company offers free job training to these youth through its classroom-based Artificial Intelligence 101 (AI 101) program, which focuses on basic digital skills, occupational skills, and job search preparation skills. Applicants to the program must have a high school certificate and English proficiency. Graduates of AI 101 pursue job opportunities at Sama, including Sama’s Delivery Center (SamaDC), and its network of hiring partners. These are formal job opportunities with generous wages: average monthly salaries at SamaDC are 2.5 times the formal minimum wage in Kenya, and workers also receive benefits such as health care, pension, meal subsidies, and transportation for night shift workers.
Details of the intervention
In partnership with Sama, researchers conducted a randomized evaluation to measure the impact of Sama’s training program and job referrals on earnings, unemployment, happiness and financial security in Nairobi, Kenya.
Youth between 18 and 35 years of age who had applied to participate in Sama’s AI 101 program and met the eligibility criteria were randomly assigned to one of three groups:
- Training only (286 participants): Participants in this group received Sama’s AI 101 program. However, they were not eligible to apply for jobs at SamaDC and did not receive any job referrals to work for Sama after completing the training. They were free to apply for employment at any of Sama’s partner work centers or other jobs.
- Training & job referral (548 participants): Participants in this group received Sama’s AI 101 program and a job referral to work at SamaDC at the end of the program. Job applicants still needed to go through Sama’s standard candidate interview and vetting process to receive an employment offer, but applicants had strong expectations that they would be hired. Hired applicants completed digital projects like image tagging, image annotation, data classification, and dataset creation at SamaDC.
- Comparison group (281 participants): Participants assigned to this group were not selected to receive Sama’s AI 101 program and were told they were ineligible to apply for the program for the next 1-2 years.
To measure the impacts of Sama’s training and job referrals, researchers conducted an initial participant survey starting in April 2017 before the program started, a midline survey around 16 months later, and a final survey around 2.5 years later. All surveys were conducted over the phone. Researchers collected data on household demographics, income and expenditure, employment outcomes, well-being and financial security, and performance in AI 101 training exercises. The final survey for around 20 percent of participants took place a month after Kenya’s first country-wide Covid-19 restrictions were implemented on March 2020, allowing researchers to also measure whether any of the programs helped participants withstand the initial shocks of the lockdown.
Results and policy lessons
Providing youth applicants to Sama’s training program with both training and a job referral increased their monthly earnings by 37 percent and reduced unemployment rates by 10 percentage points relative to the comparison group. However, providing training alone without a job referral did not increase participants’ employment or earnings.
Earnings and unemployment: Providing both training and a job referral increased participants’ monthly earnings by 37 percent (from a comparison group mean monthly earnings of 13,440 Kenyan Shillings), reduced unemployment rates by 10 percentage points (relative to the comparison group’s mean unemployment rate of 30 percent), and increased their number of hours worked by 5.7 hours a week (a 22 percent increase over a comparison group mean of 27.7 hours). These effects were sustained 16 months as well as 2.5 years later and were driven by workers who ended up working for SamaDC after receiving a referral. The impact was greater among women, who reported earnings 60 percent higher than the comparison group compared to 26 percent higher among men. However, providing only training without a job referral did not increase employment, earnings, or hours worked relative to the comparison group 2.5 years after the treatment. In fact, researchers found evidence of lower earnings after 16 months, relative to the comparison group, suggesting participants in the training-only group held out for a high-quality ICT job.
Work effort: Attendance in the AI 101 training sessions was 10.2 percentage points higher among participants in the training & job referral group compared to participants in the training-only group. AI 101 completion rates were also 13.9 percentage points higher for the training & job referral group compared to the training-only group. These findings suggest that the promise of a job referral induced additional effort during training.
Happiness and financial security: Participants in the training & job referral group reported higher levels of current and projected life satisfaction than the comparison group. This group also saw improvements in financial security relative to the comparison group, as they were 10 percentage points more likely to have bank accounts (a 12 percent increase over a comparison group mean of 83.6 percent), 11 percentage points more likely to have savings accounts (a 22 percent increase over a comparison group mean of 49.4 percent), and had 58.8 percent more savings (7,724 Kenyan Shillings more than the comparison group mean of 13,131 shillings). Participants who only received training did not have these positive effects.
Covid-19 resilience: While income in the comparison group fell among participants who were interviewed after Kenya’s Covid-19 restrictions began in March 2020, income in the training & referral group did not decrease after the restrictions began. These findings suggest that participants who obtained jobs at SamaDC were better able to withstand Covid-19 shocks–an expected finding given these jobs continued uninterrupted during this period and work at home was permitted. However, participants in the training-only group did not report a higher income than those in the comparison group following the lockdown, suggesting trainings alone did not make individuals more resilient to the economic damage of the pandemic.