Mobile Phone Data Compared to Household Surveys to Evaluate the Impact of Cash Transfers in Togo
- Rural population
- Adults
- Families and households
- Earnings and income
- Technology adoption
- Consumption smoothing
- Food security
- Health outcomes
- Cash Transfers
- Social Protection
- Cash transfers
- Digital and mobile
- Social protection
- COVID-19 response
- Unconditional cash transfers
During the Covid-19 pandemic, several governments provided cash grants to vulnerable individuals or businesses in response to the pandemic. In Togo, researchers conducted a randomized evaluation to test the effect of cash grants on people's welfare and if using mobile phone data in impact evaluations yielded the same results as survey data. They found that the cash grant program increased food security, mental health, and self-perceived socioeconomic status. However, evaluating only the mobile phone data did not produce reliable estimates of the effect of the program.
Policy issue
Unconditional cash transfers are payments made to individuals or households, often those facing poverty or economic vulnerability, without requiring any conditions to receive the funds. These grants have been studied across various settings and overall show positive impacts on recipients’ welfare. By increasing income and allowing recipients to decide for themselves what they need most, unconditional cash transfers typically increase household assets, consumption, psychological well-being, and food security. During the Covid-19 pandemic, many governments issued additional or emergency cash transfers to support vulnerable individuals or businesses, who were economically affected by lockdowns or curfews.1
Traditionally, the impact of such transfers is assessed through post-disbursement household surveys that ask recipients to self-report their spending and how the transfer affected their decision-making related to selling assets, consuming food, distributing funds for household expenses, and more. However, advances in machine learning and the growing availability of alternative data sources, such as mobile phone metadata, are opening new avenues for research. Mobile phone metadata, or the administrative records generated by mobile network operators when calls, text messages, and other transactions are placed on their networks, typically contains information about the timing and locations of calls and texts without any information on their content. These sources have proven effective in estimating living standards and offer several advantages: they are often cheaper, faster, and can reach populations that traditional surveys may miss. Did a cash transfer program during the Covid-19 pandemic increase people’s welfare, and can mobile phone data be used to not just identify low-income individuals, who would benefit from cash transfers, but also substitute for household surveys in evaluating the impact of programs like cash transfers?
Context of the evaluation
The evaluation took place in Togo during the Covid-19 pandemic. Togo is a small country in West Africa, home to approximately 8.5 million people, where about 59 percent of people in rural areas and 26.5 percent in urban areas experience poverty.2 In Togo, the Covid-19 pandemic led to trade disruptions and curfews designed to stop the spread of the virus.3 In reaction to the pandemic, the Togolese Ministry of Digital Transformation and GiveDirectly jointly implemented a cash transfer program called Novissi. Cash grants were provided to individuals in rural parts of Togo between November 2020 and August 2021. Women received XAF 8,620 (US$15.50) per month, and men received XAF 7,450 (US$13) per month for a duration of five months.
When the government first launched Novissi, it did not have a traditional social registry that could be used to assess program eligibility.4 Researchers therefore analyzed mobile phone usage patterns like call frequency and spending and linked the information to survey data, to train algorithms that could predict incomes for each mobile phone user. Using this data, researchers identified eligible participants. Eligible individuals had to be registered to vote in one of the 100 lowest-income cantons of Togo, and estimated to be living on less than US$1.25 per day (the 29 percent lowest-income individuals). The majority of cantons identified were in the Savanes region, which is the northernmost of the five Regions of Togo, and about 70 percent of recipients lived in this region. The Savanes region also had higher Covid-19 rates and received a separate government cash transfer in February 2021.
Details of the intervention
Researchers conducted a randomized evaluation to study the effect of cash grants on people’s welfare during the Covid-19 pandemic, as well as whether the impact measured via mobile phone data would yield the same results as household surveys.
Researchers assigned eligible individuals into two groups:
1. Cash transfer group (27,673 individuals): Individuals were immediately sent the first cash transfer. Subsequent cash transfers were delivered monthly between November 2020 and April 2021.
2. Comparison group (21,410 individuals): Individuals were not sent the cash transfer during the evaluation period. However, they were sent a cash transfer with the same total amount as a lump sum, beginning in June 20215, but were not told about it beforehand.
Researchers collected survey data in September 2020 (pre delivery) and during a phone survey in May 2021 after the final cash transfers were sent to participants. They asked about participants’ food security, financial well-being, mental health, perceived economic status, and access to health care.
In addition to the phone surveys before and after the program, researchers obtained mobile phone data from Togo’s two mobile network operators for the duration of the program. They used two different mobile phone data sets to train the machine learning algorithm: first data collected before the program between March-September 2020 matched with the results from the survey conducted before the start of the program in September 2020. Respondents for this first survey were drawn from active mobile subscribers, who lived in one of the 100 lowest-income cantons. They were representative for all mobile subscribers, not just those predicted to be below the poverty threshold. Second, they used mobile phone data from between November 2020-April 2021 and matched it with the final household survey data from May 2021 to compare outcomes.
Results and policy lessons
Researchers found that the cash grant program increased food security, mental health, financial inclusion, and self-perceived economic status. However, the mobile phone data alone did not produce reliable estimates of the effect of the program on participants’ welfare.
Mobile phone usage: Mobile phone use increased in response to receiving the cash transfer. Calls increased by 0.021 SD, contacts by 0.033 SD, and active days by 0.064 SD.
Welfare changes from surveys: Using the household surveys, researchers found that the program increased food security by 0.064 standard deviations (SD), mental health by 0.072 SD, and self-perceived economic status by 0.04 SD. These results were broadly consistent with other studies finding similar positive impacts of cash transfers on food security and mental health during the Covid-19 pandemic.
Welfare measures from mobile phone data: Evaluating mobile phone data from before the start of the program, researchers found no meaningful effects on any welfare indicators. However, mobile phone post-program data revealed positive impacts on mental health (0.030 SD) and health care access (0.031 SD). The results from mobile phone data did not produce the same results as the survey data and therefore could not be used as a reliable estimate of the welfare indicators. This may have been due to the relatively small cash amount given. Researchers posit that interventions that have larger impacts, like large cash transfers or graduation programs, may be easier to observe in mobile phone metadata and other non-traditional data sources. Finally, evaluating impacts with mobile phone metadata required not just detecting levels of poverty, but also detecting changes in poverty over time. This may have been challenging to capture in the algorithm and may not have affected long-term poverty levels but instead more short-term stress reliefs that led to higher well-being.
Differences by geography: The program had a stronger impact on food security and mental health in the Savanes region compared to other rural areas. Researchers looked at multiple explanations for this, such as the interaction of the two cash transfers this region received, curfew-related mobility reductions, or regional price differences, but were unable to explain the differences.
Differences by gender: Despite women receiving about 15 percent more money per month, the researchers did not find a detectable difference on participating women’s welfare compared to men.
Taken together, while mobile phone data and machine learning produced accurate estimates of poverty levels or demographic characteristics6, it did not produce reliable estimates of food security or self-perceived economic status after the intervention. Researchers suggested that mobile phone data and machine learning algorithms may be more useful in contexts where a program affects outcomes such as wealth, when effects are larger, or when there are more differences in characteristics between participants before the program.
Aiken, Emily, Bellue, Suzanne, Blumenstock, Joshua E., Karlan, Dean, Udry, Christopher. 2025. “Estimating impact with surveys versus digital traces: Evidence from randomized cash transfers in Togo”. Journal of Development Economics 175 (June).
International Monetary Fund. Policy Responses to Covid-19. https://www.imf.org/en/Topics/imf-and-covid19/Policy-Responses-to-COVID-19#A
The World Bank. The World Bank in Togo. https://www.worldbank.org/en/country/togo/overview
The World Bank. 2020. International Development Association Program Document for a Proposed Development Policy Grant in the Amount of SDR 25.5 million (US$35.0 million equivalent) and a Proposed Development Policy Credit in the Amount of EUR 31.3 million (US$35.0 million equivalent) to the Republic of Togo for the Togo Covid-19 Crisis Response Development Policy Financing. https://documents1.worldbank.org/curated/en/630971600048821354/pdf/Togo-COVID-19-Crisis-Response-Development-Policy-Financing.pdf
Innovation for Poverty Action. 2021. “Case Study: Togo’s Novissi Cash Transfer: Designing and Implementing a Fully Digital Social Assistance Program during COVID-19”. https://poverty-action.org/sites/default/files/publications/Togo-Novissi-Cash-Transfer-Brief-August%202021.pdf
Innovation for Poverty Action. 2021. “Case Study: Togo’s Novissi Cash Transfer: Designing and Implementing a Fully Digital Social Assistance Program during COVID-19”. https://poverty-action.org/sites/default/files/publications/Togo-Novissi-Cash-Transfer-Brief-August%202021.pdf
Aiken, Emily, Bellue, Suzanne, Karlan, Dean, Udry, Christopher, Blumenstock, Joshua E. 2022. “Machine learning and phone data can improve targeting of humanitarian aid.” Nature 603 (7903), 864-870.
Blumenstock, Joshua E., Cadamuro, Gabriel, On, Robert. 2015. “Predicing poverty and welfare from mobile phone metadata.” Science 350 (6264): 1073–1076.
Blumenstock, Joshua E. 2018. “Estimating economic characteristics with phone data.” AEA Paper and Precedings Vol 108. 72-76.