When should big data and algorithms be used to determine programme eligibility?
Although machine learning models using mobile phone data can make poverty targeting faster and more cost-effective, traditional survey-based methods remain more accurate. The optimal approach therefore depends on striking the right balance between cost, accuracy, and programme scale.
This article first appeared in VoxDev
In recent decades, hundreds of billions of dollars have been spent on social protection programmes, with around 52.4% of the global population covered by at least one such programme (ILO 2024). While countries devote a substantial portion of GDP to such programmes, there is evidence that in many cases, resources often do not reach the households with the greatest need. For instance, Coady et al. (2004) find that nearly a quarter of poverty-targeted programmes in low-income countries are regressive, providing more benefits to rich households than poor. These targeting errors occur because in many low-income settings, governments and programme administrators typically rely on in-person surveys or decentralised community-based nominations, which are costly and difficult to keep up to date.
Big data and algorithms could make poverty targeting more effective
Novel data sources and advances in artificial intelligence have created new opportunities for deploying algorithms to identify beneficiaries remotely, which could potentially overcome the shortcomings of traditional methods (Blumenstock 2020). During the COVID-19 pandemic in the Philippines, for example, the government used artificial intelligence methods that leveraged satellite data alongside traditional sources to identify beneficiaries for an emergency food programme, disbursing staple foods to 162,000 households in just two months (Development Asia 2020). In low-income settings, such new approaches – using satellite imagery or mobile phone usage data – are attractive because digital data can be obtained at a fraction of the cost of in-person surveys and can therefore be updated more frequently. These methods may also scale more easily when large numbers of households need to be screened. Digital data sources may also work better in remote and insecure regions, where in-person surveys are prohibitively expensive or infeasible, and yet are often precisely where many vulnerable people reside.
The dramatic expansion of mobile phone adoption across the developing world offers a potentially rich source of information about its users at a granular level. As of 2024, 84% of adults in low- and middle-income countries own personal mobile phones. Amongst adults who do not, 30-50% use someone else’s (World Bank 2025). Researchers have begun exploiting the relationship between people’s mobile phone usage behaviour and their socioeconomic status to develop machine learning models that can help identify the poor (Blumenstock 2015, 2018, Aiken et al. 2022, 2023). The basic idea is that if low- and high-income individuals use their mobile phones differently – for example, their phone credit top-up behaviour, their messaging versus phone call usage, or the size and spatial distribution of their call contacts vary – then detailed data on people’s phone usage patterns can be used to indirectly infer their poverty status.
In recent research (Aiken, Ashraf, Blumenstock, Guiteras, and Mobarak 2025), we compared the performance of this phone-based targeting approach against traditional approaches like proxy means testing and community-based targeting. Using rich data from multiple sources in the Cox’s Bazar district of southern Bangladesh, we evaluate which approaches are most accurate and most cost-effective. Our evaluation is conducted in the context of cash transfer programmes that were deployed by GiveDirectly and the Government of Bangladesh in 2023 in communities hosting over a million Rohingya refugees from Myanmar. The programme provided US$5 million in cash transfers, which were intended for the poorest 21% of households within the programme area.
Comparing methods of poverty targeting in Bangladesh
Details of data collection and each method are elaborated in Figure 1. To train machine learning models, the ‘ground truth’ of a household’s poverty status is established by conducting detailed consumption surveys. Our phone-based targeting is based on methods developed in Aiken et al. (2022), which form a poverty score for each household by merging their mobile phone records with responses to consumption surveys. The phone data were obtained from four major mobile phone operators in Bangladesh. From the phone records, we construct thousands of metrics of mobile phone use, including detailed information on the timing and frequency of calls, social network structure, and mobility.
We separately calculate a poverty score based on a more traditional proxy-means test, which uses information on roughly 40 household characteristics to estimate each household’s poverty status. Our community-based targeting is adapted from methods widely used by the largest NGO in the country, BRAC, where local community members are invited to a location within the community to discuss and rank the community’s households according to their socio-economic status.
Figure 1: Data and methodology
Big data is not the most accurate at targeting poverty
In Bangladesh, the proxy-means test (PMT) is substantially more accurate than the other methods at predicting household consumption poverty. Phone-based targeting is less accurate than PMT, but more accurate than CBT (Figure 2). This ordering is consistent across multiple measures of accuracy. The different methods use information from different sources: while the PMT relies mainly on demographics, assets and household location to predict poverty, phone-based targeting places importance on indicators like “recharge behaviour” (amount of money topped up to SIM card), aspects of mobility inferred by cell tower location, and the number of unique numbers the user has called. Although the level of accuracy in the Bangladesh programme is relatively low across the targeting methods (recall of 23-52%), it is well within the range of results reported in past work (Aiken et al. 2022, Schnitzer and Stoeffler 2022, Brown et al. 2018) (Figure 3).
Figure 2: Predictions vs. survey-based household PCE, by method
Notes: These figures show scatterplots of per-capita consumption predicted from PBT (left) and PMT (centre), and rankings from CBT (right) vs. household PCE per capita as measured in the household survey. Produced using one train-test split.
The relative performance of phone-based targeting and PMT echoes prior work in Togo that compares the accuracy of phone-based targeting and PMTs. There, the gap in performance between phone-based targeting and the PMT was narrower, but the PMT was more accurate.
Figure 3: Comparison of targeting accuracy with other studies
Notes: These figures show comparisons of our results on targeting accuracy (red stars) with past studies that also use a quota approach to targeting evaluation (green squares from Schnitzer and Stoeffler (2022), blue diamonds for Brown et al. (2018), and orange dots for Aiken et al. (2022)). The targeting error rate is shown as a function of the targeting quota.
When should we rely on algorithms to target poverty?
Accuracy alone does not tell us which method is better to implement, given that the costs of each method are quite different. In particular, the screening costs associated with conducting in-person surveys with each household are substantially higher than the cost of obtaining mobile phone data. To reason through these trade-offs, we adapt a framework proposed by Hanna and Olken (2018), which assumes that money saved on screening and data collection can instead be transferred to beneficiaries. We also assume, based on a Constant Relative Risk Aversion (CRRA) utility function, that poorer households will benefit more from an equal-sized cash transfer – so that more accurate targeting will produce larger welfare gains than less accurate targeting, all else equal. Figure 4 presents results benchmarking the welfare gains of using each method against the scenario of perfect targeting with zero costs.
Figure 4: Gains by targeting method
For a programme the scale of GiveDirectly in Bangladesh, proxy-means testing is the most cost-effective, despite it being more expensive than phone-based targeting (Figure 4). The increase in accuracy from proxy-means testing over other methods was sufficiently large to more than offset the higher cost of implementation. The opposite was found for the programme in Togo, which had a similar budget but twice the number of households to screen for eligibility. Although proxy-means testing was the most accurate there as well, phone-based targeting emerged as the most cost-effective, delivering welfare gains 3% higher (Figure 4). These contrasting results illustrate that cost-effectiveness is sensitive to the costs and accuracy of the approaches, but more crucially, to the scale of the aid programme.
Crucially, the most cost-effective approach depends on the size of the programme budget relative to the number of households that need to be screened. When funds are large relative to the number of households, it may be better to prioritise accuracy over costs. When funds are smaller relative to the number of households, lower-cost methods (like algorithmic phone-based targeting) scale better. Most national-scale programmes have smaller budgets per household screened than the GiveDirectly programmes we analyse. For example, government programmes in Bangladesh typically have budgets ranging from $10-300 million and screen all 41 million households in the country for eligibility (World Bank 2021). To show how our insights generalise to a wider range of social protection programmes, we use our estimates of accuracy and costs to assess which approach to targeting would be most cost-effective in each of the 95 real-world programmes that appear in a World Bank database.
Figure 5 plots programmes in terms of their budget and screening scope, with dashed lines to indicate the threshold at which the preferred targeting method changes. PMT is recommended for programmes with larger relative budgets (blue dots), while phone-based targeting for those with small relative budgets (green dots). For the red dots, the optimal choice for the programmes depends on the exact cost of conducting a PMT screening survey.
Figure 5: Social assistance: Total budgets vs. households screened
Final considerations for implementation
Our work compares algorithmic targeting to several traditional methods of poverty targeting and shows that the optimal approach depends not only on targeting accuracy but also on budget and programme scale. More pragmatically, we note that there are several other considerations that policymakers must account for in determining which approach to targeting is best in a given context. In particular, care must be taken to ensure that households without mobile phones can have paths to enrolment (in our case, 96% of households had phones, but in other settings, that number may be lower).
There is also an active discussion in policy circles about how to enable responsible use of ‘big’ data, while also preserving the privacy of individuals (de Montjoye et al. 2018) and minimising risks of algorithmic bias (Aiken et al. 2022, Friedman and Nissenbaum 1996). In addition, programmes must proactively address the regulatory and practical issues involved in accessing such data; in our case, this required multi-stakeholder collaboration between private data providers and the government, with robust protocols for data access and regulatory oversight.
In summary, algorithmic approaches to targeting social protections are far from a panacea. However, there are specific circumstances – and in particular, when a large population needs to be screened for a programme with a relatively small budget – where phone-based targeting should be carefully considered.