Unpacking Anomalies: A Machine Learning Approach to Detecting and Explaining Outliers in Consumption-Based Welfare Data

While much of the literature (e.g., Oliveira et al., 2016; Rizkina et al., 2024) focuses on aggregate measures of consumption and their relationships to poverty, there is scant attention paid to systematically uncovering and understanding outlier households—those whose consumption patterns deviate significantly from predicted norms. This project proposes using unsupervised machine learning (like clustering and outlier detection algorithms) on detailed consumption microdata to flag households or subgroups that don’t fit established trends (e.g., high consumption but low welfare, or vice versa). The real innovation lies in pairing these statistical flags with explainable AI methods to generate hypotheses about why such anomalies exist (e.g., informal transfers, unrecorded debts, or coping strategies). This approach could reveal groups overlooked by standard poverty targeting and contribute to more nuanced welfare policy, directly addressing the heuristic of investigating deviations from expectations.

References:

1. Construction of a Consumption Aggregate Based on Information from POF 2008‐2009 and Its Use in the Measurement of Welfare, Poverty, Inequality and Vulnerability of Families. L. Oliveira, Debora F. De Souza, Luciana A. Dos Santos, Marta Antunes, Nícia C. H. Brendolin, Viviane C. C. Quintaes (2016).
2. The Reaction of Poverty to Consumption and Inflation in Indonesia. Azka Rizkina, Nurhayati Nurhayati, M. Rasyidin, Al Mahfud Saputra, Hijrah Purnama Sari Ariga (2024). Electronic Journal of Education Social Economics and Technology.