Sub-Population Identification of Multi-morbidity in Sub-Saharan African Populations
This talk explores a data science approach to identifying high-risk subpopulations for multimorbidity in sub-Saharan Africa, contrasting traditional hypothesis-driven analysis with a discovery-driven method that automatically uncovers non-obvious patterns in health data.
This work addresses the growing concern of multimorbidity—the co-occurrence of two or more chronic diseases like hypertension, diabetes, and heart disease—which is becoming increasingly common in sub-Saharan African populations. As Africans live longer and adopt different lifestyles, the burden of these chronic conditions grows, leading to premature mortality and a lower quality of life. The research presented is part of a larger NIH-funded initiative, DSI-Africa, aimed at improving data science for healthcare on the continent.
Defining and Visualizing Multimorbidity
For this analysis, multimorbidity is defined as having two or more chronic conditions from a list including diabetes, hypertension, cardiovascular disease, and chronic kidney disease. Based on survey data from Nairobi, approximately 10% of the population meets this definition.
To better visualize the complex intersections of these diseases, the talk advocates for using upset plots as a more structured and readable alternative to traditional Venn diagrams, especially when dealing with four or more categories. An upset plot clearly shows the size of each disease population and the sizes of all possible intersections between them.
Manual vs. Automatic Stratification
A key question following the 10% prevalence finding is: “Who are these people?” The talk contrasts two primary methods for identifying high-risk subpopulations.
1. Hypothesis-Driven (Manual) Stratification
This is the traditional approach, where analysts stratify the outcome based on prior domain knowledge and pre-existing hypotheses. For example, one might analyze the data by:
- Sex: Females were found to be 1.6 times more likely to have multimorbidity.
- Age: Individuals aged 55 and older are 3 times more likely.
- Combination: Older females have an even higher odds ratio.
While useful, this manual, trial-and-error approach is limited. With dozens of potential features (BMI, blood sugar, diet, etc.), it’s impossible to manually explore the exponentially large number of potential subpopulations.
2. Discovery-Driven (Automatic) Stratification
The core of this work is a computational approach that brings scalability and discipline to the search for anomalous subsets. Instead of presupposing which groups to investigate, this method automatically and efficiently searches through all possible combinations of features to find subpopulations where the rate of multimorbidity is unusually high.
This approach is not meant to replace domain expertise but to augment it, uncovering patterns that manual exploration might miss. The underlying method is a form of anomalous pattern detection that maximizes a likelihood ratio statistic to identify the most unexpected subgroup.
Key Findings and a New High-Risk Profile
The automatic stratification method identified a novel, high-risk group that was not conditioned on the usual suspects like age or sex. This group was nearly six times more likely to have multimorbidity and was defined by a combination of three factors:
- Waist-to-hip ratio greater than 0.94
- Triglycerides greater than 1.0 mmol/L
- Carotid intima-media thickness (CIMT) greater than 0.64 mm
This discovered group is distinct from the “older females” group identified through manual analysis, with a Jaccard overlap of only about 10%. Furthermore, the risk profiles were found to be transferable; a high-risk profile identified in the Nairobi dataset also showed elevated rates of multimorbidity when applied to a dataset from South Africa, and vice-versa.
A Machine Learning Perspective
From an ML standpoint, this technique can be interpreted as a method for bias detection or error analysis. The baseline statement that “10% of the population has multimorbidity” can be seen as a simple, underfitting classifier. The automatic stratification process effectively answers the question: “Where is this naive classifier most wrong?” It finds the subpopulation where the model is most biased and its predictions are least accurate.
The code for this method, known as multidimensional_subset_scan or bias_scan, is available as part of IBM’s open-source AI Fairness 360 toolkit. The underlying methodology is broadly applicable beyond healthcare and has been used to analyze representations in foundation models, detect toxicity and stereotypes in language models, and compare outputs of molecular generative models.