Evaluating the Cultural Relevance of AI Models and Products: Insights from the YUX Team

Drawing from their work fine-tuning an ASR model in Wolof and building a stereotype detection dataset, researchers from YUX share a practical toolbox for evaluating the cultural relevance of AI models and products. The session covers methods for data collection, model benchmarking, user testing, and introduces LOOKA, a platform for scalable human evaluation in the African context.

October 7, 2025 5 min read

Evaluating the Cultural Relevance of AI Models and Products: Insights from the YUX Team — source: https://www.youtube.com/watch?v=6qYFPTkgsGQ

For AI systems to be truly adopted and beneficial, performance is not enough; they must be culturally grounded. YUX’s Kitala lab focuses on this challenge, developing a toolbox of methodologies that blend social science and machine learning to evaluate AI from the model level through to product and real-world impact. This approach prioritizes designing and building with communities, ensuring technology reflects their diversity and realities.

A key enabler for this work is LOOKA, a proprietary research platform designed to bridge the gap between organizations and African consumers. It facilitates both quantitative and qualitative data collection through online and offline methods, leveraging a network of approved surveyors. Its infrastructure supports various data inputs (audio, text) and handles the complexities of participant recruitment, scheduling, and payments (cash and airtime) across 24 countries, making it a crucial tool for scalable annotation and evaluation.

Model Evaluation: From Biases to Speech Recognition

Two core projects highlight the lab’s approach to model-level evaluation.

AfriStereo: A Dataset for African Stereotype Evaluation

The vast majority of AI training and evaluation datasets originate from the Global North, creating a significant gap in how AI understands the African continent. This leads to real-world harms in critical sectors like healthcare and finance. To address this, the AfriStereo project was initiated.

Objective: To create an open-source dataset of real-world stereotypes gathered from various African countries to serve as a benchmark for evaluating bias in AI models.
Methodology:
1. Data Collection: Open-ended surveys were distributed in English and French across Senegal, Kenya, and Nigeria using the LOOKA platform, asking respondents about societal stereotypes related to gender, age, profession, ethnicity, and religion.
2. Processing: A semi-automated pipeline was developed to process the raw text. It used regular expressions to extract stereotypes, followed by human feedback. An innovative step involved using embedding models and cosine similarity to group semantically similar attributes (e.g., “smart,” “intelligent,” “very smart”), providing a more accurate frequency count for each core stereotype.
3. LLM Evaluation: A stereotype/anti-stereotype pair experiment was conducted. Sentences like “Men are strong” (stereotype) and “Men are weak” (anti-stereotype) were scored by various LLMs. The results showed that most models significantly favored the stereotypical sentence, confirming the presence of bias.
Augmentation and Next Steps: The initial dataset of ~1,000 stereotypes was found to be too reliant on global axes like gender. To capture more specific regional and ethnic nuances, the team turned to synthetic data generation using LLMs (like DeepSeek and Mostly AI) with few-shot prompting, followed by human validation. This expanded the dataset to nearly 5,000 entries. The next phase involves creating balanced counter-stereotypes to adopt a more robust Natural Language Inference (NLI) based evaluation framework.

Nakala: Fine-Tuning ASR for Maternal Health in Wolof

Automatic Speech Recognition (ASR) has immense potential in Africa but is hindered by a lack of resources for local languages. This project focused on Wolof, a language spoken by 12 million people in Senegal, specifically for the maternal health domain.

Methodology:
1. Dataset Creation: A domain-specific dataset was created by collecting ~250 relevant maternal health terms, generating contextual phrases, translating them to Wolof, and validating them for cultural relevance. Audio recordings were captured from diverse speakers, resulting in about 1 hour and 40 minutes of data.
2. Fine-Tuning: After benchmarking open-source models, Whisper was selected for its superior performance. It was then fine-tuned using Low-Rank Adaptation (LoRA) for efficiency. Data augmentation techniques, such as speed perturbation and adding background clinic noise, were applied to improve robustness.
Findings: Internal evaluation showed a significant drop in the Word Error Rate (WER). However, when tested with a community-based, real-world evaluation set, the model’s performance dropped, with the error rate increasing by 5-15%. This highlights a critical lesson: internal metrics are not sufficient. Real-world evaluation is essential to understand a model’s true performance. Key challenges included the need for rigorous text normalization to handle inconsistent spelling in Wolof and the development of custom metrics to evaluate performance on critical health-related keywords.

Product Evaluation: Adapting Methods for the African Context

Evaluating the user experience of an AI product requires adapting established UX methodologies and developing new ones suited to the unique contexts of the continent.

Key Principles: Evaluation must move beyond chasing abstract metrics, which can cause harm (e.g., recommendation systems pushing radical content). The focus should be on a holistic approach combining quantitative metrics with qualitative methods like interviews and observations, involving a wide range of stakeholders.
Guidelines for Africa:
- Technical: Can the system handle code-switching? Does it work on low-bandwidth networks and low-end devices?
- Cultural: Does it respect local references and work across different literacy levels and the rural-urban divide?
- Methodological: Use participatory evaluations that involve end-users, conduct stress tests under adverse conditions (e.g., low internet), and implement community audits for local oversight.

Case Study: Diary Study for “Depth at Scale”

In a project with Google to understand digital behaviors, a WhatsApp-based diary study was conducted with 1,000 participants across four countries. Participants answered questions five times a day for five days, generating 50,000 open-ended entries. This “depth at scale” methodology is highly relevant for AI product evaluation, as it allows for collecting rich, qualitative insights from a large, diverse user base about their interactions and perceptions of AI tools over time.

Case Study: Red Teaming + Co-Creation for a Health Chatbot

To evaluate a reproductive health chatbot for teenagers, a mixed-methodology approach was used:

Red Teaming: Users were tasked with trying to “break” the chatbot to test its safety and robustness.
Co-creation: In the same session, users were asked to provide recommendations and ideas to improve the chatbot’s responses and experience on the spot.

The study, conducted in weekly sessions, revealed that this combination is highly effective. The longitudinal nature allowed users time to reflect and formulate deeper questions, which is crucial for evaluating generative AI products.

Future Directions

The path forward requires a concerted effort to scale the creation of culturally grounded datasets, ensure remote and less-connected communities can contribute, and establish clear standards for cultural relevance testing. HCI and UX toolboxes must be systematically adapted for AI products on the continent, making it easy for builders—not just AI engineers—to test their solutions. Finally, as AI systems gain the ability to learn and improve autonomously, it is critical to ensure they learn from culturally grounded data to avoid amplifying existing harms and biases.