Session on Inclusive AI: Data, Models, Evaluation

This session, titled “Inclusive AI: Data, Models, Evaluation,” featured insights from leading researchers on bridging divides in AI across languages, cultures, and contexts, with a particular focus on the Global South, especially India. The speakers highlighted the critical need for equitable AI development and deployment, addressing challenges in data, model adaptation, and evaluation.

Sunayana Sitaram: Bridging Divides in AI Across Languages, Cultures, and Contexts

Sunayana Sitaram initiated the session by highlighting the uneven diffusion of AI globally, which leads to unequal benefits and disproportionate harms, potentially widening existing inequalities. A key cause of this disparity is that AI models do not perform consistently across different languages, cultures, and contexts. She identified three critical gaps:

• Data Gap: Most models are trained on data scraped from the web, which is typically English-centric and high-resource. Post-training data generation is expensive. To address this, Microsoft Research India explored synthetic data generation. A project led by Prashant combined translation (from English) with “bottom-up generation” using native speaker-authored data, aiming to capture cultural nuances. This resulted in a dataset of 9.5 million data points across 13 Indian languages and English, demonstrating strong performance on Indic language benchmarks. Future work involves replicating this for African languages, exploring preference data, and analyzing trade-offs between real and synthetic data.
• Modeling Gap: While acknowledged, the talk did not delve deeply into English-centric architectural choices or training methods.
• Evaluation Gap: Emphasizing that “if you can’t measure it, you can’t improve it,” Sitaram detailed Microsoft’s journey in evaluation research:
  • Early Work (2019-2020): Developed the first code-mixed benchmarks for multilingual societies.
  • Expanding Coverage (Post-ChatGPT): Audited over 250 multilingual benchmarks, revealing persistent gaps in language, script, and family coverage, with Indo-European and Latin-script languages dominating.
  • LLM Evaluation Rigor: Focused on bringing more rigor to LLM evaluation, including the use of LLM judges.
  • Representativeness: Identified that many benchmarks are mere translations of English, lacking crucial cultural and local context. The goal is to create more representative evaluation frameworks.
  • Safety: Expanded evaluation to include safety considerations alongside accuracy.

Microsoft’s Samiksha project, in collaboration with KIA, focuses on community evaluations. This three-stage pipeline involves civil society organizations in defining benchmarks and rubrics, community members (kara workers) in creating data points, and the same kara workers in evaluating model responses for cultural appropriateness, correctness, and local applicability. A large benchmark (23,000 data points across 11 languages, 150,000 evaluations) found that while current models perform adequately on language quality, they struggle significantly with cultural and local nuances. Healthcare, in particular, was identified as the most challenging domain, often yielding impractical or culturally inappropriate suggestions for Indian contexts. Sitaram concluded by posing a fundamental question: how can AI development achieve multilinguality, multicontext, and multiculture by design, rather than as an afterthought?

Danish Pruthi: Richer Outputs for Richer Countries? Geographical Disparities in Image and Language Generation

Danish Pruthi presented compelling evidence of geographical disparities in AI outputs, challenging the

• Tags:
• Synthetic Data
• Tokenization
• Feature
• Low-Resource Languages
• Machine Learning
• AI Ethics
• Legal AI
• Inclusive AI
• Multilingual AI
• Data Gap
• Evaluation Metrics
• Generative AI Bias
• Indian Sign Language