The mathematics of AI uncertainty

Zoubin Ghahramani, a pioneer in machine intelligence built on the mathematics of uncertainty, argues that representing and reasoning about uncertainty is fundamental to building truly intelligent AI systems. He contends that current AI, particularly large language models (LLMs), often provide answers with unwavering authority, even when wrong, lacking a crucial element of self-doubt.

The Indispensable Role of Uncertainty

Ghahramani posits that intelligence inherently involves decision-making under uncertainty, as perception is limited and the future is unpredictable. He differentiates between two types of uncertainty:

• Aleatoric Uncertainty: Inherent randomness of the world (e.g., a pedestrian’s unpredictable turn).
• Epistemic Uncertainty: Lack of knowledge or novelty (e.g., a self-driving car encountering a never-before-seen scenario like a horse in a hailstorm). He emphasizes that an intelligent system needs self-awareness about its uncertainty, prompting cautious behavior like slowing down in unfamiliar situations. Mathematically, all forms of uncertainty can be mapped to probabilities.

Human Intelligence and Probabilistic Reasoning

Cognitive science has embraced probabilistic ideas to understand human perception and decision-making. Our senses, refined by evolution, combine perceptual information with prior beliefs (e.g., a rustling in Costa Rica might be a jaguar) through a process akin to probabilistic inference. While humans are often poor at explicitly estimating probabilities, our unconscious perceptual systems excel at it, crucial for survival.

Correctness Versus Confidence

A critical distinction exists between a system being correct and its confidence level. Adversarial examples (e.g., imperceptibly altering an image of a school bus to make a neural network confidently classify it as a cheetah) demonstrate that AI can be overconfidently wrong. This issue persists in LLMs, which confidently generate incorrect or inconsistent answers, eroding trust. Ghahramani asserts that humility and awareness of limits are essential for advanced AI, including Artificial General Intelligence (AGI).

Historical Perspective and the Rise of Bayesian Thinking

Ghahramani’s journey in AI began in the late 1980s, witnessing the transition from brittle expert systems to flexible, data-learning neural networks, which he worked on for natural language processing. By the early 1990s, as the mechanics of neural networks seemed understood, his interest shifted towards the deeper mathematical connections between neural networks and probability theory, statistics, and stochastic processes. This led him to Bayesian machine learning. His 2015 Nature paper argued for the crucial role of careful probabilistic representation of uncertainty in intelligence.

Understanding Bayesian Thinking

Bayes’ rule provides a formal framework for updating beliefs:

1. Prior Beliefs: Initial probabilities before observing evidence.
2. Likelihood: Probability of observing evidence given different hypotheses.
3. Posterior Beliefs: Updated probabilities after incorporating evidence.

This process is iterative; the posterior becomes the new prior with new evidence. This framework models both perception (updating understanding of the world) and learning (updating model parameters) to accumulate knowledge rationally.

Current State of Uncertainty in AI and LLMs

While LLMs inherently use probabilities (predicting the next token), they often lack an explicit and coherent representation of their own uncertainty. They appear to

• Tags:
• Large Language Models
• AI Safety
• DeepMind
• Feature
• Neural Networks
• AGI
• Cognitive Science
• Machine Learning
• Continual Learning
• Bayesian Inference
• Decision Making
• Uncertainty Quantification
• Probabilistic Models
• Epistemic Uncertainty
• Aleatoric Uncertainty