909: Causal AI — with Dr. Robert Usazuwa Ness

From Correlation to Causality in AI

While humans and even animals have an intuitive grasp of cause and effect, most AI systems have been built on correlation-based learning. This is largely because traditional causal inference methods are rooted in classical statistics, focusing on complex problems like isolating a true causal relationship from statistical association that may be influenced by confounding variables (shared common causes). Early applications of deep learning in this space were aimed at scaling these statistical methods to handle non-linear relationships and higher dimensions, but the underlying framework remained the same.

In contrast, the field of cognitive science studies how humans reason causally, focusing not on statistical ground truth but on emulating the reasoning processes themselves. Causal AI draws inspiration from this, aiming to build systems that can understand and generate “causal narratives.” The goal is to shift the focus from what can be gleaned from a dataset to making explicit, honest assumptions about the data-generating process itself.

How to Add Causal Elements to an AI System

A practical way to introduce causality into AI is by leveraging the deep connection between probabilistic graphical models and causal models. A Bayesian Network, for instance, becomes a causal model when its directed edges are interpreted as causal relationships. The theoretical underpinnings, such as Judea Pearl’s do-calculus, are based on manipulating these graphs to simulate an intervention in the data-generating process.

This approach allows practitioners to exchange explicit causal assumptions for the ability to simulate an experiment, something that is often expensive, unethical, or infeasible to run in the real world. For example, using a do operator in a probabilistic model allows one to simulate an intervention post-hoc—analyzing what would happen if a variable were changed, even with observational data. This is not a “free lunch”; the validity of the simulation hinges entirely on the accuracy of the causal assumptions provided.

Key Libraries and Tools for Causal AI

Modern tools have made implementing these concepts more accessible by separating causal abstractions from the underlying statistical and computational complexity.

• Probabilistic Programming Languages (PPLs): Libraries like Pyro (built on PyTorch), NumPyro (NumPy/JAX), and PyMC provide the scaffolding for building sophisticated deep probabilistic models. They can model Bayesian networks, handle latent variables (critical for representing confounders), and use modern inference algorithms like stochastic variational inference. Many, like PyMC, now include a do operator to directly model interventions. These are ideal when you need to embed neural networks within your causal model or work with high-dimensional data (e.g., vectors, images).

• Dedicated Causal Libraries: The DoWhy library (part of the broader PyWhy suite) is designed for applied causal inference, particularly with conventional tabular data you might find in a pandas DataFrame. It allows the user to focus on specifying the causal graph and the question, while it handles the estimation using various statistical techniques (e.g., propensity scores, double machine learning). This abstracts away the choice of a specific statistical estimator, letting the user focus on the causal modeling itself.

The Intersection with Generative AI and LLMs

The relationship between causal reasoning and Large Language Models (LLMs) is a burgeoning area of research with two primary dimensions:

1. Using LLMs for Causal Reasoning: LLMs can act as a “causal knowledge base.” By synthesizing information from their vast training data, they can perform tasks like “causal discovery” (e.g., determining if A causes B or B causes A) without statistical data. They can be prompted to propose a causal graph (DAG) for a given problem, generate the corresponding analysis code in a library like DoWhy, and formalize natural language questions into precise causal queries. However, they are still prone to hallucination and can make subtle but critical errors, requiring careful expert validation.

2. Using Causality for Better Generative AI: A more forward-looking application is using causal structures to improve generative models. For example, one could build a generative model where each node in a causal graph is its own generator, conditioned on its parents. This would create a system that both generates realistic data and adheres to a specified causal structure. In multimodal models (text, video), causality could help the model learn underlying physics or game mechanics, enabling it to generate novel content that respects those rules, rather than just mimicking the training data’s surface-level statistics.

A Typical Causal AI Workflow in Practice

A structured workflow helps navigate the complexities of a causal analysis project:

1. Define the Causal Question: Start with a clear, specific question. For example, “Does increasing engagement in side-quests cause players to spend more money on in-game items?”

2. Specify Causal Assumptions: Formalize your beliefs about the data-generating process, typically by creating a Directed Acyclic Graph (DAG). This graph should include the treatment, the outcome, and any potential confounders (e.g., “guild membership” might influence both side-quest engagement and spending).

3. Identification: Use a library like DoWhy to determine if the causal question can be answered given the specified DAG and the available data. If the answer is no, you may need to collect data on additional variables (e.g., an instrumental variable) to make the effect identifiable.

4. Estimation: Once the effect is identified, choose a statistical method to estimate its magnitude. This could be anything from a linear model to more advanced techniques like propensity score matching or double machine learning. The choice involves standard statistical trade-offs regarding bias, variance, and stability.

5. Refutation (Sensitivity Analysis): Test the robustness of the conclusion. This involves systematically checking how sensitive the result is to violations of your initial assumptions. For example: What if an unobserved confounder was missed? What if the relationships aren’t linear? This step is crucial for understanding the confidence in your causal claim.

• Tags:
• LLMs
• Feature
• Causal AI
• Probabilistic Programming
• DoWhy
• Pyro
• Judea Pearl
• Machine Learning