Tensor Logic "Unifies" AI Paradigms [Pedro Domingos]

A field cannot truly take off until it finds its language. Physics found its language in calculus, and circuit design found its in Boolean logic. Artificial Intelligence has been missing its unifying language, leading to a split between two paradigms that don’t integrate well: deep learning and symbolic AI. Tensor Logic is proposed as this unifying language, merging the tensor algebra of neural networks with the logic programming of symbolic systems.

The Unification of AI Paradigms

Modern AI is divided into two camps:

1. Deep Learning: Systems like transformers are excellent at learning from vast, messy, real-world data but are opaque and poor at logical reasoning, leading to issues like hallucination.
2. Symbolic AI: Systems based on logic programming (like Prolog) are transparent and excel at verifiable reasoning but are brittle and cannot learn effectively from data.

Tensor Logic bridges this gap. It’s not a superficial combination but a deep unification built on a single, core construct: the tensor equation. This unification extends beyond just deep learning and symbolic logic to also encompass graphical models and kernel machines, creating a single framework for the entire gamut of AI.

The Core Idea: Einsum is a Logical Rule

The “gobsmacking observation” at the heart of Tensor Logic is that an Einstein summation (einsum)—the fundamental operation that can express all of deep learning—is mathematically the same thing as a rule in logic programming. The only difference is that one operates on real numbers and the other on Booleans.

Tensor Logic generalizes the core components of database logic—joins and projections—into tensor joins and tensor projections. When the tensors are boolean, these operations become the standard symbolic database operations. When they are numeric, they become generalizations of einsum, allowing for a seamless spectrum between logic and deep learning.

This provides several advantages over using einsum in frameworks like PyTorch:

• Better Abstraction: It provides a cleaner, more intuitive syntax for expressing complex operations, making it easier to think and reason about models. A transformer can be coded in a dozen tensor equations instead of a vast amount of code.
• Efficiency: It allows for deep optimization at the CUDA level, overcoming the performance bottlenecks that currently limit the use of einsum.
• Unified Learning and Reasoning: The same construct handles both numeric computation and symbolic reasoning, including learning the symbolic parts.

Learning Structure and Discovering Concepts

A primary challenge in AI is not just learning weights within a fixed architecture but learning the structure of the model itself. Tensor Logic addresses this directly.

I would say that in some sense discovering representation like that is the key problem in AI is the holy grail.

In Tensor Logic, structure learning falls directly out of gradient descent. By defining a general rule schema with a tensor equation, gradient descent can discover the optimal architecture within that space. This is analogous to how matrix factorization discovers latent factors.

This process enables predicate invention: the discovery of new concepts and relations that are not explicitly in the data but help explain it better. For example, a system can learn to see “objects” instead of just pixels. This happens naturally through a process similar to tensor decomposition (e.g., Tucker decomposition), where gradient descent finds the most compact and powerful representations.

Reasoning in Embedding Space: The Temperature Knob

Perhaps the most powerful feature of Tensor Logic is its ability to perform sound and transparent reasoning directly within a continuous embedding space. This allows for a controllable spectrum between analogical and deductive reasoning, governed by a temperature parameter.

• At high temperatures, the system performs analogical reasoning. It generalizes inferences between objects with similar embedding vectors (i.e., a high dot product), borrowing knowledge from related concepts. This is powerful for discovery and navigating vast search spaces.
• At zero temperature, the system performs purely deductive reasoning. The similarity matrix becomes discretized (identity or zero), and the rules operate in a purely logical mode. This guarantees soundness—conclusions are guaranteed to follow logically from the premises.

This “temperature knob” is a direct solution to the hallucination problem. Unlike LLMs, which can hallucinate even at zero temperature, a Tensor Logic system can be put into a mode where it produces zero hallucinations. This is critical for enterprise and real-world applications where business logic, security, and factual accuracy are non-negotiable.

Every Fortune 500 company if it’s going to use AI needs to have a mode where the logic of the business is just obeyed… and transformers can’t do that. Tensor Logic can do that.

The Trillion-Dollar Waste

A provocative claim is that the AI industry is squandering immense resources by ignoring decades of established research.

Why don’t you spend an afternoon reading a couple of chapters of Russell and Norvig and save a hundred billion dollars in wasted compute? Please, just do that.

Companies are trying to “reinvent reasoning” from scratch with brute-force compute, spending trillions on ever-larger models to re-learn lessons that the field of symbolic AI solved long ago. Tensor Logic offers a path to integrate this knowledge, leading to far more efficient and reliable systems. The goal is to move beyond brute force and build systems that can learn from small examples and generalize to problems of any size, much like humans do.

• Tags:
• Deep Learning
• Reasoning
• Hallucination
• Tensor Logic
• Symbolic AI
• Pedro Domingos
• Master Algorithm