Tensor Logic "Unifies" AI Paradigms [Pedro Domingos]

Pedro Domingos posits that a field truly takes off only when it finds its fundamental language—calculus for physics, Boolean logic for circuit design. He argues that Artificial Intelligence has been missing its language, leading to a deep schism between its two dominant paradigms:

• Deep Learning: Excels at learning from vast, messy data but is poor at transparent, verifiable reasoning.
• Symbolic AI: Provides rock-solid logical reasoning but struggles with learning and scalability.

Tensor Logic is proposed as the language to bridge this divide, creating a unified framework for AI.

The Core Idea: Unifying Tensors and Logic

The “gobsmacking observation” at the heart of Tensor Logic is the fundamental equivalence between an Einstein summation (einsum) from tensor algebra and a rule from logic programming. They are the same operation, distinguished only by the data type they operate on—real numbers for einsum, Booleans for logic.

Tensor Logic formalizes this by generalizing the core database operations of join and projection to the numeric domain, creating tensor join and tensor projection. These constructs are more general than einsum and can express the full gamut of operations in deep learning, symbolic logic, graphical models, and kernel machines within a single, consistent syntax. The language is built on one primary construct: the tensor equation.

Key Capabilities of Tensor Logic

Reasoning in Embedding Space and the “Temperature Knob”

One of the most powerful features is the ability to perform sound, transparent reasoning directly within embedding spaces. This is achieved by controlling a “temperature” parameter, which adjusts the stiffness of a sigmoid non-linearity.

• At Zero Temperature: The sigmoid becomes a step function. The similarity matrix (formed by dot products of embedding vectors) is discretized to 0s and 1s. In this mode, the system performs purely deductive reasoning, guaranteeing soundness—conclusions are guaranteed to follow logically from the premises. This directly addresses the hallucination problem in LLMs, providing a verifiable, reliable mode essential for enterprise applications.
• At High Temperature: The system operates in an analogical reasoning mode. It generalizes inferences from one object to another based on the similarity of their embeddings (their dot product). This allows the system to explore a vast search space analogically, like a mathematician using intuition, before “cooling” the temperature to produce a rigorous, deductive proof.

The temperature can even be set on a per-rule basis, allowing a mix of hard constraints (laws of physics, business logic) and softer, learnable rules.

Structure Learning and Predicate Invention

Tensor Logic moves beyond merely learning weights within a fixed architecture. It enables structure learning through gradient descent. By defining a general rule schema as a tensor equation, the learning process, analogous to a Tucker decomposition, discovers the optimal structure.

This extends to what Domingos calls the “holy grail” of AI: predicate invention. The system can discover new concepts, relations, and predicates that were not explicitly present in the initial data but are necessary to explain it more effectively. This is how humans move from seeing pixels to seeing “objects.”

Universality and Abstraction

Tensor Logic is Turing-complete and provides the right level of abstraction for AI development. Complex models like Transformers or Convolutional Neural Networks (CNNs) can be expressed in just a few tensor equations, as opposed to pages of Python code. This clarity not only simplifies development but also makes the models and the learning process more interpretable. The goal is to provide the language for a “universal induction machine”—the learning equivalent of the Turing machine for deduction.

The Path to Adoption and the “Trillion-Dollar Waste”

Domingos outlines a clear path for adoption:

1. Solve Critical Pain Points: Tensor Logic offers potential solutions to hallucination and the black-box nature of current models.
2. Education: Its elegance and unity make it an ideal language for teaching the entire spectrum of AI concepts.
3. Gradual Migration: A pre-processor can convert tensor equations into standard PyTorch or NumPy code, allowing developers to integrate it into existing workflows without rewriting everything at once.

He concludes with a provocative claim: the AI industry has “wasted trillions of dollars” on brute-force compute. Companies are painfully “reinventing reasoning” when 40 years of research in symbolic AI has already laid the foundations. Instead of building ever-larger models to approximate reasoning, he argues, we can build it in from the ground up.

“Why don’t you spend an afternoon reading a couple of chapters of Russell and Norvig and save a hundred billion dollars in wasted compute? Please, just do that.”

• Tags:
• Feature
• Deep Learning
• Reasoning
• Hallucination
• Tensor Logic
• Symbolic AI
• Unification
• Pedro Domingos
• Master Algorithm