Karl Friston - Why Intelligence Can't Get Too Large (Goldilocks principle)

The Free Energy Principle: A 20-Year Retrospective

The Free Energy Principle (FEP) posits that to exist, a system must act in ways that minimize surprise or uncertainty, effectively creating a predictive model of its environment to maintain its own structure. Professor Karl Friston reflects that while the principle itself is almost tautologically simple—rooted in the mathematics of conditional probabilities—its profound consequences can make it notoriously difficult to communicate. The core idea is that any living thing, from a single cell to a complex organism, must conform to this principle to survive. This involves a constant process of inference and belief-updating to make sense of the world.

From Agency to Consciousness: The Nature of “Strange Things”

A key development in the FEP framework is the categorization of different kinds of systems, or “particles,” based on their causal structure and Markov blankets. Friston distinguishes between simple particles and what he calls “strange things”—systems like us.

In simple organisms, the internal states model the external causes of their sensations. Their active states are directly accessible, and their world is one they can directly touch and feel. In contrast, “strange” systems possess a hierarchical structure where the active states become sequestered from the internal states. This seemingly small change has a profound mathematical consequence:

“From the point of view of the internal states… the active states are now become causes of the sensory states. So now your world is caused not just by the external world… but also by my own actions. So now I’m inferring the causes of my sensorium where I’m actually a cause. I’m inferring myself.”

This creates a recursion, or a “strange loop,” where the system must model itself as an agent. This self-modeling is the foundation of true agency. It necessitates inferring the consequences of one’s actions, creating a “private future” inside the model. The ability to select between different paths into this future is a hallmark of agentic behavior.

The Distinction Between Agency and Consciousness

While this self-modeling provides a robust account of agency, Friston, agreeing with thinkers like Anil Seth, emphasizes that agency and consciousness are orthogonal concepts. Consciousness is a more complex phenomenon, and several theories attempt to explain its emergence:

• Deep, Hierarchical Self-Awareness: Consciousness may not just be about modeling oneself, but about recognizing the act of modeling itself. This involves layers of recursion: recognizing that you are attending to something, and then recognizing that you were recognizing, and so on.
• Precision and “Felt Uncertainty”: Thinkers like Mark Solms argue that consciousness relates to the brain’s representation of precision or confidence in its beliefs, not just the content of those beliefs. Consciousness is the “felt uncertainty” that accompanies our predictions.
• The Inner Screen Hypothesis: Drawing from Chris Fields’ quantum information formulation of the FEP, this idea posits an irreducible “inner screen” or Markov blanket deep within the brain’s hierarchy. This metacognitive core knows itself only by acting upon the rest of the brain, creating a basis for higher-order thought and self-consciousness.

A common thread is that consciousness likely requires a minimum threshold of complexity and a specific causal structure, including significant temporal or “counterfactual depth”—the ability to model a deep and branching future.

The Scale of Intelligence: A “Goldilocks Zone”

The conversation explores whether intelligence is a universal property (pan-intelligence) found in everything from viruses to planets, or if it is restricted to a specific scale. Friston suggests a “Goldilocks zone” for the kind of intelligence we recognize.

• At the Smallest Scales: At the quantum or atomic level, systems are dominated by random, dissipative fluctuations. There is no stable, recurring, or “conservative” dynamic needed to establish the persistent, circular patterns of life and intelligence.
• At the Largest Scales: At the astronomical scale of planets and solar systems, random fluctuations are averaged away, leaving only the predictable, conservative (solenoidal) dynamics of classical mechanics. A planet or the weather system doesn’t plan or model a future; it is too big and lacks the necessary admixture of dissipative and conservative forces.

Intelligence thrives in the complex middle ground where systems like us exist “on the edge of chaos,” constantly using conservative, recurring dynamics (e.g., life cycles, routines) to resist the dissipative, itinerant chaos of the world. Even large human constructs like corporations must artificially impose structure to maintain this balance and function intelligently, but there may be a fundamental size limit, dictated by physics, beyond which this becomes impossible.

Building Conscious AI: The Challenge of Embodiment

Can we build a truly conscious AI? Friston expresses significant doubts about whether this is possible using current von Neumann computer architectures. The separation of processing and memory in such systems makes it difficult for the memory (the substrate) to self-organize in the way required by the FEP.

Instead, genuine consciousness might require what has been termed “mortal computation.” This aligns with Anil Seth’s concept of a “beast machine,” where the physical hardware and the “mind” are inseparable.

• Morphology as the Model: In biological systems, the physical structure is the model. To be a good model of its environment, an organism must embody the causal structure of its world. The substrate itself is the parameterization of the system’s beliefs.
• The Role of DNA: The DNA can be seen as specifying the priors for the structure of this generative model—not a rigid blueprint, but a set of policies and starting conditions that guide how the organism learns and adapts its physical form throughout its lifetime in response to its unique environment.

This implies that building conscious AI may require fundamentally different hardware, such as neuromorphic or in-memory processing architectures, where the physical form of the machine can dynamically embody the models it learns.

• Tags:
• Computational Neuroscience
• Embodied AI
• Consciousness
• Agency
• Karl Friston
• Free Energy Principle
• Active Inference
• Markov Blanket