Intelligence = Doing More with Less (David Krakauer)

We are fundamentally confusing being very knowledgeable with being very intelligent. The current trajectory of AI, which relies on consuming ever-more information, is a manifestation of this confusion. True intelligence is not about how much you know, but about the ability to do a lot with very little information—to adapt to novelty and solve problems efficiently. Stupidity, conversely, can be seen as doing less with more.

The Evolutionary Origins of Brains and Culture

The evolution of intelligence can be understood through the limitations of genetic adaptation. Based on Manfred Eigen’s quasi-species theory, there is a fundamental speed limit to evolution, roughly “one bit per genome per generation,” known as the error threshold. This means genetic evolution cannot keep up with information in the environment that changes faster than the generational frequency of an organism.

To overcome this, nature evolved extra-genomic systems capable of acquiring high-frequency information:

• Brains and Nervous Systems: These are inferential organs that process information from the world at a rate far exceeding what genes can.
• Culture: This can be described as “evolution at light speed.” By storing accumulated knowledge externally—in books, libraries, and tools—we can innovate and generate new variations at a much faster rate without corrupting the historical record of what we’ve already learned. Culture effectively breaks the evolutionary speed limit.

The True Meaning of Emergence

The AI community’s use of the term “emergence” is a superficial and misleading appropriation of a deep concept from complex systems.

• The LLM Community’s View (Critiqued): The idea that a sharp, discontinuous jump in a model’s ability to perform a task (like three-digit addition) constitutes emergence is flawed. Achieving this capability by scaling a model to hundreds of billions of parameters is not impressive; it’s what Krakauer calls “really shit programming.” A simple calculator can do this with a memory footprint a billion times smaller. The discontinuity itself is an irrelevant distraction from the core concept.

• The Complex Systems View: True emergence, as articulated by physicist Phil Anderson in his paper “More is Different,” is about a fundamental change in the internal organization of a system. This change allows for the creation of a new, more parsimonious macroscopic description that renders the microscopic details irrelevant. The classic example is the transition from tracking the chaotic interactions of individual water molecules to using the elegant, predictive laws of fluid dynamics (e.g., the Navier-Stokes equations). You do more with less information. The goal is to find a “clever way” to average over the low-level details to get a powerful high-level theory.

What Real Emergence Would Look Like in AI

For a Large Language Model to demonstrate true emergence, we would need to observe more than just improved task performance.

The big objection that Melanie, John, and I have to emergence claims is they’re based only on the external manifestation of a task and not on the corresponding internal microscopic dynamics which you want to somehow map onto the macroscopic observable. That’s the essence of emergence, the micro-to-macro map.

Evidence of true emergence would involve:

1. Breaking Scaling Laws: Current model capabilities are highly correlated with scale (more data, more parameters). A truly emergent system would show a deviation from this trend, achieving greater capabilities without a commensurate increase in scale, signaling a more efficient internal organization has been found.
2. New Internal Representations: The model would need to move from learning “fractured, entangled, microscopic representations” to developing “factored, unified representations” that carve the world at its causal joints. This would be a form of endogenous coarse-graining, where the system itself develops a simpler, more powerful internal model.
3. A New, Parsimonious Theory: We would be able to describe the model’s behavior with a new, simpler set of principles, much like fluid dynamics describes water molecules. We would no longer need to track the individual weights and activations.

“Exbodiment”: The Externalization of Mind

While embodiment refers to outsourcing computation to the physical constraints of our bodies, exbodiment is the process of outsourcing our cognitive load to external objects and cultural artifacts.

• We create tools like an abacus, a map, or even language itself. These artifacts are often created collectively.
• We then internalize the skills these tools teach us. For example, you can memorize a map and then navigate without it.
• This creates a powerful feedback loop, which can be called the embodiment helix: we create external artifacts, internalize the knowledge they represent, use that new internal skill to improve upon the external world, and so on, continually enhancing our collective intelligence.

The boundary of an individual mind is therefore fluid and scale-dependent. An “individual” can be defined as any system that contains sufficient information to propagate itself into the future. For complex ideas like projective geometry, the “individual” capable of propagating that knowledge is not one person but a collective.

The Danger: AI and the Atrophy of Human Thought

Historically, technology has been a net positive, creating tools that complement our natural deficits. We invented calculators because we are bad at arithmetic and rackets because we are not great at hitting balls with our hands.

However, modern AI presents a new kind of risk. Humans have a powerful, evolutionarily-ingrained drive to increase fidelity while reducing energy expenditure. We will, if we can, outsource everything.

The brain is an organ like a muscle. If I outsource all of my thinking to something or someone else, it will atrophy just as your muscles do. There’s nothing confusing about that. It’s just a fact of physiology.

The fear is not that AI will degrade our thinking, but that it already has. We are outsourcing cognitive tasks to systems that are more effective, and as a result, we are losing the skills ourselves. This will lead to a “significant diminution and dilution of what it means to be a human.” As we increasingly rely on LLMs to write and think for us, our own thoughts will become less original and more homogenous.

Superintelligence is only valuable to the extent that it makes us more intelligent, not to the extent that it makes us more stupid, servile, or dependent. The current path risks making us the cognitive equivalent of a deep-sea angler fish—a creature reduced to its absolute minimum, wholly dependent on an external system for its existence.

• Tags:
• Large Language Models
• Emergence
• Complex Systems
• Intelligence
• Cognitive Science
• Evolutionary Theory