Everyone Has the Same AI Models. Here's What Sets Agents Apart - Richmond Alake, Oracle
Richmond Alake argues that after the control plane, agent architectures need an intelligence plane covering what an agent knows, remembers, is allowed to see, and how it improves. He draws on the WWII Midway codebreaking story to separate information from intelligence, then demos a governed agent using an ontology and semantic layer that answers correctly about 99% of the time with far fewer tokens than a naive agent. The talk also covers Oracle's agent memory package, deep data security, private inference, and fine-tuned decision models for memory routing.
Beyond the control plane: the case for an intelligence plane
Agent memory is the mechanism an agent uses to retain, reuse, recall, and refine information. Its taxonomy runs from short-term versus long-term storage to cognitive functions such as semantic, episodic, procedural, and working memory. The argument here is that once memory is modeled deliberately, a further layer becomes necessary. The control plane, a centralized system for governing agents through identity, policy and guardrails, orchestration and permissions, and auditing, answers what an agent is allowed to do. It does not answer what the agent knows, what it remembers, or how it improves.
The term software factory, coined by Bob Bemer at General Electric in 1968, describes a repeatable, measurable way of producing software. Multi-agent systems that follow systematic workflows are a modern descendant of that idea. Layered on top of these is the intelligence plane, which cuts through the whole agent stack.
Information is not intelligence
The central analogy comes from World War II codebreaking. Two groups held fragmented intercepted Japanese naval communications: one in Washington, centralized and well resourced, and one at Pearl Harbor, working in a basement and recording messages on paper over months. The Pearl Harbor team, which functioned as a memory system, worked out that the code word AF stood for Midway, and that conclusion shaped the Pacific War. Both groups had the same information. The difference was in how the information was stored, interpreted, and built on over time.
The same applies to agents today. Teams have access to the same frontier models and much of the same data, so raw access does not create a differentiator. Intelligence comes from what the system knows, how it recalls and structures memory, and how it learns from what it observes.
Defining the intelligence plane
The intelligence plane is the part of an agentic or AI infrastructure that lets an agent improve over time. It covers four concerns. Knowledge is what the agent should know, supplied through ontologies and semantic layers. Memory is what it should retain. Governance and data access define what it is allowed to see. Continual learning determines how it gets better.
The framing extends a familiar pattern from networking, where a knowledge plane was proposed above management and control planes to observe and reason about network state. The proposed architecture for agents places an agent runtime on top of a memory manager and data platform. Traces from the runtime feed the memory manager, which feeds a continual learning loop that updates the models the runtime uses. A useful equation summarizes the progression: agent equals model plus harness; control adds governance and security; intelligence adds knowledge, memory, and learning, which multiplies the value of the rest.
The industry has moved through stages: prompt tooling, then context layers, then control planes. The next stage is the intelligence plane. Several conditions make this the right time. Scaling laws have made frontier model gains predictable, open models trail by a few months, and decision models, which are small models that make routing and selection choices, have changed how architects think about retrieval pipelines. Models can now sit at many points inside the retrieval pipeline rather than only at the generation step.
Demo: governed versus naive analyst
In a comparison of two analyst agents answering the same questions, the governed agent, built on an ontology layer and semantic layer, answered correctly about 99% of the time while using roughly 32K tokens. The naive agent used about 200K tokens. The evaluation considered tokenomics as a combined measure of cost, accuracy, and latency. Prompt caching and guardrails were also part of the governed setup.
The ontology mattered most for entity resolution. When asked about a counterparty, keyword search returned entities without the right answer, while the ontology-backed agent resolved the term correctly. For the question of what concentration means in a financial context, the naive agent answered incorrectly and the semantic-layer agent answered correctly. The remaining error gap is small, but in mission-critical finance workloads such as fraud detection, a few percent can translate into very large costs.
Security, private inference, and data ownership
Several practical points follow from this architecture. Memory should be treated as data, with entities and definitions written down explicitly and entitlements pushed into the data layer. Traces of agent activity should be captured deliberately, because they become the training data for fine-tuning. Memory is the durable moat, so teams need to own it.
Deep data security makes security a first-class property of the database. The user’s identity, row-level access, and application context propagate into the data layer, so an agent acting on someone’s behalf can only reach the data that person is permitted to see. For organizations that cannot send data to external providers, private inference runs embedding models, LLMs, and re-rankers inside a container on premises or in an air-gapped environment. Hosted options are also available for models such as Grok and some OpenAI models.
Oracle’s agent memory package packages memory engineering as a Python library that generates context for the agent. A private agent factory offers a no-code way to assemble agents on the Oracle AI Database, with the same memory system underneath.
Fine-tuning small decision models for memory routing
A second open-source project, Memories, shows how the intelligence plane can be extended with models. The example uses three model types: a decision model, an embedding model, and a re-ranker. Memory routing is a key case. Output from the context layer or LLM must be sent to the right store, such as conversation memory or workflow memory. Hand-written code can do this, but a fine-tuned decision model does it more cheaply and more easily.
The training data comes from conversations with the agent, where each memory item is classified by type. Fine-tuning a small open-source model on that data gives a router that is fast and inexpensive, and it can improve quality and in some cases reduce latency. Memory routing, embeddings, and re-ranking are presented as the lowest-hanging fruit for teams starting on this path.
Continual learning and owning your memory
Continual learning can happen in token space, weight space, or latent space, the last of which includes prompt caching. The demo showed the token-space version in practice. The main takeaway is that engineers building these systems need to own their memory, engineer trace capture deliberately, and build the pipeline that lets models keep learning from it.