How to Build AI-First Organizations — with Jacob Miller and Jeremy Mumford

The Philosophy of Architected Intelligence

The core goal of “Architected Intelligence” is to establish evergreen principles for building AI-first organizations that will remain relevant for years, moving beyond transient, tool-specific tutorials. The key is to shift focus from the models themselves to the architecture, processes, and data that enable them.

Core Principles for AI-First Design

The User Agnosticism Tenet (UAT)

A foundational principle is to design products and processes so they can be executed by a human, an AI agent, or any hybrid combination. This approach avoids costly refactors and creates resilient systems.

A concrete example is a sales or customer service chatbot. The system should be architected so that a human representative can seamlessly enter the conversation at any point, picking up exactly where the AI left off. This prevents user frustration and allows for the optimal application of human and artificial intelligence. This pattern applies broadly, from sales and support to hybrid coding workflows with tools like Copilot.

Failing Fast is a Superpower

In the traditional software development lifecycle—define, build, feedback—the “build” phase was the primary bottleneck. This is no longer true with AI, where prototypes can be created almost instantaneously. The new bottlenecks are definition and feedback.

Organizations must adapt to this new reality:

• Prove Value on Day One: If a prototype doesn’t show immediate value, it should be discarded quickly. The ease of creation means there’s little reason to stick with a flawed initial concept.
• Empower Small Teams: To encourage rapid iteration and reduce the fear of failure, empower small, autonomous teams. Giving a few people a clear objective and the freedom to experiment within that lane allows them to fail and learn without the pressure of a large audience.
• Shift Focus: The most critical work is no longer coding but getting the specifications and definitions right from the start. As one guest noted, some engineers have gone from spending 80% of their time coding to 80% of their time perfecting spec documents.

Data as the True Foundation of AI

A recurring theme is that most AI problems are fundamentally data problems. The analogy from the movie Anchorman serves as a perfect illustration: “Anything you put on the teleprompter, Ron Burgundy will read it.” An AI system operates on the same principle; it will treat the data and context it’s given as authoritative truth.

Hallucinations are a Data Curation Problem

Teams often mistakenly blame the model for hallucinations. However, a hallucination is more accurately a signal of a data or context gap. It’s the model’s attempt to improvise because it wasn’t given the right script. A hallucination indicates one of two failures:

1. Missing Context: The necessary information was never provided to the model.
2. Failed Retrieval: The information exists, but the retrieval system failed to find and surface the correct document or data chunk.

The Rise of the “Canon Manager”

To address this, organizations need a new role: the Canon Manager. This person or team is responsible for curating the “canon”—the body of information the AI system accepts as truth. This role moves the responsibility of context accuracy from engineers, who are not subject matter experts, to the people who actually own the information (e.g., HR, product, legal). The challenge is to create frictionless tools that allow these experts to easily publish, maintain, update, and resolve conflicts within the organization’s knowledge base.

The Proper Path to Building with AI

There is a natural and more robust progression to developing AI capabilities that many teams skip at their peril. The idea is to build the “hands” before building the “brain.”

1. Start with Skills

A skill is a simple, well-defined, and reusable tool or action that an AI can perform. Teams can quickly create and iterate on these skills, which are often just simple markdown files describing an action.

2. Assemble Workflows

Once a collection of reliable skills exists, they can be composed into a workflow. A workflow is a deterministic, predictable sequence of operations (e.g., “use skill A, then B, then C”). It is orchestrated and controlled. The key advantage is reliability; if you cannot get a deterministic workflow to produce the desired outcome, an unpredictable agent that has to guess the sequence will certainly not perform better. Workflows are also faster, cheaper, and less risky.

3. Graduate to Agents

Only when a process is too complex or dynamic for a predetermined workflow should you graduate to an agent. An agent is given an objective and a set of tools (skills) and must autonomously determine the correct sequence to achieve the goal. Rushing to agents without first building the foundational skills and workflows is a common cause of failure.

Models, Performance, and Practical Trade-offs

The Fine-Tuning Trap

The story of BloombergGPT serves as a cautionary tale. The company spent an eight-figure sum to create a specialized financial model that, for a few weeks, outperformed general models on financial tasks. It was then quickly surpassed by the release of GPT-4, rendering the investment obsolete.

Fine-tuning should be approached with extreme caution. It is often a misallocation of resources unless the use case meets specific criteria:

• Narrow, Stable Domain: The problem is highly specific and not subject to frequent change (e.g., a massive-scale classification task).
• Clear ROI: The performance gain (e.g., from 93% to 95% accuracy) justifies the significant opportunity cost of engineering time that could be spent on other projects.

Cheating the Intelligence-Cost-Latency Triangle

You can optimize for any two of the following three, but not all three: high intelligence, low cost, and low latency. However, there are techniques to “cheat” this triangle:

• Streaming: The most effective technique for user experience. It provides perceived speed by showing the response as it’s generated.
• Caching: Semantic caching and provider-side caching can dramatically reduce cost and latency for repeated queries.
• Dynamic Routing: The hardest to implement effectively. The idea is to route simple queries to cheap/fast models and complex ones to smart/slow models, but accurately classifying query difficulty on the fly is a major challenge.
• UX Design: Clever user experience design can hide latency. For example, an application can kick off an AI process in the background while the user is typing or completing another step, making the result appear instantaneous.

Velocity: The Only Durable Moat

In a world where all organizations have access to the same frontier models, the only durable competitive advantage is velocity. The winner is the organization that can run the build, measure, learn, improve loop the fastest. This advantage is a function of two things:

1. Velocity: The raw speed at which you can complete cycles.
2. Taste: The quality of improvement you achieve with each cycle, driven by deep subject matter expertise and understanding of user value. The ability to out-iterate the competition is the ultimate moat.

• Tags:
• AI Agents
• LLM
• Data Curation
• AI Strategy
• AI Engineering
• AI Product Development