Notion's Token Town — Sarah Sachs, Notion

Sarah Sachs, Head of AI Engineering at Notion, opens by addressing the common trap where reasoning models are upgraded at the same price per token but quietly burn three times the output tokens, or new versions cost 40% more and deprecate predecessors within months. She posits that most companies aren’t growing at a rate to absorb these costs, leading to a “bad deal” for customers or investors. This scenario creates a significant structural barrier for companies, making it difficult to scale AI products successfully.

Notion’s journey into AI began with “AI as a thought partner,” evolving through “AI as an assistant” for individual tasks, and then “AI as teammates” for repetitive workflows. The ultimate goal, “AI as a system,” where processes interface and entire systems run autonomously, remains elusive for most (88% don’t get past assistant level). Sachs attributes this challenge to siloed data and the absence of a durable system of record, which Notion aims to provide.

Supplier as Competitor: The Cost of Vendor Lock-in

Sachs asserts that “your supplier is your competitor.” Buying tokens from a lab that also sells a first-party product means paying a markup on a markup, resulting in a bad deal and value that is hard to defend. Tying a product to a single provider eliminates exit options, creating a precarious business model. She highlights the significant discrepancy between what frontier labs charge customers for first-party products versus what they sell tokens for, terming it “a bad deal.”

Win on Product, Not Token Economics

Instead of focusing on token economics, Sachs advocates for winning on product. This involves:

• Building data flywheels: Understanding customer needs better than anyone else.
• Optimizing for capability, price, and latency: Recognizing that the slowest, most capable model isn’t always necessary.
• Compelling UI and orchestration: Justifying the cost of necessary “bad deal” tokens through superior product experience.

The goal isn’t necessarily to train the best model, but to build the best product that intelligently uses many models, betting on the “frontier” of AI rather than a specific “lab.”

The Model-Agnostic Playbook: Notion’s “AI Switzerland” Approach

Sachs emphasizes that optionality is your leverage in this market. If a company cannot walk away from a provider, it is stuck, making model interoperability a critical investment. Notion’s “AI Switzerland” approach leverages an auto model that routes about 75% of its traffic, dynamically switching between state-of-the-art models and handling vendor lock-in.

Key tenets of this playbook include:

• Cost per Capability per Second: Evaluating models based on this metric, rather than just per token. For instance, triaging an email inbox with an expensive frontier model like Opus is “ripping off the customer and ourselves,” while large-scale data analysis might warrant it.
• Traffic Segmentation: Not all traffic requires frontier models. The market operates as an oligopoly where pricing doesn’t always correlate with true capability growth, meaning companies must be experts in their product’s specific complexity needs.
• Building for Multimodal: Investing in technology to kill the cache and switch models mid-transcript, ensuring model interoperability across the system.
• Granular Evaluation: Making decisions based on entire web search trajectories and all trade-offs, not just single calls or latencies.
• Switching Fast and Often: Adapting rapidly to new and improved models from various providers.
• Partnership Value: Exchanging use case expertise with Frontier Labs through evaluation programs, rather than relying solely on large financial commitments, to gain leverage.

Open-Weight Models and Resource Optimization

Open-weight models are presented as a credible alternative, capable of handling moderate tasks and providing negotiation leverage by putting downward pressure on pricing. Sachs notes models like Kimmy 26 and GLM 52 as examples that are increasingly capable, even for tasks traditionally reserved for frontier models, with the gap closing rapidly.

Furthermore, she argues for CPUs over GPUs where appropriate. Many tasks, such as turning a CSV into a PDF, deterministic SQL queries, or basic tool calls, do not require an LLM. Using expensive LLMs and GPUs for these simpler tasks quickly leads to companies becoming “token poor.” Effective governance, including visibility and control over data and model usage, is also crucial.

Addressing Security: The Lethal Trifecta and Multi-Agent Orchestration

The talk shifts to the critical challenge of security, introducing Simon Wilson’s “lethal trifecta”: the combination of access to private data, exposure to untrusted content, and the ability to communicate externally. This trifecta creates significant risk, especially in autonomous, unsupervised systems. Sachs emphasizes that building valuable product in this context requires robust sandboxes and deterministic systems, enhancing both security and token economics.

A live demo showcases Notion’s approach to multi-agent orchestration, illustrating how agents can scope tasks, tag in teammates for collaboration, and even generate a pull request. This demonstrates Notion’s internal “software factories,” where writing to teams and initial coding steps are coordinated by AI, achieving massive ROI gains (e.g., over three minutes saved per task). The core idea is to automate and augment work through AI, not committing to a specific lab, but to the concept of AI augmenting human effort.

Conclusion

Sachs concludes by stressing the need for critical thinking in the fast-moving, opaque AI market. She encourages professionals to engage in open discussions, prioritize optionality, and understand the specific model requirements for different tasks to build responsible and effective AI products.

• Tags:
• Feature
• AI Strategy
• AI Engineering
• Open-Weight Models
• AI Orchestration
• Vendor Lock-In
• Model Agnosticism
• Model Contracts
• Token Economics
• Notion AI
• LLM Cost Management