Your Moat Is Your Data Model — Mike Phipps, Gates Foundation

The talk by Mike Phipps centers on identifying the durable moat in an era where AI models, frontends, and agent frameworks are rapidly commoditizing. He posits that this moat is not in the ephemeral layers of AI technology, but in an organization’s unique internal processes, tacit knowledge, and, critically, its data model. This foundational understanding, he argues, remains defensible and valuable regardless of how advanced new AI releases become.

Phipps introduces the Strategic Intelligence Platform (SIP), an enterprise-wide system deployed across the 4,000-person Gates Foundation. The scale of the problem SIP addresses is immense: modeling 25 years of grantmaking data, involving over $7 billion in annual disbursements, 2,000 grants, 100+ countries, and numerous internal divisions and strategies. The core challenge is to extract scalable, data-driven insights from this vast and complex operational landscape.

SIP Architecture and Data Modeling

SIP is designed to structure this operational data specifically for agentic retrieval, meaning the knowledge graph is built with AI agents as the primary consumers. The end-to-end architecture involves:

1. Data Lakehouse: Siloed structured and unstructured data from various systems of record are consolidated into a central data lakehouse.
2. Data Curation Layer: This crucial layer processes the raw data:
   • Pre-processing: Filtering, deduplication, handling inconsistencies across documents.
   • Extraction: Structured field extraction, semantic chunking for unstructured documents (including converting figures to text), and extensive tagging to establish connections in the graph. Additional metadata is created during this process.
   • Governance: A critical component given AI’s increased accessibility to data. This includes PII masking, sensitive data classification, and ensuring appropriate entitlements for each user, acknowledging an expanded risk sphere.
3. Neo4j Knowledge Graph: The processed data is modeled into a Neo4j graph, serving as the cross-system semantic layer.
4. Agentic UX: AI agents consume this graph, facilitating chat-based interactions and constrained workflows.

The “Tacit Knowledge” Moat

Phipps emphasizes that engaging data owners is paramount for modeling this tacit knowledge. This involves understanding the full meaning of fields, data structure, how to join data, limitations, systematics, safeguards, security, and especially reporting conventions. The ability to answer a question in the way the organization has historically reported it is a key differentiator and a core component of their defensible moat.

Complex Hierarchies and Connectedness

The data model itself captures intricate organizational structures:

• Funding Hierarchy (Additive DAG): A five-level hierarchy where all levels contribute to the understanding. It includes in_path shortcuts and links internal funding teams with their portfolios to specific investments. Multiple funding teams can fund a single investment.
• Management Hierarchy: This hierarchy focuses on investment management, where each level holds independent significance. It allows for direct and indirect team management and enables the pre-computation of derived edges (e.g., rollup_manages) for efficient querying.
• People and Org Charts: Integrates human resources data, modeling roles (owners, attendees, directors), reporting structures, and team compositions. This stitches together traditionally siloed HR systems with programmatic data.

Unifying Structured and Unstructured Data

A significant feature is the seamless integration of unstructured documents. Meetings, for instance, are linked to documents. These documents are broken into semantic sections or chunks, which are then indexed for full-text search. Importantly, these document chunks are also mapped back to the structured organizational entities within the graph, enabling agents to reason across both types of data. The resulting knowledge graph is a deeply interconnected model representing four distinct systems as a single, discoverable semantic layer for AI agents.

Serving Agents via MCP and Evaluation

The knowledge graph is served to AI agents, specifically Claude, through a modified MCP (Managed Chat Platform) server, acknowledging that the chat interface itself is not the moat. While Neo4j provides off-the-shelf MCP servers, the Gates Foundation team has forked and customized theirs for schema updates and state management (e.g., conversation IDs). This allows users to interact with the platform within their existing Claude or ChatGPT environments. Beyond general chat, they are actively developing constrained workflow experiences via MCP apps, offering more structured interactions while still leveraging the same underlying knowledge graph.

Retrieval Evaluations and Feedback Loop

To ensure accuracy and alignment with organizational reporting standards, a robust retrieval evaluation (eval) process is in place. Evals are crucial for identifying ambiguities and gaps in the data model.

• They involve data owners in building targeted eval questions.
• These questions generate specific graph queries, which are run against the live graph at runtime.
• An LLM acts as a judge to compare the agent’s output against the expected answers, measuring metrics like “pass at one” and “stability” (consistency of answers to repeated queries).
• This creates a vital feedback loop, allowing for continuous updates to the data model, domain rules, and schema descriptions, thereby improving the agent’s performance and relevance. Phipps notes that while their system achieves high accuracy, remaining issues often stem from inherent ambiguities in the questions themselves rather than outright errors in the model.

Future Directions

Future plans for SIP include expanding the primary graph with more enterprise-wide data sets, exploring federated graph experiences to link team-specific data, and developing more sophisticated agentic interactions.

• Tags:
• Data Curation
• Enterprise AI
• LLM Agents
• Semantic Layer
• Knowledge Graph
• Data Modeling
• Neo4j
• Machine Learning Engineering
• Evaluation Frameworks
• Retrieval Augmented Generation (RAG)
• Tacit Knowledge
• Gates Foundation
• Operational Data