Why Agentic Systems Need Ontologies — Frank Coyle, UC Berkeley

Frank Coyle discusses the inherent flaws of purely probabilistic LLMs in agentic systems and proposes a neuro-symbolic solution. He advocates for wrapping LLM tool-use loops with external ontologies and validation mechanisms (like RDFS, OWL, and Pydantic) to provide logical guardrails, preventing common, critical errors that probabilistic reasoning alone cannot reliably avoid.

July 23, 2026 4 min read

Why Agentic Systems Need Ontologies — Frank Coyle, UC Berkeley — source: https://www.youtube.com/watch?v=Sir59K8ZDPU

Frank Coyle, an educator with extensive computer science experience, proposes a neuro-symbolic approach to developing robust AI agents. He argues that while large language models (LLMs) and agentic systems leverage probabilistic reasoning to achieve remarkable conversational and decision-making capabilities, their inherent probabilistic nature leads to critical failures when deployed in complex, real-world systems. These failures manifest as illogical actions like issuing a “second refund on the same order,” sending a “payout to the support desk instead of the buyer,” or assigning vague statuses like “probably shipped.” Such errors stem from LLMs reasoning over domains they only partially understand, a gap that cannot be fully closed by prompt engineering alone.

Coyle traces the lineage of AI agents back to early AI pioneers like John McCarthy and Marvin Minsky, defining them as entities that perceive, decide, and act. He highlights that modern agentic AI, powered by LLMs, often employs “loops”—sequences, conditionals, and iterations—which make them Turing complete and capable of performing any computable task. However, these loops are prone to breaking, drifting, or incurring excessive computational costs, further underscoring the need for external control.

The proposed solution integrates symbolic AI, specifically ontologies, as logical guardrails around probabilistic LLMs. Ontologies, a concept tracing back to Aristotle and formalized by figures like Gruber, are defined as “a formal specification of a shared conceptualization.” They represent entities, their relationships, and properties in a structured manner, akin to graph databases which offer greater flexibility than traditional relational databases. Ontologies can be developed top-down (expert-driven) or bottom-up (data-driven), and Coyle emphasizes reusing existing taxonomies like schema.org, FOAF (Friend of a Friend), and Dublin Core to avoid reinventing the wheel.

To enforce logical consistency, ontologies are augmented with technologies like RDFS (Resource Description Framework Schema) and OWL (Web Ontology Language). These provide mechanisms for inference and constraints:

RDFS (rdfs:domain, rdfs:range): Allows for inferring types. For example, if a teaches relationship has a domain of Teacher and a range of Student, then stating “Bob teaches Scooter” infers that Bob is a Teacher and Scooter is a Student.
OWL (Transitive and Functional Properties):
- Transitive properties (e.g., ancestorOf): If A is an ancestor of B, and B is an ancestor of C, then A is an ancestor of C, automatically augmenting the knowledge graph.
- Functional properties (e.g., hasFather): Enforce uniqueness. If an entity can only have one father, and two different names are associated as a father, the ontology can infer they refer to the same individual, or flag an inconsistency.

Coyle demonstrates this neuro-symbolic approach within an agent’s tool-use loop. When an LLM (e.g., Claude) proposes a tool call, the system intercepts it. Before execution, two levels of validation are applied:

Pydantic at the door: Ensures type correctness of the proposed tool parameters, guarding against basic data mismatches.
Ontology at the ledger: The results of the tool call, or even the parameters themselves, are checked against the domain ontology and its RDFS/OWL constraints. This ensures semantic validity and prevents actions that violate established business rules or logical inconsistencies. For instance, an OWL owl:disjointWith property can ensure Customer and SupportRep are distinct entities, preventing a payout from being sent to the wrong role. An enumerated value constraint can prevent free-form text like “probably shipped” from being accepted, ensuring only predefined statuses (e.g., “paid”, “shipped”, “refunded”) are used.

By implementing these logical guardrails, the system can either correct the LLM’s proposed action, prompt the LLM to reconsider, or flag the issue for human intervention, thereby keeping the probabilistic agent “on track” and “honest,” making its operations reliable and predictable.