Semantic Blindness: 500,000 Sensors Confused an LLM - Raahul Singh & Vanč Levstik, Phaidra

This talk dissects the fundamental architectural mismatch between probabilistic language models (LLMs) and deterministic engineering systems, particularly in mission-critical physical infrastructure with vast sensor networks and strict operational constraints. The core problem highlighted is that “you cannot solve a combinatorial engineering problem with a next token prediction engine,” a lesson learned “the hard way” by deploying state-of-the-art LLMs for real-time operations in industrial and AI factory environments.

The Problem: Semantic Blindness

The authors introduce a specific failure mode called Semantic Blindness, defined as the inability of general-purpose LLMs to maintain structural awareness of physical systems, even with extensive context. This issue becomes apparent when LLMs are tasked with tasks like root cause analysis, alarm triage, and operational decision support in environments like 1-gigawatt scale AI factories with hundreds of thousands of sensors and GPUs.

Three Critical Failure Modes

The presentation details three specific failure modes that expose gaps in current industry thinking about scaling LLMs to complex systems:

1. The Topology Trap: Vector embeddings, often used in RAG architectures, inherently lack an understanding of physical causality, relationships, or topology. For an LLM, “Sensor_445_Temp” is merely a string, devoid of the critical context that it’s attached to “Valve B,” which in turn controls coolant to “Generator 3.” This leads to a lack of structural awareness.
2. The Illusion of Scale: While dumping a small number of sensors (e.g., 100) into an LLM’s context window might work for demos, this approach collapses at large scales (e.g., 500,000 sensors). At scale, attention degrades, critical anomalies get buried due to the “lost in the middle” phenomenon, and latency spikes to unusable levels for real-time industrial responses. LLM context windows are finite and quickly saturated.
3. The Repetition Kill Switch: Industrial tag naming conventions often involve highly similar patterns across numerous variants (e.g., “Chiller 6,” “Chiller 7”). Feeding these similar names extensively to an LLM triggers its internal repetition penalty, causing the model to believe it’s stuck in a degenerate loop. Consequently, it may abruptly stop generating output, even if the data itself is correct and relevant.

A Hybrid Architectural Solution

Instead of relying on prompt engineering alone, the speakers propose a practical hybrid design approach built on four key insights, aiming to make AI reliable in real-world engineering environments:

1. Leveraging Hierarchical Structure (The Linearizer): AI factories exhibit a hierarchical structure (data centers > data halls > aisles > racks > GPUs). The key insight is to grow the system’s understanding with tree depth (which grows slowly) rather than width (which grows exponentially with equipment count). A “linearizer” provides the LLM with a summarized, concise representation of the system’s graph by describing paths from root to leaf, rather than listing every node. This consolidated context, which remains relatively constant in size regardless of equipment count (e.g., a 64 GPU system versus a 460,000 GPU system), allows the LLM to understand the plant’s arrangement.
2. LLMs for Planning, Not Searching: Recognizing LLMs’ poor recall with sharded solutions, the approach dictates that LLMs should plan how to search rather than directly perform the search. The LLM’s role is to produce structured outputs (e.g., “collect GPUs,” “scope: Data Hall 11,” “filter: running hot”), instructing a deterministic backend on how to identify the target equipment, rather than sifting through fuzzy names itself.
3. Deterministic Resolver for Backend Operations: Once the LLM provides a structured search plan, a deterministic backend takes over. This involves building pre-indexed trees or subsets of equipment based on location and interaction. Set operations are then used to perform highly accurate and reliable retrieval, ensuring perfect recall and accuracy regardless of query fuzziness. This backend handles the exact logic, counting, and deduplication that LLMs struggle with.
4. Pattern-Based Search: For vague queries, instead of passing entire lists of names to the LLM, the system leverages the LLM to identify patterns in naming conventions or data. The LLM creates a pattern, which the backend then executes. This ensures the LLM never has to process a large number of tokens, maintaining a relatively constant and low operational cost.

This end-to-end process involves a Planner LLM discerning user intent and generating a search pattern, followed by a Deterministic Resolver performing indexing and set operations to create the final result set. This two- or three-step process is more efficient and reliable than multi-step agentic loops.

Validation and The Inversion of Software 1.0/3.0

Extensive testing against old approaches (same LLM, same data) showed dramatic improvements:

• Correctness: The old approach degraded from 80% at 64 GPUs to 30% at 460,000 GPUs. The new hybrid approach maintained 100% accuracy across all scales, including 66 cases on six real production systems with zero failures.
• Efficiency: The old approach consumed 116 million tokens for a single validation pass at 1 GW scale, while the new approach used only 390,000 tokens (nearly 300x fewer). Crucially, the cost per query remained flat (around 9,000 tokens) regardless of system size, eliminating the exponential cost growth associated with scaling.

Relating to Karpathy’s Software 1.0 (deterministic code) vs. Software 3.0 (LLM-driven behavior) paradigm, the authors present an inversion for AI-native systems. While legacy software sees 3.0 eating 1.0, AI-native development starts with 3.0 (for rapid demoing) but matures towards 1.0 for production reliability. The LLM excels at ambiguous requests, planning, and synthesizing human-readable answers. However, any part of the system that can be data-modeled or involves structured data, exact retrieval, or 100% reproducibility (like hierarchies, graphs, schemas, counting, deduplication) should be moved to deterministic 1.0 code. This provides a “reliable ground for the LLM to stand on,” ensuring robust and scalable performance.

The key takeaway is to “let the LLM keep the hard decisions” (ambiguity, planning), but for everything it shouldn’t be guessing on, implement it as deterministic code and provide that structure back to the model. Develop with Software 3.0, but productionize by strategically integrating Software 1.0. This is how to build AI systems that respect physical reality and scale reliably against massive datasets.

• Tags:
• LLM Architecture
• Scalable AI
• Hybrid AI
• Industrial AI
• Semantic Blindness
• Context Window Limitations
• RAG Limitations
• Deterministic Systems
• Combinatorial Problems
• Data Center Management
• Software 1.0 vs 3.0
• Reliable AI