Why 95% of All AI Projects Fail - Boris Scharinger | Podcast #174
Boris Scharinger, Industrial AI Strategist at Siemens and author of "Industrial AI: From Pilot to Profit," discusses why 95% of AI proofs of concept fail in industrial settings. He highlights the 5-10x cost multiplier from pilot to production, the critical role of the "harness" around AI models, and the unique challenges of data access and legacy systems in physical environments. Scharinger advocates for a two-tiered investment strategy, separating foundational data capabilities from use-case-specific ROI, and addresses cultural resistances from engineers. He provides guidance on differentiating commodity from strategic AI applications and how to effectively scale industrial AI for profit.
Boris Scharinger, an Industrial AI Strategist at Siemens and author of “Industrial AI: From Pilot to Profit,” delves into the prevalent issue of high failure rates—specifically, 95%—for AI Proof of Concepts (PoCs) in industrial settings. He attributes this to a fundamental mismatch between the high expectations set by consumer AI, which scales easily in purely digital environments, and the complex realities of industrial AI operating in physical spaces.
The Challenge of Expectations and Prioritization
Scharinger emphasizes the need to reset expectations, particularly for top management and venture capitalists, who are often influenced by the rapid scalability of AI in digital businesses like ChatGPT or social media. He advocates for a two-pronged approach to AI initiatives:
- Learning Curve / Commodity Use Cases: These aim to upskill the workforce (e.g., using LLMs for HR or marketing content generation) with marginal productivity gains. They should not be subjected to stringent Return on Investment (ROI) calculations, as their primary goal is enablement.
- Strategic / Differentiating Use Cases: These are designed to fundamentally alter a company’s competitive position. While strategic use cases might improve existing operations (e.g., AI to enhance sales configurators for complex products), truly differentiating use cases (e.g., AI in design space exploration, generative design for material discovery as exemplified by Tesla’s Gigapress alloy development) offer unique competitive advantages. For these, a rigorous ROI calculation is imperative.
He suggests that companies establish separate cost centers to fund foundational learning initiatives differently from high-stakes, differentiating projects requiring significant investment.
Why 95% of Pilots Fail: The “Harness” and Cost Multiplier
The primary reason for PoC failure, according to Scharinger, is a narrow focus on model performance in controlled environments, neglecting the extensive effort required for real-world deployment. He highlights the concept of the “harness”—the surrounding infrastructure and system integration required for an AI model to function effectively in an industrial context. Referencing Anthropic’s observation that nearly 90% of code isn’t the model itself but its harness, he illustrates challenges such as:
- Hardware Incompatibilities: Model behavior can differ based on regional hardware choices (e.g., Nvidia vs. Huawei chips), impacting floating-point calculations and model outputs, a factor rarely covered in PoCs.
- Systemic Integration: Industrial AI models are part of larger, complex systems that demand meticulous design and integration beyond mere model accuracy.
This leads to a 5 to 10 times cost multiplier from a successful PoC to a deployed solution. A $200,000 PoC, for instance, can easily demand over $1,000,000 for full deployment. Such escalated costs often render initial business cases unviable, leading to project cancellations and what Scharinger terms “pilot purgatory.” Many PoCs, he argues, would never start if a realistic cost assessment were performed upfront.
Unique Challenges of Industrial AI Deployment
Industrial AI faces distinct hurdles:
- Data Access and Legacy Systems: Siemens, despite its extensive automation technology sales, often lacks access to customer operational data (e.g., from PLCs) due to ownership and regulatory constraints. Internally, Siemens leverages its own manufacturing data (e.g., electronics production, PCB designs) to develop AI solutions. Legacy systems, such as PLCs with short cycle times, may miss critical transient error states if not specifically programmed for data buffering. The “never touch a running system” mentality further complicates data acquisition from existing infrastructure.
- Physical Asset Depreciation: AI initiatives tied to physical assets (machines) are constrained by long depreciation cycles (7+ years). Convincing CFOs to upgrade or replace equipment solely for AI-driven data points is challenging, causing project delays.
- Fragmented Supply Chain: The industrial supply chain (component providers, machine builders, system integrators) is highly fragmented. Implementing holistic AI solutions requires complex alignment, change requests, and negotiations across multiple parties, significantly slower than in more vertically integrated digital companies.
Convincing the CFO and Addressing Cultural Resistance
To secure investment for foundational data capabilities (e.g., data platforms, OPC UA compatibility), Scharinger proposes a two-tiered investment strategy: separating these strategic, capability-building investments from use-case-specific ROI calculations. Leveraging strategy consultants to highlight competitive risks from rivals’ AI adoption can be effective in gaining CFO buy-in.
Cultural resistance is also a significant barrier:
- Competence Penalty: Engineers using AI may be perceived as less competent or lazier, especially if AI-generated output is generic or perceived as “slop.”
- Engineers Hate Fragility: AI’s probabilistic nature introduces uncertainty, which conflicts with engineers’ preference for robust, predictable systems.
Scharinger suggests rebranding AI as a valuable tool for tasks beyond human capacity, such as exploring vast design spaces, thereby making it an enabler rather than a threat.
Practical Advice and Future Outlook
For companies embarking on AI journeys, Scharinger recommends:
- Buy for Commodity: Utilize existing commercial off-the-shelf solutions for common enterprise functions (e.g., AI-based time sheeting).
- Make/Partner for Strategic: Dedicate internal talent or forge partnerships for differentiating capabilities like product design optimization or outcome-based services.
Looking ahead, Scharinger is optimistic that agentic AI, when seamlessly integrated into engineering workflows (e.g., for meshing in CFD), will significantly reduce the environmental impact of next-generation products by 2030. To combat “pseudo-productivity”—where engineers blindly prompt AI—he advocates for embedding AI tools directly into established engineering pipelines, making their use implicit and guided, thereby ensuring intelligent application.