SAP: Bringing the ‘Operating System’ of a Company into the AI Era with CTO Philipp Herzig

The Enduring Enterprise: From Standard Software to AI Operating System

SAP’s durability through multiple technological shifts—from mainframes to AI—is rooted in its founding principle: creating standard software that delivers business outcomes. The company began by recognizing the inefficiency of re-implementing finance systems for each customer, leading to the birth of standard enterprise software. This focus on outcomes over technology has allowed SAP to evolve, serving as the core “operating system” for over 400,000 enterprise customers managing everything from finance and supply chain to HR and procurement.

While technology has changed, the fundamental customer need for efficient, end-to-end business management has not. AI is the latest powerful tool to help achieve this, and SAP is focused on closing the “AI adoption gap” by embedding it deeply into its entire portfolio to help customers get more done.

The Three Pillars of AI Transformation in the Enterprise

The current AI-driven shift is as fundamental as the move from on-premises to the cloud, re-engineering the entire system on three primary levels:

1. User Interface (UI): The paradigm of designing UIs to teach a human how to perform tasks is over. The future is generative UI, where interfaces are dynamically generated, multimodal, and proactive. The system, not the user, provides the intelligence. For example, it can proactively analyze the impact of a geopolitical event like disruptions in the Strait of Hormuz on a customer’s supply chain and present options for replanning.

2. Business Processes: AI agents are transforming rigid, standard operating procedures into dynamic, intelligent workflows. By blending the structured world of enterprise data with the unstructured world of text and documents, agents enable a move from “Software as a Service” to “Outcome as a Service.” They can execute complex tasks across the business, getting more work done with less human intervention.

3. Data Layer: The core belief is that “AI is only as powerful as the data is.” SAP possesses incredibly valuable business data—general ledgers, invoices, inventory information—but the key is to combine this with a plethora of other external and internal data sources. The goal is to build a harmonized, semantical view of data that can effectively fuel powerful and accurate AI models.

Overcoming the Enterprise AI Adoption Gap

The biggest engineering challenge in enterprise AI is not building a model, but making it work reliably and effectively at scale.

• The Problem of Scale: A proof-of-concept, like a RAG chatbot built on ten documents, is simple. Scaling that to handle thousands of documents, 20,000 APIs, and the specific context of individual users (e.g., providing different travel policy answers to a US vs. a German employee) is a massive engineering hurdle. Disambiguating overloaded business terms like “order” (is it a sales, maintenance, or purchase order?) requires a sophisticated orchestration layer.
• Data Fragmentation: Most enterprises suffer from disaggregated data landscapes due to M&A activity and past purchasing decisions. This data fragmentation is a primary obstacle that limits the potential of AI.
• Security and Reliability: Enterprises cannot simply deploy the latest open-source innovation from GitHub. Innovations must be hardened and secured to meet enterprise-grade requirements for security, data privacy, and reliability.

Verifiability, Evals, and the Rise of “Agent Mining”

A critical component for building reliable AI agents is verifiability. Agentic coding works well because the outcome is verifiable—the code either compiles and passes unit tests, or it doesn’t. To achieve this in business processes, a new mindset focused on creating high-quality evaluation sets (“evals”) is required. This marks the return of “test-driven development” principles, where developers must first define the boundary conditions and expected outcomes before the agent writes the code.

To create these evals and continuously improve agent performance, SAP is pioneering a concept called “agent mining.” When an agent is unsure, it queries the user for input. This interaction—the decision traces and context provided by the user—is captured. This process surfaces “tribal knowledge” that previously only existed in people’s heads or scattered across Slack and Teams channels. This captured data then feeds a flywheel:

1. The knowledge is used to identify anomalies or discover process improvements.
2. Good improvements can be elevated to new standard operating procedures.
3. The captured traces become new data points for creating more robust evals, further optimizing the system.

While both computer use (UI automation) and tool calling (API use) are viable for agents, the majority of enterprise automation will likely rely on the more structured and reliable approach of tool calling.

Beyond Language: The Limits of LLMs and the Need for Relational Transformers

Large Language Models (LLMs) are phenomenal in the unstructured world of text and images, but they are not designed for predictive analytics on structured, tabular data—the bedrock of finance, supply chain, and sales. For tasks like demand forecasting or cash flow prediction, businesses rely on classical machine learning models (e.g., XGBoost), but this approach doesn’t scale. It requires deep data science expertise and leads to a massive proliferation of models; a single pharmaceutical company might need 180 separate models just to predict payment delays across 90 countries.

To solve this, SAP has invested two years of research into RPT1 (Relational Pre-trained Transformers). Published at NeurIPS, RPT1 applies the transformer architecture that powers LLMs to the structured world of tabular data. The goal is to create a foundational model for relational data that can make highly accurate predictions (classification, regression, time-series) from a small amount of context, democratizing predictive analytics in the same way LLMs democratized natural language processing.

The Future of Work and Business Models in the AI Era

AI will fundamentally “uplevel” every role in the enterprise. By automating mundane work like collecting information and preparing reports, it frees professionals to focus on higher-value, strategic thinking. A finance professional, for example, will transition from manual data entry to supervising an AI agent, reviewing its work, and thinking about the next strategic problem to solve.

This shift also necessitates a change in business models. SAP is moving from its traditional seat-based licensing to a hybrid consumptive model. While the future is consumption-based, customers currently require a hybrid approach that provides the predictability and cost control needed to manage their transition to AI-driven operations. The companies that will win in the AI era are those that focus relentlessly on customer adoption and business outcomes, making the underlying technology disappear.

• Tags:
• Agentic AI
• LLM
• Feature
• Enterprise AI
• Predictive Models
• AI Adoption
• Tabular Data
• SAP
• Business Process Automation
• RPT1