An evaluation platform is more than a simple test runner; it's a complex system for creating shared definitions of quality. This talk explores the evolution of eval platforms from basic spreadsheets to sophisticated, integrated systems, highlighting the hidden data and systems engineering challenges involved in making them credible, scalable, and usable for building trustworthy AI agents.
AI agents represent a paradigm shift from conversational AI to autonomous systems that can perform actions. This is achieved through an 'agentic loop' combining Large Language Models (LLMs) with tools, as exemplified by the OpenClaw framework, which enables complex, automated workflows while also raising important security considerations.
This talk explores how to overcome the context window limitations that prevent AI agents from accessing large APIs. It introduces "Codemode," a technique where agents write code against a typed SDK in a secure, sandboxed environment, moving beyond static tool definitions and enabling full API accessibility.
![[FULL WORKSHOP] AI Coding For Real Engineers - Matt Pocock, AI Hero (@mattpocockuk )](https://img.youtube.com/vi/-QFHIoCo-Ko/maxresdefault.jpg)
A workshop on building a complete AI-assisted development workflow, covering how to translate ambiguous requirements into agent-ready plans and run autonomous coding agents to ship production-ready features.
SAP CTO Philipp Herzig discusses the company's AI-driven transformation, focusing on three core pillars: generative UI, AI-native business processes, and a unified data layer. He explores the primary challenges to enterprise AI adoption—scale, data fragmentation, and security—while emphasizing the critical role of verifiability and "agent mining" in creating reliable, compounding value. Herzig also details the limitations of LLMs for predictive analytics on tabular data and introduces SAP's alternative, Relational Pre-trained Transformers (RPT1).
Jacob Lauritzen, CTO of Legora, argues that as AI agents tackle more complex work, the bottleneck shifts from task execution to planning and review. He proposes a framework for human-agent collaboration based on increasing 'trust' and 'control', and advocates for moving beyond simple chat interfaces to high-bandwidth, domain-specific artifacts like documents and structured reviews for more effective collaboration.