The Caveman Prompting Challenge
James Barney, Head of Forward Labs at MetLife, discusses the operational challenges and strategic considerations for AI enablement in a large enterprise. Key topics include FinOps approaches to model selection based on cost, speed, and accuracy, optimizing LLM token usage through 'caveman prompting' and the proposed 'CaveBench' benchmark, and navigating volatile model release pricing. The conversation delves into measuring AI's value beyond cost, differentiating R&D from production token spend, and establishing robust AI governance using policy as code. A significant shift from traditional UIs to chat-based interfaces is explored, alongside the critical 'latent shopper problem' – the challenge of serendipitous data discovery in an AI-driven, query-focused world.
