Cost reduction

We Cut 94% of AI Coding Tokens With a Local Code Index - Rajkumar Sakthivel, Tesco

We Cut 94% of AI Coding Tokens With a Local Code Index - Rajkumar Sakthivel, Tesco

Rajkumar Sakthivel details how an unexpected surge in AI coding tool costs led to the discovery that sending excessive, irrelevant context was the primary culprit. He introduces the Code Context Engine (CCE), a local retrieval layer that intelligently prunes context using AST-aware chunks, hybrid search, and relevance scoring, resulting in up to 94% token reduction and significant cost savings. The talk emphasizes that optimizing input context, not just the AI model, is paramount for efficient and accurate AI-assisted coding.

LLM Compression Explained: Build Faster, Efficient AI Models

LLM Compression Explained: Build Faster, Efficient AI Models

Learn how AI model compression and quantization techniques are essential for optimizing Large Language Model (LLM) performance and significantly reducing inference costs in production. This deep dive covers practical examples, benefits like reduced latency and increased throughput, and strategies for different AI use cases, demonstrating how to deploy scalable AI with minimal accuracy degradation.

Build Hour: Prompt Caching

Build Hour: Prompt Caching

Explore prompt caching to significantly reduce latency and costs for your AI applications. This guide breaks down the mechanics of KV caching, best practices for maximizing cache hits using `prompt_cache_key` and the Responses API, and real-world implementation insights from the agentic development platform, Warp.