Tokenless

Llm routing

AI Agents Don't Read Your Policy Docs. They Hit Your APIs — Gravitee

AI Agents Don't Read Your Policy Docs. They Hit Your APIs — Gravitee

Sam, CTO at Gravity, argues that AI agent governance must be enforced in infrastructure rather than in prompts or policy documents. Through five live demos against a hotel-booking agent, he shows a gateway blocking destructive tool calls, preventing data exfiltration, rate-limiting runaway agents, serving cached semantic responses, and rejecting oversized prompts before any LLM call. He also covers observability, vendor decoupling via LLM routing, trimming bloated MCP tool lists, and managing shadow AI.

What Is Jev? The AI Model That Doesn't Generate Text

What Is Jev? The AI Model That Doesn't Generate Text

Jev is a System 1 AI model from TypeSafe that skips text generation entirely, returning calibrated probabilities for fixed-option questions in a single fast request. This explores how it works, how it is trained with reinforcement learning for calibrated decisions, how its probabilities drive software thresholds and guardrails, and how it can complement LLMs in workflows.

The $10 Trillion Token Economy — Alex Atallah, OpenRouter & Anjney Midha, AMP

The $10 Trillion Token Economy — Alex Atallah, OpenRouter & Anjney Midha, AMP

Alex Atallah and Anjney Midha trace OpenRouter's path from early open-weight models and Discord experiments to a neutral routing layer serving over 10 million developers, and explain why its Stripe acquisition is as much about fraud defense as payments.

Preferences Over Benchmarks: Model Routing — Archana Kamath & Tyler Gillam, DigitalOcean

Preferences Over Benchmarks: Model Routing — Archana Kamath & Tyler Gillam, DigitalOcean

This talk argues against the common practice of picking LLMs based solely on leaderboards, emphasizing that there's no single best model, only the right one for a given request. It introduces Digital Ocean's Inference Router, a customizable, open-source solution that intelligently selects models based on user-defined preferences (cost, latency, task, quality) rather than benchmarks, demonstrating significant cost savings and performance improvements in live demos.