Tokenless

Auto research

Your LLM Judge Is a Confident Liar: Building Better Verifiers — Browserbase

Your LLM Judge Is a Confident Liar: Building Better Verifiers — Browserbase

Browserbase and Microsoft Research unveil their Universal Verifier for computer-use and web agents, addressing the critical flaw of inaccurate LLM-as-judge benchmarks that yield "confident liars." They detail its four guiding principles, demonstrating how it drastically cuts false positives and provides high agreement with human evaluations, leading to better training data. An auto-research experiment highlights the synergy between AI and human expertise in developing robust evaluation systems.

Long-Horizon Agents Need Experiments, Not Just Prompts — Erina Karati

Long-Horizon Agents Need Experiments, Not Just Prompts — Erina Karati

Erina Karati and Arunachalam Manikandan describe Project Paradox, a Supercell framework for game agents with per-agent memory, emotions and trust scores, and how they wrapped it in an autoresearch loop to fix long-horizon failures such as lost rumor sources and hardened uncertainty. The talk covers controlled scenario suites, a balanced scorecard, a small frozen policy surface, rollback discipline, and lessons applicable to support, research, coding and personal assistant agents.

An AI Research Agent That Runs Your Experiments — Tim Sweeney, Weights & Biases

An AI Research Agent That Runs Your Experiments — Tim Sweeney, Weights & Biases

Tim Sweeney, principal engineer at Weights & Biases, demos ARIA, W&B's agent for automated ML research. ARIA launches and monitors GPU training jobs on Karpathy's autoresearch project, surfaces patterns across 200+ experiments, and builds reports and dashboards. The talk then covers how the team built ARIA with Weave observability, LLM judges, YAML-based tasks as unit tests, and nightly evals, and closes with practical tips for productionizing agents.

The Loop Is the Product — Roland Gavrilescu, Introspection

The Loop Is the Product — Roland Gavrilescu, Introspection

Roland Gavrilescu, co-founder and CEO of Introspection and formerly at xAI, presents a blueprint for autoresearch in 2026 built on three ideas: the loop is the product, system distillation is the moat, and valued work per watt is the score to optimize. He illustrates these with the OpenClaw car-haggling loop, the concept of portable agent recipes, and a talent-sourcing agent example that shows how to spot patterns in traces, calibrate judges with humans, and A/B test taste in production.

Large clusters for small models — Daniel Svonava, Superlinked

Large clusters for small models — Daniel Svonava, Superlinked

Daniel Svonava argues that small open-source models, when specialized and properly served, can achieve frontier performance for specific tasks, offering significant cost savings and latency improvements over large, generalized models or managed endpoints. He highlights challenges with existing serving infrastructure for a fleet of diverse small models and introduces Superlinked's open-source solution, which leverages a pull-based, centralized queuing system, a Rust sidecar for runtime abstraction, and an autoresearch loop for automated model tuning, enabling high throughput (e.g., half a million embedding tokens/sec on one GPU) and efficient deployment of specialized models and LoRAs.

Andrej Karpathy on Code Agents, AutoResearch, and the Loopy Era of AI

Andrej Karpathy on Code Agents, AutoResearch, and the Loopy Era of AI

Andrej Karpathy discusses the paradigm shift to AI agent-driven development, the future of engineering and research with his AutoResearch project, the impact on jobs, and the evolving landscape of open-source models, robotics, and education.