An AI Research Agent That Runs Your Experiments — Tim Sweeney, Weights & Biases

Tim Sweeney, principal engineer at Weights & Biases, demos ARIA, W&B's agent for automated ML research. ARIA launches and monitors GPU training jobs on Karpathy's autoresearch project, surfaces patterns across 200+ experiments, and builds reports and dashboards. The talk then covers how the team built ARIA with Weave observability, LLM judges, YAML-based tasks as unit tests, and nightly evals, and closes with practical tips for productionizing agents.