Your Coding Agent Should Do AI System Engineering — Ben Burtenshaw, Hugging Face
Ben Burtenshaw from Hugging Face demonstrates how coding agents are tackling complex AI systems engineering tasks. He outlines a three-tiered approach: interactively writing CUDA kernels, autonomously fine-tuning LLMs, and deploying a multi-agent research lab (AutoLab) to parallelize experiments, all powered by file-based "skills" and open primitives on the Hugging Face Hub.
Coding agents have crossed an acceptance threshold and are now capable of tackling some of the hardest problems in the field: AI systems and machine learning engineering. The key to unlocking this capability lies in combining agents with standardized, open repositories and “skills”—file-based context that transforms complex zero-shot problems into manageable few-shot workflows. This approach allows engineers to work “closer to the silicon” and achieve significant performance gains.
This process can be broken down into three progressively autonomous levels.
The Efficiency Bottleneck: Why Custom Kernels Matter
Before diving into agentic workflows, it’s crucial to understand performance bottlenecks in deep learning. Efficiency is generally split into three areas:
- Compute: The raw FLOPs for mathematical operations.
- Memory: The time spent moving tensors between different levels of memory.
- Overhead: Everything else, including the Python environment and kernel dispatch.
Contrary to common belief, the primary bottleneck for modern GPUs is not compute but memory bandwidth. An H100 GPU can perform a petaflop of computation per second, but its memory bandwidth is only around 3 terabytes per second. This means the GPU often sits idle, waiting for data.
Custom CUDA kernels, like the well-known Flash Attention, address this by increasing arithmetic intensity. They fuse operations to perform more calculations for each memory read/write, effectively “keeping the GPUs warm” and maximizing utilization. However, writing and distributing these kernels has historically been difficult due to hardware/software fragmentation and installation complexity.
Level 1: Hybrid Agent-Assisted CUDA Kernel Development
Writing custom kernels, once considered an unattainable goal for agents, is now feasible. The Hugging Face kernels library provides a standardized framework for distributing and versioning kernels on the Hub, much like models. Each kernel repo includes a toml file specifying its compatibility with different hardware, CUDA versions, and software, solving a major distribution challenge.
Agents are empowered to write these kernels through skills. In this context, skills are simply file-based context—version-controlled files containing examples, benchmarking scripts, and references. This gives the agent a “few-shot” starting point, guiding it on how to write, test, and benchmark a kernel for specific hardware. Using this interactive, skill-based approach, a 94% speedup was achieved for a Qwen 3 8B kernel on an H100, demonstrating the potential for capturing low-hanging optimization fruit for specific hardware configurations.
To validate and improve these skills, the open-source library upskill allows for evaluating different models on the same skill, enabling users to compare cost and accuracy and iterate towards more efficient solutions.
Level 2: Zero-Shot Autonomous Fine-Tuning
The next level of autonomy involves an agent performing an entire fine-tuning pipeline from a single prompt, such as “fine-tune Qwen 3 6B on this dataset.” This workflow is fully integrated with the Hugging Face Hub, using HF CLI skills to manage the process and even run the necessary GPUs on Hub infrastructure. This demonstrates a complete, end-to-end task accomplished with minimal human intervention.
Level 3: AutoLab - A Multi-Agent Automated Research Lab
The most advanced level is a multi-agent research system called “AutoLab,” inspired by Andrej Karpathy’s Auto Research project. Instead of a single agent iterating on a problem, AutoLab distributes the work across a team of specialized agents to run parallel experiments:
- Researcher: Acts as a literature scout, using tools like the
hf papersCLI to find relevant papers and formulate hypotheses for model improvements. - Planner: Takes hypotheses from the Researcher and maintains a queue of jobs.
- Workers: Pick up jobs from the queue, implement the changes in training scripts (e.g., altering architecture or hyperparameters), and run them as
HF Jobson the Hub. - Reporter: Monitors all running jobs and pushes metrics and results to a live dashboard.
This entire system is orchestrated using open tools. Experiments are managed in a Git repository, compute is handled by HF Jobs, and monitoring is done via Trackio. Trackio is particularly well-suited for agentic workflows because it uses an open data layer (Parquet files). This allows agents—or developers—to directly access the underlying data for custom analysis and visualizations, such as creating a Gantt chart of agent activity over time.
Key Takeaways for Agentic Engineering
Two core principles emerge from these examples:
- Agents Thrive on Open Primitives: Agents work best with well-exposed, open, low-level tools rather than heavily abstracted APIs that can create a “ceiling” of capability. Tools like the
kernelslibrary andTrackio’s open data layer provide the control and transparency agents need to perform complex tasks. - The Hub is Ready for Agentic Workloads: The Hugging Face Hub provides the foundational pillars—storage, tracking, and compute—necessary to support and scale these advanced, autonomous engineering workflows.