Beyond Sonic Pi: Tau5 & the Art of Coding with AI • Sam Aaron • GOTO 2025
Sam Aaron, creator of Sonic Pi, discusses the journey from teaching children to code with music to building the next generation of live coding environments. He details the limitations of Sonic Pi (security, deployment) that led to Tau5, a new system built on Elixir and the BEAM. Tau5 is designed to be web-based, secure via a sandboxed Lua environment, and collaborative. A key focus is the integration of AI as a creative partner, using sophisticated tooling to allow AI agents to safely improvise and interact with the system.
The Philosophy of Code as Expression
Sam Aaron challenges the conventional view of programming as a tool exclusively for business. He draws a parallel to the written word, which evolved from specialized use (legal, religious texts) to a universal medium for expression (poems, letters, shopping lists). Aaron’s mission is to foster a similar evolution for code, transforming it into a tool for human creativity and connection, making it accessible and engaging for everyone, not just professional developers.
Sonic Pi: Live Coding for Everyone
Sonic Pi is the primary vehicle for this mission. It’s a live-coding music environment designed with a “low floor, high ceiling” philosophy, making it simple enough for a 10-year-old to start but powerful enough for professional live performances.
- Core Concepts: The system is built on simple commands like
playandsleep. With just these two, users can construct any melody. The design is intuitive: higher numbers mean higher pitches, and longer sleep times mean longer delays. - Concurrency and Engagement: A pivotal moment in Sonic Pi’s development was the introduction of
live_loop. This was not driven by a curriculum but by a student’s desire to play a drum and bass line simultaneously. The initial single-threaded model couldn’t handle this, threatening to disengage the user. The solution was to create a new kind of loop with built-in threading, sneakily introducing a core computer science concept (concurrency) to solve a creative problem. - “In Time” vs. “At the Same Time”: Sonic Pi’s loops go beyond simple concurrency. They are synchronized, allowing different musical parts to play “in time,” not just “at the same time.” This is achieved through a sophisticated scheduler and technology like Ableton Link, enabling cohesive, multi-part musical compositions.
Limitations and the Path Forward
After over a decade of development, Sonic Pi faced fundamental limitations that hindered its growth:
- Technical Debt: Aaron had essentially re-implemented a concurrency and fault-tolerance model (similar to Erlang’s OTP) on top of Ruby, which became a maintenance burden.
- Security: Because Sonic Pi runs raw Ruby code, creating a shared online platform like Scratch was impossible due to the risk of malicious code (
rm -rf /). - Deployment: Installing software in schools is a significant social and political barrier, limiting accessibility.
These challenges pointed towards a new architecture. Influenced by the work of Joe Armstrong (Erlang) and José Valim (Elixir), Aaron recognized that the BEAM virtual machine offered the perfect foundation. An experimental project called “Bleep,” developed with the band The Black Dog using Elixir and Phoenix LiveView, served as a successful proof-of-concept for a web-based live coding environment.
Tau5: The Next Generation of Collaborative Coding
Tau5 is Aaron’s next major project, building on the lessons from Sonic Pi and Bleep. It’s designed to be a secure, web-based, and collaborative live coding system.
- Architecture: Built on Elixir, Phoenix LiveView, and the BEAM, Tau5 runs on the server, streaming events to the client. This web-native approach eliminates installation barriers.
- Security through Sandboxing: To solve the security problem, Tau5 runs user code in a sandboxed Lua environment. This sandbox is resource-limited, preventing denial-of-service attacks (both CPU and memory) and enabling a safe, multi-user online platform.
- AI as a Creative Partner: Tau5 experiments with AI as an improvisational partner, not a replacement for human creativity. The sandboxed environment is crucial for safely interacting with AI agents.
- Advanced Developer Tooling: Aaron has built sophisticated developer tools for Tau5, including integrated REPLs and a custom MCP (Meta-programming and Code Protocol) server called
Spectra. This allows an AI agent to programmatically inspect and modify the entire system—from the Elixir backend to the frontend visuals (powered by Hydra)—within the safe confines of the sandbox.