A Worm With 302 Neurons Inspired Their Architecture — Ramin Hasani, Liquid AI

The conversation delves into the origins, evolution, and future vision of Liquid, a company at the forefront of developing highly efficient and adaptive AI. Raine, CEO of Liquid, outlines their journey from academic research to a foundational AI lab.

Origins and Liquid Neural Networks (LNNs)

Liquid’s research began 12 years ago at TU Wien (formerly Vienna University of Technology), aiming to integrate inspirations from biological brains into machine learning, particularly continuous-time mathematics. This led to the development of Liquid Neural Networks.

A key inspiration was the C. elegans worm, chosen for its simplicity (302 nerve cells) and unique neuronal behavior. Unlike spiking neurons in the human brain (which are complex and poorly understood), C. elegans neurons exhibit “electrotonic” or graded, non-spiking behavior, similar to artificial neurons. This characteristic allowed for the creation of continuous-time, differentiable systems amenable to backpropagation. Despite its small size, this worm could control 95 muscle cells more effectively than 2016-era robotic systems, highlighting the efficiency of its neural architecture. Humans share a 78% genome similarity with the worm, and its study has yielded four Nobel Prizes, suggesting its fundamental relevance.

LNNs, also known as “liquid time constant neural networks,” are described as highly nonlinear systems governed by differential equations, yet fully differentiable. Their “liquid” nature refers to their flexibility. They proved excellent representation learners, capable of controlling robots with just tens to hundreds of neurons, which was a significant advantage for compute-constrained environments.

Scaling and the STAR Architecture Search

The research moved to MIT in 2017, collaborating with robotics pioneer Daniela Rus. The focus shifted to scaling these continuous-time dynamical systems and applying them to domains like cars and drones, and predictive sequence modeling. This period saw a broader academic interest in continuous-time systems (e.g., neural ODEs, state-space models), which offer greater adaptability in the time dimension compared to discretized recurrent neural networks.

However, scaling highly nonlinear recurrent systems presented significant computational challenges, primarily due to sequential computation, which is difficult to parallelize on GPUs. While Transformers excel at parallelization (matrix multiplication), their quadratic attention cost grows exponentially with data. Recurrent networks are theoretically more efficient (sub-quadratic or linear cost) but historically hard to implement at scale.

To address this, early approaches included linearization (e.g., State Space Models - SSMs), which allows for parallel algebraic operations. However, this often sacrifices expressivity, leading to models that are efficient but struggle with tasks like long-context language understanding.

Liquid’s breakthrough was the development of a meta-AI system called STAR (Synthetic Tailored Architecture). This system takes an “Avengers” approach, combining elements from various alternative architectures (e.g., Hyena, SSMs, and LNNs). STAR is an unbiased, recursive, and self-improving search algorithm that optimizes hybrid architectures based on four key criteria:

1. No sacrifice on quality against pure transformer models.
2. Minimize memory consumption.
3. Minimize latency.
4. Maximize speed of computation.

This framework led to the Liquid Foundation Model (LFM) family. LFM1 was a complex blend of architectures. LFM2, however, optimized for CPUs, features an elegant and simplified architecture comprising 80% double-gated 1D convolutions and 20% group query attention. This architecture has been scaled from tens of millions to 70 billion parameters, with current models being LFM 2.5 instances and LFM3 in development, constantly incorporating new architectural innovations.

Business Model and On-Device/Edge AI

Liquid’s core business revolves around deploying intelligence outside traditional data centers. They partner with car manufacturers (e.g., Mercedes-Benz), robot manufacturers, and tech companies (e.g., AMD for AI PCs, mobile device makers) to bring AI directly to the edge.

The motivations for on-device AI are compelling:

• Cost Reduction: Moves AI inference from expensive cloud infrastructure to local devices, increasing margins for companies.
• Privacy and Security: Enables local processing of sensitive data, acting as privacy filters or guardrails.
• Offline and Air-gapped Capabilities: Provides intelligence without constant internet connectivity.
• Proactive Agents: Allows AI to analyze local device data and offer insights directly.

Examples of real-world deployment include:

• Mercedes-Benz: An LFM-powered 600MB multimodal model provides in-car intelligence, handling voice commands, connecting to cameras, and offering text-based reasoning. This will be deployed via OTA updates in all North American Gen3 Mercedes cars this year, running on a low-cost chip (~$100).
• Shopify: While not on-device, LFMs are extensively used in Shopify’s private deployments, serving latency and quality-sensitive applications like the Shop app’s search functionality, handling over a billion requests per month.
• Community Adoption: Liquid’s open-sourced models (100M to 24B parameters, including Mixture of Experts for larger versions) see over 40 million downloads in total and 1.5 million weekly, indicating strong developer adoption for various production uses. These models also support multimodal input (audio, vision, text) and audio/text output.

Robotics, an initial inspiration, was deliberately paused due to the complexities of verification, validation, and regulatory hurdles. However, with the maturity of Liquid’s models, the company is now re-engaging with this vertical, using vision-language models for industrial robots (e.g., RoboEtech, Gbionics) to enable vision-action capabilities and instruction following.

The Vision for Sovereign AI and Customization

Liquid offers its technology through design partnerships, where an applied ML team works with clients to build solutions, followed by yearly licensing fees based on deployment. As solutions mature within a vertical, subsequent clients benefit from reduced service needs.

A major upcoming initiative is a self-serve platform for model development, aimed at enabling enterprises to build, customize, and deploy LFMs at production quality. This addresses the common challenge where 80% of enterprise ML projects fail to reach production and developers often generate “useless tokens” without proper guidance.

This platform champions the concept of “customization tokens” – a new wave of monetization beyond inference tokens, covering the entire spectrum from pre-training to inference, including data generation and reinforcement learning. Liquid aims to provide the tools for enterprises to own their intelligence, control costs, and ensure continuous improvement. The Leap platform (a library with cookbooks for fine-tuning and deploying models in formats like GGUF/Llama CPP) is an early iteration of this vision, designed to be human-friendly and integrate with existing open-source tools like Hugging Face.

Research Insights and Future Directions

Liquid’s research has yielded several key insights:

• Architecture Bias: For large, general-purpose models, less architectural bias and simpler designs (e.g., 1D convolutions, unstructured attention) are more effective for scaling. Conversely, smaller models benefit from more complexity and bias (e.g., multiple feedback loops) for increased expressivity in memory-constrained environments.
• Modality Specificity: Continuous-time systems like SSMs and RNNs are highly effective for audio (time-series data) but less so for text. Word modeling, as an algorithmic approach, can extract different physical world understandings than next-token prediction.
• Hardware and Substrate: Current GPU infrastructure is optimized for parallel matrix multiplication, favoring architectures like Transformers. Future breakthroughs, potentially in photonics, could unlock efficient serial/nonlinear computation, fundamentally changing architectural possibilities. The idea that “intelligence is a property of carbon” suggests that the underlying substrate profoundly influences intelligence.

Looking ahead, Liquid’s research agenda focuses on:

• Massively Multimodal Systems: Training all data modalities simultaneously for longer-horizon reasoning and reliable on-device intelligence.
• Efficiency Across the Stack: Optimizing everything from kernel design and inference to pre-training infrastructure.
• Non-Human Understandable Data: Exploring applications in signals and biology, such as DNA foundation models (long-context, high sequence length despite small vocabulary).
• Adaptive Intelligence: Developing systems that combine forward and backward passes, enabling continuous evolution and in-context learning beyond current static evaluations.
• Cognitive Architectures: Building systems from composable, smaller units of compute (e.g., LFMs as agents) to create sophisticated “systems of systems.”

While “attention is definitely one element that you need,” Liquid believes the future of AI also heavily relies on hardware awareness, deep customization, embodiment (physical or virtual), and adaptive intelligence through continual learning, pushing towards a future where intelligence is owned and tailored by its users.

• Tags:
• Feature
• Multimodal AI
• Model Customization
• Edge AI
• On-Device AI
• Sovereign AI
• Liquid Neural Networks
• Continuous-Time Systems
• C. Elegans Worm
• Neuroarchitecture Search
• STAR Algorithm
• Liquid Foundation Models
• Adaptive Intelligence
• Robotics AI