Faster Chips That Don't Melt — Anima Anandkumar & Benedikt Jenik, Accelerated Understanding

Accelerated Understanding (AU) is developing a ‘foundation model for the physical world,’ aiming to replicate the universality and scale seen in language models for diverse physical systems. Co-founders Anima Anandkumar and Benedikt Jenik articulate their core bet: that common underlying principles across physics domains (like fluid dynamics, semiconductors, and energy) allow for significant transfer learning.

The Vision: A Foundation Model for Physics

The company’s premise is that just as OpenAI consolidated disparate language tasks into a single powerful model, a similar approach can succeed for physical simulation and understanding. While acknowledging the greater challenges in the physical world—such as the lack of readily available real-world data and the need for scientific discovery to go beyond existing data—AU believes transferability is achievable. They argue that fundamental physics principles, like variations in time, energy conservation, object permanence, and causality, implicitly offer shared features even across systems governed by different partial differential equations (PDEs).

Overcoming Challenges with Physics-Driven AI

Purely data-driven AI is deemed insufficient for scientific discovery. Instead, AU integrates the laws of physics as a critical guiding principle. This allows the model to leverage existing simulators for initial training data and, crucially, to self-improve using physics laws as a dense, objective training signal. This contrasts sharply with language models, where self-improvement often relies on sparse human feedback.

Architectural Innovations and Scaling

AU has systematically reinvented architectural components to handle the unique characteristics of physical data:

• 4D Context: Unlike 1D language tokens, physical data involves 3D space plus time. AU models the full rollout over time (avoiding autoregressive methods that accumulate errors) across these four dimensions. When multiplied, these dimensions quickly lead to massive context lengths. AU has achieved training with trillion-context inputs/outputs and inference with 5 trillion-context lengths, yielding outputs measured in tens of terabytes.
• Neural Operators & Resolution Invariance: Neural operators are fundamental to their approach, enabling resolution invariance. This allows the model to flexibly handle varying context lengths—from high-resolution demands of complex physics to lower-resolution needs for initial design exploration—without fixed-resolution limitations seen in many video/vision models. This flexibility is critical for engineering design and scientific discovery where physical accuracy is paramount.
• Hardware and Infrastructure: Standard sharding techniques (like FSTP) are insufficient as AU’s layers and data samples exceed the capacity of single GPUs or even full nodes. They have re-engineered their sharding strategy and infrastructure to train models at scale, performing hundreds of runs and training up to trillion-parameter models, integrating HPC methods with modern ML techniques.

Data Generation and Self-Improvement

• Simulator-Generated Data: AU primarily uses numerical simulators to generate vast quantities of training data, circumventing the bottleneck of real-world data scarcity. This also helps overcome the

• Tags:
• Foundation Models
• AI for Science
• Physical AI
• Self-Improvement
• Neural Operators
• Physics-Informed ML
• Computational Physics
• Scientific Machine Learning
• Resolution Invariance
• Multi-Physics
• Semiconductor Design
• Energy Modeling