Diffusion transformers

Runway’s Bet Beyond Video: World Models, Robotics, and the Neural OS — Anastasis Germanidis

Runway’s Bet Beyond Video: World Models, Robotics, and the Neural OS — Anastasis Germanidis

Anastasis Germanidis, co-founder of Runway, discusses the company's journey from early generative creative tools to frontier video models, emphasizing their pivot towards general-purpose world simulation. He details the evolution of Gen-1, Gen-2, and Gen-3 (scaling 10x after Sora's release), highlighting the importance of controllability and the inevitability of real-time generation. The conversation delves into cutting-edge applications like Interface World Models (a fully neural operating system) and World Models for Robotics, where third-person video data and robust simulation are key to training intelligent agents.

Infra behind Krea 2: How to train and serve at scale — Gabriel Jorge Menezes, Krea.ai

Infra behind Krea 2: How to train and serve at scale — Gabriel Jorge Menezes, Krea.ai

Krea's Gabriel Jorge Menezes shares critical insights into building the infrastructure for large-scale ML model training and serving. Key takeaways include the necessity of custom metrics beyond standard GPU utilization, aggressive checkpointing on ultra-fast storage to counter frequent cluster crashes, and a dynamic Kubernetes-based system using gang scheduling, virtual-kubelet, and taints to seamlessly shift inference workloads to external providers when training consumes on-prem GPUs. The approach highlights practical solutions for silent failures, thermal management, and optimizing resource utilization in a unified production and training environment.

OpenAI Sora 2 Team: How Generative Video Will Unlock Creativity and World Models

OpenAI Sora 2 Team: How Generative Video Will Unlock Creativity and World Models

The OpenAI Sora team discusses the technology behind their video generation model, from the diffusion transformers and space-time tokens that enable object permanence to their vision for Sora as a world simulator capable of scientific discovery. They also cover their product philosophy, which prioritizes creative inspiration over mindless consumption, and their strategy for building a new creator economy with IP holders.

OpenAI Sora 2 Team: How Generative Video Will Unlock Creativity and World Models

OpenAI Sora 2 Team: How Generative Video Will Unlock Creativity and World Models

The OpenAI Sora team discusses the technology behind their video generation model, from the Diffusion Transformers and space-time tokens that enable object permanence to their vision for Sora as a world simulator for scientific discovery. They also cover the intentional product design of the Sora app, which optimizes for creative inspiration over mindless consumption, and their strategy for building a new creator economy with IP holders.