You Can't Tell What's Real Anymore. Here's What Comes Next - Cristóbal Valenzuela, Runway
Runway co-founder and CEO Cristóbal Valenzuela traces the company's progress from a 128-by-128-pixel AI film in 2019 to today's photorealistic video, and argues that next-frame prediction at scale teaches models physics and object permanence. He then covers world interface models that make video interactive, applications in robotics, and Continuum, a research prototype where any on-screen content becomes a live, explorable video.
From Pixels to Simulation: Next-Frame Prediction as a World Model
Runway’s first AI film in 2019, The Orange, was rendered at 128 by 128 pixels and upsampled to a larger resolution. Watching it today feels like viewing an early silent film: charming, historically meaningful, and clearly a starting point. The distance between that work and current state-of-the-art video, where generated characters and locations are indistinguishable from real footage, is the core argument of the talk. The speaker’s view is that the same trajectory continues, and that the gap between generated and real video will keep closing in just a few years.
Video as Next-Frame Prediction
The framing begins with an old observation from cinema: video is an optical illusion. Stitch frames together fast enough and motion appears. Runway’s bet is that next-frame prediction, scaled far enough, is more than a way to make pleasing clips. It is a route to learning how the world works. The team believes scaling laws that hold for language models also hold for video, and they are ramping up compute behind their models accordingly.
The interesting part is what emerges. A model trained only to predict the next frame has to internalize physics, object permanence, gravity, occlusion, and how objects relate to one another. Nobody writes these rules down. The model infers them from pixels. The speaker notes that people can no longer reliably tell generated video from real footage in their measurements, and suggests the Turing-style test for video has effectively been passed. Even so, benchmarks for video models are increasingly tied to physical plausibility and geometry, not only to whether a clip looks appealing.
From Video to Simulation
Once next-frame prediction runs in an environment that resembles a game, the output starts to behave like a physics engine or simulator, except that no one encoded the physics. The model learns behavior “by pure force of scale.” The speaker describes this as a simulation of a simulation, a world that exists in no software and has no predefined rules.
The key shift is from batch generation to interactivity. Earlier results took seconds or minutes to render. If the model runs in real time and accepts input at any frame, each frame becomes a branching point. A user can alter the scene, and the model produces the consequences. This turns video generation into a controllable world model.
Robotics and World Interface Models
The most immediate applications the speaker describes are robotics and interfaces. Runway has a robotics lab in its New York office, and these models are being explored as the reasoning and “brain” layer for embodied, autonomous systems that must move through and understand physical space.
The more unusual direction treats any pixel on screen as something a video model can drive. A single car image can become an interactive scene where the user turns the car, opens the hood, or switches the lights, all without a 3D engine, Unity, Unreal, or Blender. The model has learned how actions in an interface change the visual result. The speaker calls this a world interface model: an interface and a world model at once. Similarly, objects in a still photo can be moved around, and the model predicts how they should move within the space, even when that space was never modeled in 3D.
Continuum: Extrapolating Forward
The talk’s most ambitious section is Continuum, an early research prototype. The speaker argues that extrapolation (asking what happens in the next six to eighteen months as models scale, gain data, and become more accurate simulators) may be one of the most important skills in AI and in company building. Continuum imagines a universal simulation system that can predict anything on a screen, and it imagines an operating system where what appears on your display may be generated by a video model.
The demonstrations sketch what that could look like:
- Portals. Any part of a web page can become a live, interactive video grounded in the surrounding content. Research on San Francisco fog, batteries, and photosynthesis appear as explorable visualizations rather than static text. Resizing a window changes the experience, and the portals persist as a kind of memory of prior learning.
- Visual thinking for coding agents. Continuum has been connected to a coding agent. Instead of reasoning about a 3D bike model purely in code, the agent can spin up a visual imagination of the problem, which the user can see, manipulate, and use to guide the agent in ways that text alone cannot. The speaker frames this as a new paradigm, drawing an analogy to visual thinkers who reason through ideas visually before putting them into words.
- Live, reactive experiences. Searching for Mars produced an interface generated in real time, a Mars outpost with navigable spaces and robots, that was designed for the query on the fly.
The speaker admits that the category is unclear. These experiences feel like something between a visualization, a video, and a game, and the industry lacks a name for them. That open question is part of the work ahead.
Everything Will Be Simulated
The closing argument is that much of what appears novel today will soon seem obvious, and Continuum may one day look as primitive as the orange does now. Continuum and the simulation work are still early, but the speaker says the systems now work and scale. Runway plans to release most of what was shown over the next eight to twelve months.