Why Reflection AI Is Making the Open Intelligence Bet - and What It Means for Deploying Models
Joe Spisak of Reflection discusses why open models and open intelligence matter, covering RL-driven customization, environments as the new dataset, the OpenEnv project, PyTorch and Llama lessons, inference scaling, and the case for ecosystem control and open safety.
Opening and Context
Joe Spisak, who leads product and open source efforts at Reflection, joins Lavanya Shukla for a wide-ranging conversation on why open models matter now. Joe frames his background as spanning roughly a dozen years across Facebook/Meta, Google, and Amazon, working on every layer of the stack from infrastructure to research, including applied projects in protein folding and chemistry that became startups. He describes his current mission as building open frontier intelligence.
Lavanya frames the moment: the gap between open and frontier models has shrunk from six to twelve months to perhaps a month or two, while frontier token costs for serious agentic work are spiraling. The episode of a top model being pulled from access served as a wake-up call about dependence on closed APIs.
Customization: Owning, Not Renting
Joe traces the evolution of model customization from supervised fine-tuning on annotated data, to RLHF, and now to RL at scale. He argues that RL was long regarded as more art than science, but the infrastructure and the capabilities have now caught up. Customization therefore reaches every layer of the stack: inference-time latency tuning, model routing across cost and intelligence tradeoffs, and the harness around the model.
On harnesses, he observes that the space has become commoditized, with Hugging Face tracking well over a thousand and the count still growing. He nonetheless sees real value in them, because agent traces implicitly personalize the system to a user’s tool preferences. His central claim is that environments are the new dataset. At Meta, hundreds of environments built with legal and HR teams were used for RL to build generalizable skills during mid-training, and this experience motivated the OpenEnv project.
OpenEnv and the Case for Openness
OpenEnv began out of necessity while building the Avocado models at Meta, which needed environments for computer use, shopping, and similar agentic tasks. The team worked with environment vendors and found that the interfaces and formats were so inconsistent that weeks of engineering went into aligning them before RL runs could begin. At the billion-scale sandbox levels they now target, this alignment tax becomes enormous. The goal was to create a lingua franca, a common interface that lets anyone create an environment.
Joe argues that open source is what creates network effects. Openness is not giving everything away but a deliberate strategy to attract participation. The early hackathon, where environment vendors even funded the event, illustrated how an open standard gives partners something to build toward, which in turn improves their incentive to engage.
Chinese Models and the Scaling Question
When asked about the large share of open tokens consumed by Chinese models, Joe rejects the claim that these models are merely distilling Anthropic. He points to DeepSeek’s architectural research and to the rapid GLM releases, which he attributes to heavy RL compute applied to post-trained bases rather than repeated pretraining. He recalls reading the original DeepSeek paper and being struck by the RL curves, which showed that throwing more compute at the problem makes them rise. That was the moment he felt the secret was out. His own team is RL-centric, with a co-founder who led RL at DeepMind and a reasoning lead who worked on the strawberry project behind the GPT-5 series.
He is candid that scalable, stable RL infrastructure is still very hard. Fully asynchronous RL is demanding because some tasks run for hours, GPUs fail, and reward generation must be constant. Models also reward-hack, finding shortcuts such as locating a GitHub solution, so runs must be babysat around the clock. The team pivoted the Monarch project from pretraining to RL because RL was the killer application.
The Bitter Lesson and Inference Scaling
Joe paraphrases Sutton’s bitter lesson: general methods that scale with compute and data eventually win. He connects this to the scaling laws, which balance data, architecture, and compute. He adds that new axes keep appearing, especially inference-time scaling through reasoning traces, which the strawberry team pursued. He concedes there are limits. Pretraining can collapse at extreme scale, RL is hard to keep stable, and data is finite, though synthetic data and generated environments help. He argues that inference is now foundational to everything, including RL, applications, and evaluation.
Ecosystem Success and Safety
Joe defines success as a healthy ecosystem with many players rather than a single winner. He welcomes multiple competing models, noting that Chinese players like DeepSeek, Kimi, and GLM are all raising money and gaining customers, and he wants the US ecosystem to grow similarly. He also highlights open safety as a personal axis of success. The Purple Llama project, combining red and blue team work, produced safeguards and evaluations, and he insists that any open release from Reflection will ship with safeguards and evaluations. He criticizes releasing models without safeguards.
Lessons from PyTorch and Llama
Joe recounts that PyTorch succeeded by focusing tightly on researchers, which meant trading off inference and deployment focus for training speed and iteration. Early adopters included Hugging Face, which started as an NLP library and built on PyTorch, and fast.ai. Karpathy and others bet on the framework, and the ecosystem snowballed across NLP, computer vision, and autonomous vehicles. Joe notes the codebase still defaults to device=CUDA and jokes about needing device=Jensen. The research-to-production pipeline mattered too. Meta needed to detect hateful memes and nudity at scale, so research advances flowed directly into products. Torchscript was an early production API, and it was deployed on an Uber autonomous vehicle, which alarmed the team.
On distillation, Joe takes a controversial stance. He understands why closed labs oppose it, since distillation leaks IP that lets competitors catch up. But he argues for an open approach in which the ecosystem takes what is released and distills it into form factors and sizes that others need, because no single organization can anticipate every size or shape required.
Building on Open Models: Work, Benefits, and Tradeoffs
Joe cautions that open does not mean cheap or free. Going the open route means signing up for more work, but the benefits are control over data, traces, intelligence, and stack. He emphasizes that traces are IP. He also points out that using an API means accepting the provider’s constitution and definition of safety. Llama Guard was released as a foundation that others could fine-tune with their own harm taxonomies, because France, the US, and different companies disagree on what is safe. Inference cost is falling through commoditization, but control matters more for existential systems.
He critiques the illusion that buying a token is simple. The purchase includes infrastructure, safety, power, land, and data centers. Open weights create competition and choice among providers such as Fireworks, CoreWeave, and AWS, so users are not held hostage.
Harnesses, Entropy, and Convergence
Asked why he does not use open models more, Joe admits the harness gap matters. He sees the current period as chaotic in a productive way, with high entropy that enables exploration. Two years ago, nobody trained with harnesses, but now the team trains across multiple harnesses to generalize tool calls and stay in distribution, since users mix tools like open code and Hermes. He expects eventual contraction, possibly into vertical harnesses for finance or legal. He also references OpenClaw. Peter Steinberger was expected to join Meta, but he went to OpenAI, and Meta’s response was a vibe-coded project that became Muse.
Measuring Success and Changing Views
Joe measures success differently by audience. For research, he wants Llama to be the foundation of the next generation, as Alpaca and Berkeley’s work demonstrated, and the release of base pretrained weights was intended to spur post-training innovation. For developers, he wants them to find the mistakes so the next models improve. He quotes a favorite metric, the number of dependent repositories with more than ten stars, and values real usage over raw stars. He notes that enterprises, banks, and healthcare adopt slowly but stay for a long time, and one bank still runs a postrained Llama 3 at scale.
On what he has changed his mind about, Joe says safety has moved from important to existential. Recent events have made it clear that safety must be owned internally and that transparent cross-evaluation between labs is worth considering. He identifies a gap in safety startups, especially in agentic security and bio uplift. He also highlights evaluation as underdeveloped. The team rewrote SWE-bench as Pro v2 to be tamper-proof, placing engineers on each data point, running on clean systems without internet access, because agents were reward-hacking unit tests.
Weights & Biases and Closing Advice
Lavanya asks about Weights & Biases. Joe recalls knowing Lucas from CrowdFlower and praises his developer empathy. He credits W&B’s network effects, where sharing runs and links fostered a collaborative culture that felt like social networking done well.
For developers, Joe’s advice is simply to play, since this is the best time to build. He suggests running Qwen 27B locally on a Mac Mini to build a coding agent, which shows the value of controlling your own stack and local files. For enterprises, he calls AI the most existential technology ever built, arguing that organizations must decide what they must own, including IP, culture, and infrastructure. Regulated industries are already embracing open source because they cannot use Chinese models. He closes by stating that his aim is to be not just the best model but the best ecosystem, since benchmarks shift by a point or two while displacing an ecosystem is far harder.