Continual learning

Aug 11, 2026

How Harvey Built a Research Lab on a Budget | Gabe Pereyra

Gabe Pereyra of Harvey details a playbook for application companies to compete with frontier AI labs by leveraging the ecosystem. Key strategies include building specialized benchmarks like Legal Agent Bench, using domain experts for synthetic data generation to overcome sensitive client data issues, partnering with multiple 'neo labs' for post-training, and developing robust model serving and evaluation infrastructure. He emphasizes open-sourcing data for validation and the 'Moneyball' philosophy for success, addressing challenges like talent acquisition and long-context management in the Q&A.

Jul 31, 2026

Reinforcement Learning without Verifiable Rewards — Will Brown, Prime Intellect

Will Brown's talk addresses the critical challenge of applying Reinforcement Learning to real-world tasks where verifiable rewards are absent. He outlines how Primordial AI tackles this by leveraging environments as the core anchor for building reward signals. Key strategies include using LLMs as "judges," grounding tasks in production traces or document corpora, and employing "reverse direction" techniques to generate training data. Brown also details methods for calibrating task difficulty, identifying reward hacking, and fostering continual learning by treating model optimization as a science, emphasizing the use of compute to refine environmental signals and abstract human expertise.

Jul 29, 2026

Building the Automated AGI Lab: Core Automation's Jerry Tworek and Rohan Anil

Jerry Tworek and Rohan Anil, founders of Core Automation, argue that the transformer architecture has reached its limits and the primary bottleneck to smarter AI systems is now architectural. They contend that current models lack continual learning and test-time adaptation capabilities, critical for real-world deployment. They advocate for new architectures that learn from experience more efficiently than current reinforcement learning, optimize pre-training and RL end-to-end, and build a highly automated lab focused on accelerating innovation through kernel generation, aiming for models that can improve themselves without human intervention.

Jul 13, 2026

The AI Memory Problem: Why Long Context Isn’t Enough — Dan Biderman, Engram Co-founder & CEO

Engram co-founder Dan Biderman discusses building AI that learns from users, critiquing long context, RAG, and compaction's limitations. He introduces Engram's approach of compressing knowledge into "cartridges" and model weights via continual learning and gradient-based updates, aiming for "intuition" over retrieval. The vision extends to personal, "Tamagotchi" AI models and addresses the critical need for token efficiency and "doing more with less" in both enterprise and personal AI.

Jun 24, 2026

Memory and Continual Learning: Engram's Dan Biderman and Jessy Lin

Dan Biderman and Jessy Lin of Engram introduce their "always training" paradigm, focusing on baking a team's knowledge directly into a model's weights to achieve true memory and continual learning. This contrarian approach, which they call a "RAG killer" for specific use cases, promises up to 100x token savings and superior performance by internalizing context rather than relying on ever-larger context windows or external retrieval, envisioning a future where everyone has their own continually learning, personalized AI model.

Jun 21, 2026

⚡️Every product of the future will be a living system — Ronak Malde, Trajectory.ai

Ronuk Malde, CEO of Trajectory.ai, discusses his journey from building AI coding agents at Windsurf to his current focus on continual learning for enterprise AI. He shares insights on leveraging real-world user data, the unique challenges of model acquisition, and how Trajectory.ai's platform, powered by innovations like scaled SDPO and a novel training stack, enables dynamic, always-learning AI models for diverse industries from legal to finance.