Llms

OpenAI’s Plan to Make ChatGPT the Everything App — Akshay Nathan, OpenAI

OpenAI’s Plan to Make ChatGPT the Everything App — Akshay Nathan, OpenAI

Akshay Nathan, head of Core Product Engineering at OpenAI, discusses the journey and rationale behind ChatGPT Work. He explains how Codex's unexpected adoption by non-developers led to a unified agent harness, blurring the lines between developer and knowledge worker tools. The conversation delves into model capabilities, the role of artifacts and interactive 'Sites' in replacing traditional documents, and how AI fosters a new era of 'T-shaped' generalists. Nathan emphasizes the shift in productivity bottlenecks to 'ideas and taste' and the importance of 'quality at-bats' over mere 'motion' in an AI-powered development landscape.

Llama.cpp vs vLLM: Which Local LLM Engine Actually Scales?

Llama.cpp vs vLLM: Which Local LLM Engine Actually Scales?

Explore the strengths and optimal use cases of `llama.cpp` and `vLLM` for local LLM inference. `llama.cpp` excels on consumer hardware with optimizations like quantization and CPU support, while `vLLM` is designed for production-scale efficiency with features like continuous batching and speculative decoding on high-performance accelerators.

Why Large? Tiny LMs & Agents on Edge/Robotics — Cormac Brick, Google

Why Large? Tiny LMs & Agents on Edge/Robotics — Cormac Brick, Google

Cormac Brick from Google AI Edge discusses how the increasing constraint of DRAM cost on edge devices necessitates the development and deployment of increasingly smaller AI models. He outlines the work of his team in optimizing models like Gemma, achieving 2.9 bits per weight for a 2 billion parameter model capable of running on a Raspberry Pi at 7.6 tokens/second, or on an NPU at 31 tokens/second decode for vision tasks. The talk delves into 'tiny models' (50M-500M parameters) that extend AI to older devices and enable features like robust voice-to-function calling via fine-tuning with synthetic data, exemplified by an offline voice dictation app.

What Big Tech Missed And How Startups Can Still Win

What Big Tech Missed And How Startups Can Still Win

Alexandre LeBrun, CEO of AMI Labs, discusses his career building and selling AI companies, emphasizing his strategy of tackling problems "20 years too early." He delves into AMI Labs' contrarian bet on "world models" over traditional LLMs, highlighting their ability to learn directly from real-world sensory data, unlike LLMs which learn from human-written text. LeBrun explains how this approach is critical for developing intelligent robots and avoiding the pitfalls of Vision-Language Assistants (VLAs). He also touches upon the challenges of securing talent, data, and compute for such an ambitious project, the strategic choice of location, and the importance of holding an extremely large vision while solving a narrow problem for early founders.

Building Closed-Loop Evals for a Multimodal Agent at Scale — Soumya Gupta & Jai Chopra, Uber

Building Closed-Loop Evals for a Multimodal Agent at Scale — Soumya Gupta & Jai Chopra, Uber

This talk details how Uber Eats designed and implemented a multimodal AI agent to enhance food photography for independent merchants, addressing challenges like maintaining authenticity, merchant brand, and marketplace diversity while operating at scale. It covers the intricate evaluation strategies for routing and image editing agents, including continuous learning loops, managing drift, countering reward hacking, and balancing creative freedom with rigid safety guardrails. The speakers explain how they built a closed feedback loop combining offline human labeling, internal dogfooding, and online production signals to ensure robust and adaptive performance.

Coding Agents Are Secretly General Agents

Coding Agents Are Secretly General Agents

Jay Hack, head of AI at ClickUp, discusses the evolution of AI from early computer vision to generalist coding agents, highlighting how 'positive transfer' makes coding an 'AGI-complete' domain. He delves into the brutal economics of AI startups facing foundation model giants, ClickUp's strategy for convergence and first-party data as a moat, and the challenges of verifiability and catastrophic forgetting. The conversation also explores LLMs at the scientific frontier, the 'car wash test' revealing limits of world models, and speculative future applications like LLM resorts and game integration.