A tour of MLX, the on-device AI framework for Apple Silicon. This talk explores real-world applications from real-time vision and multimodal omni models to sub-100ms speech synthesis and video generation, all running locally. It highlights breakthrough techniques like Turbo Quant for 1M context and showcases community projects in robotics and native apps, arguing for a future where powerful AI runs without the cloud.
A deep dive into Google's AI Edge stack for on-device AI, covering the new Gemma 4 models, the LiteRT framework for cross-platform deployment, and practical use cases in agent skills, tool calling, and hardware acceleration on CPUs, GPUs, and NPUs.
Cormac Brick from Google's AI Edge team details the dual trends of on-device AI: large, system-level models like Gemma 4 enabling complex agent skills, and fine-tuned tiny LLMs for high-performance, in-app tasks. The summary covers the architecture of on-device function calling, the engineering trade-offs for edge deployment, and the practical workflow for fine-tuning and deploying models under 1B parameters on platforms like Android and iOS.
Maxime Labonne from Liquid AI shares a playbook for post-training frontier small models (under 1GB) for on-device deployment. The talk breaks down the LFM2.5 recipe, which includes on-policy preference alignment and agentic reinforcement learning, and addresses unique challenges at the 1B scale, such as capability interference and 'doom loops', offering concrete solutions to build efficient models for tasks like data extraction and tool use.
Cassidy Hardin from Google DeepMind introduces Gemma 4, a new family of open-weight models with significant architectural and performance improvements. This summary covers the four new models (31B Dense, 26B MoE, and two "Effective" on-device models), deep dives into architectural changes like mixed global/local attention and Per-Layer Embeddings (PLE), and details the new native multimodal capabilities for vision and audio.
Adria Grondin, developer of the Locally AI app, provides a technical walkthrough on running large language models like Google's Gemma on an iPhone using Apple's MLX framework. The talk covers the necessary tools, performance expectations, the importance of quantization, and the growing MLX ecosystem.