Sid Sheth, CEO of d-matrix, details their memory-centric approach to AI inference hardware, focusing on their Digital In-Memory Compute (DIMC) architecture. He explains how DIMC, an augmented SRAM technology, minimizes data movement to solve the memory bottleneck, delivering significant gains in latency and energy efficiency, particularly for the 'decode' phase of large language models.
In a reflective and forward-looking conversation, Daniel Terhorst-North and Kevlin Henney explore the evolution of software development over the past 20 years and predict the key challenges and innovations for the next 20. They delve into the philosophy of programming language design, the critical need for hardware-sympathetic programming, the untapped potential of concurrency models like CSP and the Actor Model, and the future of user interfaces and decentralized technology.
The current AI inference stack, reliant on general-purpose GPUs, is economically and technically unsustainable for real-time AI at scale. AI hardware expert Robert Wachen argues that the future is specialized hardware, like Transformer-specific ASICs, which can unlock currently bottlenecked applications such as real-time video, code generation, and large-scale enterprise deployments by solving critical latency and cost-per-user challenges.
André Prager, former Chief Engineer at Wing, discusses the core engineering philosophy of simplicity and cost-effectiveness that enabled the drone delivery service. He covers the design of key systems like the passive charging pad, the intelligent winch, the non-powered autoloader, and the iterative process of making the drones acoustically unobtrusive.