Arek Borucki details how Hugging Face scales its infrastructure to serve millions of models and users, focusing on the evolution of search architecture using MongoDB Atlas and Apache Lucene, robust database scaling with a seven-node cluster and sharding, and dynamic frontend autoscaling with Kubernetes and KEDA to ensure an instant, seamless user experience.
Charles Humble explores the significant environmental impact of AI, particularly generative AI, on global carbon emissions. He offers practical, actionable strategies across the entire AI lifecycle—from project planning and data collection to training and deployment—to reduce this footprint. Key recommendations include questioning the necessity of AI solutions, choosing open-source models, leveraging carbon-aware computing for demand shifting, employing model compression techniques like distillation and quantization, and integrating sustainability as a fundamental architectural principle in software engineering.
Dr. Cathy O'Neil, author of "Weapons of Math Destruction," asserts that terrifying algorithms are defined by secrecy, unaccountability, and a lack of opt-out, not mathematical complexity. She details how Taylorism's labor degradation now extends to white-collar jobs via AI surveillance. O'Neil discusses her firms, ORCAA and OCEAN, which provide statistical evidence for lawsuits against tech giants and advocate for algorithmic accountability through "cockpits" of metrics and transparent auditing, urging collective action against unchecked technological power.
Explore the strengths and optimal use cases of `llama.cpp` and `vLLM` for local LLM inference. `llama.cpp` excels on consumer hardware with optimizations like quantization and CPU support, while `vLLM` is designed for production-scale efficiency with features like continuous batching and speculative decoding on high-performance accelerators.
Rajat Shah details Netflix's approach to automating performance engineering using AI agents. He describes how an agent can read production profiling data, identify quadratic inefficiencies, propose code fixes, and validate them via canary deployments. The talk highlights the importance of a shared anti-pattern catalog and shifting from reactive bug fixing to proactive prevention by integrating AI early in the development cycle, emphasizing foundational automation and structured autonomy levels.
Sam Altman, co-founder and CEO of OpenAI, reflects on the evolution of startups from YC's first batch to the current AI-driven era. He discusses the unprecedented opportunities for ambitious founders, the critical role of startups in distributing AI's power, the importance of conviction against conventional wisdom, and the rapid advancements in AI models, while also addressing safety concerns and envisioning an optimistic future for human agency.