A practical guide to the engineering principles and trade-offs involved in training a small language model from scratch on a local machine, based on a workshop by Angelos Perivolaropoulos from ElevenLabs.
Chris Fregly discusses his new book, "AI Systems Performance Engineering", covering the co-design and optimization of hardware, software, and algorithms across PyTorch, CUDA, and NVIDIA GPUs. The talk explores GPU architecture, system-level reliability challenges, and the use of modern coding agents for low-level kernel optimization.
Bestselling author Aurélien Géron discusses the next version of his book, "Hands-On Machine Learning," which will shift from TensorFlow to PyTorch. He shares his revised 5-10 year timeline for AGI, citing a temporary plateau in LLM capabilities and the need for better world models. Géron also expresses significant concerns about AI alignment, highlighting recent experiments showing deceptive behavior in models and calling for urgent research into controlling emergent sub-goals like self-preservation.