Memory bandwidth

Modern Computer Architecture & Organization • Jim Ledin & Maciej «MJ» Jedrzejewski • GOTO 2026

Modern Computer Architecture & Organization • Jim Ledin & Maciej «MJ» Jedrzejewski • GOTO 2026

Jim Ledin, author of *Modern Computer Architecture and Organization*, discusses the integration of GPUs and LLMs into his book, emphasizing architectural principles over fleeting trends. He highlights memory bandwidth as the primary bottleneck in AI, explaining the impact of High Bandwidth Memory (HBM) demand on consumer RAM. Ledin also identifies understanding pipelining and cache hierarchy as the most crucial skill for software developers to optimize code performance.

Why AI Engineers Need to Understand GPU Hardware (with Chris Fregly)

Why AI Engineers Need to Understand GPU Hardware (with Chris Fregly)

Chris Fregly, author of 'AI Systems Performance Engineering', explains that true performance gains in AI come not from raw compute but from a deep, holistic understanding of the entire hardware and software stack. He emphasizes that memory bandwidth is the most critical GPU metric and introduces the concept of 'mechanical sympathy'—the co-design of hardware, software, and algorithms—as the key to unlocking efficiency and overcoming modern bottlenecks.