In Case You Missed It in July 2026 (Ep. 1016 with Jon Krohn)
This episode of ICYMI delves into the profound implications of AI, featuring insights from Dr. Cathy O'Neil on algorithmic harm beyond complexity, Ben Todd on career resilience in an AI-automated world, Steve Mock on AI's role in empowering healthcare advocacy, and Dr. Catherine Williams on the enduring significance of deep mathematical understanding for data professionals.
The episode explores the evolving landscape of AI, discussing its societal impact, career implications, life-saving applications, and the enduring importance of foundational skills.
Weapons of Math Destruction, a Decade On
Dr. Cathy O’Neil, author of “Weapons of Math Destruction,” argues that the true danger of algorithms stems not from their complexity or technological advancement, but from a combination of secrecy, unaccountability, and the lack of choice users have in engaging with them. She emphasizes that even simple algorithms, such as flowcharts or linear/logistic regressions, can be terrifying and exploitative. A prime example given is recidivism risk algorithms, which are often “simple as pie” (e.g., questionnaires with points added up) yet wield immense power, potentially leading judges to sentence individuals to longer prison terms based on future crime risk, a practice O’Neil finds “unconstitutional.”
While acknowledging the rapid advancements in AI since her book’s release in 2016 (e.g., LLMs, deepfakes, GenAI, surveillance pricing), O’Neil explains her decision not to write a sequel by the sheer pace of change, rendering books quickly outdated. She instead launched the “AI Skeptics Podcast” to address current issues more dynamically.
O’Neil draws a parallel between the historical surveillance and dehumanization faced by blue-collar workers (e.g., truckers working for algorithms, teachers fired based on arbitrary metrics) and the emerging trend where white-collar workers (like Meta engineers whose keystrokes are tracked) are now experiencing similar degradation. While not advocating for it, she sees this as potentially fostering greater solidarity among workers, as more people realize “these systems are working against them.” She maintains skepticism about the total replacement of digital work by AI, despite some tasks being automated.
How to Find Solid Career Ground in the AI Era
Ben Todd, founder of 80,000 Hours, considers a hypothetical future with “fully automated digital workers” capable of independently performing tasks traditionally handled by remote human workers. In such a scenario, he suggests that “solid ground” for human careers would shift to areas requiring:
- Legal ownership: AIs cannot currently own companies or be liable.
- Strong consumer preference: Where consumers strongly prefer human interaction or service.
- Physical presence: Jobs requiring physical manipulation or presence in the real world.
Todd highlights that bottlenecks would then move to the physical world, citing areas like construction, energy, and data center building as growing and complementary to AI. However, he also acknowledges that sophisticated AI systems would likely rapidly automate physical tasks through robotics. While robotics scales slower than software due to manufacturing and testing requirements, advancements in simulation-based training could accelerate their development.
He further explores the potential for rapid scaling of robotics, drawing an analogy to World War II airplane production, where car factories were converted. He estimates that if car factories were repurposed, a “billion a year” robots could be produced on a mass-per-mass basis. Additionally, AI-powered firms could achieve “superhuman abilities” not only through individual agent speed (50x faster) but also by leveraging advantages like:
- Instant learning propagation: AI agents can share state space, allowing immediate propagation of learning across all copies.
- Efficient coordination: AI workers could communicate instantly and seamlessly, eliminating human-like communication overheads.
- Direct supervision: CEOs could effectively “personally supervise every single worker,” dramatically improving oversight and efficiency.
How AI Is Quietly Saving Lives
Steve Mock, from aiavedme.org, shares emerging patterns from stories of AI use in healthcare. He emphasizes that the most impactful use of AI in healthcare is not for direct medical advice, but for empowering individuals to become better-informed advocates for themselves and their loved ones. Users leverage AI to educate themselves on medical conditions, enabling them to ask more thoughtful and targeted questions to their healthcare providers. This transformation makes complex medical advocacy, previously an expensive proposition, more accessible.
He cites a compelling example of a relative initially diagnosed with Parkinson’s disease. After years of medication worsening symptoms, the caregiver used AI to question the observed effects. This led to a re-evaluation by a different physician and a completely different diagnosis, illustrating how AI helped navigate a misdiagnosis. Another story involves a woman whose dog had a chronic illness that baffled vets; AI suggested a rare disease, which was then confirmed by veterinary testing.
An extraordinary case involves Paul Cunningham, who used AI for his dog’s cancer treatment. When conventional treatments failed, Cunningham had tumor biopsies DNA-sequenced. Using AI, he identified a matching experimental treatment, and when unable to purchase it, located mRNA vaccine labs to custom-make the treatment. This bespoke therapy led to significant tumor shrinkage, demonstrating AI’s role in highly personalized, cutting-edge medical solutions.
The Math Still Matters: Deep Skills in the Age of AI
Dr. Catherine Williams, Chief Data Officer at Candid, addresses the crucial question of whether a deep understanding of mathematics remains essential given LLMs’ growing proficiency in math and programming. Her answer is a resounding yes, contextualized within a “hierarchy of different levels of complexity” (e.g., arithmetic, algebra, linear algebra, systems thinking).
Williams explains that while AI might perform lower-level tasks, the “conceptual fluency” derived from understanding these foundations is invaluable. It’s akin to needing to understand arithmetic even with calculators. As AI handles more rudimentary layers, human expertise ascends the hierarchy. The shift from a purely mathematical background to engineering, and now with LLMs, to a “systems view” reflects this upward movement.
Ultimately, the best data professionals will not specialize in a single “slice” but will be those who can fluidly move “up and down that chain” of complexity, building and continuously updating their mental models. This deep, interconnected understanding of how different layers fit together will be the differentiator in the age of AI.