The Real AI Frontier Isn't Smarter Machines (with Catherine Williams)
Dr. Catherine Williams, a former black-hole physicist and early data science leader, explores the field's evolution from Bayesian models to LLMs. She passionately argues that deep mathematical understanding and the ability to build robust mental models are more crucial than ever, even as AI automates technical tasks. Williams also discusses the impact of embeddings, the changing economics of frontier AI, and her work at the nonprofit Candid, advocating for a human-centric approach to intelligence in the age of AI.
Dr. Catherine Williams, Chief Data Officer at Candid, traces the evolution of data science from its nascent days to the current era of large language models, emphasizing the enduring importance of deep mathematical understanding. Her journey began with a PhD in mathematics, researching general relativity and black holes with pen and paper, which instilled intellectual habits like rigor and the ability to build complex mental models. She highlights that while initially a “bottom-up” thinker, the business world demanded a shift to “top-down” reasoning, making decisions without complete granular understanding.
Joining AppNexus in 2012, around the time the “data scientist” title emerged, Williams witnessed the field’s rapid transformation. Early data science relied on mathematical fluency for Bayesian predictions on aggregated data. However, with the advent of Hadoop and log-level data, the focus shifted to traditional machine learning techniques like logistic regression and random forests. This period, the “Wild West” of data science, required individuals (often PhDs) who could tackle ill-defined problems and forge new paths.
Williams discusses the “BERT moment” and the embeddings revolution, noting how these advancements overhauled previous syntactic and keyword-based text analysis methods. She acknowledges the “magical” feeling of capabilities emerging from large-scale transformers, even while admitting a lack of full intuition into their underlying mechanisms, a common sentiment for many post-BERT. The past era saw data scientists fine-tuning open-source models to achieve state-of-the-art results, a practice now largely superseded by zero-shot capabilities of powerful proprietary LLMs.
Addressing the economics of AI, Williams argues that while venture capitalists currently subsidize LLM usage, the long-term trend of frontier AI capabilities becoming exponentially cheaper (e.g., 1% of current cost in two years due to distillation and engineering tricks) will mitigate concerns about high operational costs. This continuous cost pressure, coupled with open-source alternatives, will drive further innovation and accessibility.
In her current role as Chief Data Officer at Candid, a nonprofit providing data to other nonprofits, Williams faces unique challenges related to external data quality (e.g., from IRS forms) rather than instrumented data. Candid’s mission-aligned work, though sometimes abstract, focuses on making the social sector more efficient and equitable. She highlights their cutting-edge partnership with Anthropic, leveraging the Model Context Protocol to deliver real-time, verified data directly into AI tools like Claude.
For aspiring leaders, Williams advises thinking “one level up” – understanding the constraints and problems faced by those in the next leadership tier – as a key to career progression. Regarding the future of data science in an LLM-enabled world, she stresses the importance of investing in one’s own “mental models.” While machines handle technical tasks, humans must retain conceptual understanding and fluency to build the next level of complexity, rather than simply outsourcing all knowledge. She recommends Max Tegmark’s “A Brief History of Intelligence” for insights into intelligence and AI’s future.