Ciro Greco, CEO of Bauplan, outlines a new paradigm for data engineering that applies software engineering principles like version control, transactionality, and a code-first approach to build a programmable lakehouse designed for AI-driven automation and enhanced developer productivity.
Joe Reis and Matt Housley, authors of "Fundamentals of Data Engineering," reflect on the book's principles three years after its publication. They discuss how the rise of AI has created both powerful tools and dangerous "bear traps" for engineers, the critical role of expertise in a world of AI-generated content, and why foundational knowledge is more important than ever.
This talk explores a data science approach to identifying high-risk subpopulations for multimorbidity in sub-Saharan Africa, contrasting traditional hypothesis-driven analysis with a discovery-driven method that automatically uncovers non-obvious patterns in health data.
Jay Fenton discusses the challenges of "big messy data" in the telecommunications industry, highlighting how legacy systems fail to handle the scale and complexity. He presents a suite of modern tools, including Graphistry, IPython, and D3.js, for large-scale data visualization and analysis, advocating for a "human-computer symbiosis" to empower engineers in managing complex networks.
OpenSAFELY offers a new paradigm for medical research by bringing analysis code to sensitive patient data, rather than exporting the data itself. This interview explores its secure, privacy-preserving platform, its custom DSL `ehrql`, and how this model enforces reproducible, hypothesis-driven science while navigating the complexities of real-world clinical data.