Aug 26, 2026

The Death of Developer Advocates — Stephanie Jarmak, Sourcegraph

Stephanie Jarmak argues that Developer Relations (DevRel) isn't dead, but its audience has shifted to include AI agents. She introduces the "Agent Advocate" role, focusing on measuring agent experience (e.g., through CodeScaleBench), optimizing for "Generative Engine Optimization" (GEO), and adapting traditional DevRel principles like enablement and community for this new user. The core message is that designing for agents, like a curb cut, ultimately improves the experience for all human developers.

Aug 20, 2026

Going In Deep On Data | YC Paper Club

Three experts discuss the evolving landscape of data in AI, covering its critical role in model performance, challenges in sourcing and evaluating expert data, the development of advanced data generation techniques for diffusion models, and the complexities of multilingual pre-training.

Aug 13, 2026

When to Build Your Own Agent Harness | Harrison Chase, LangChain

Harrison Chase, co-founder of LangChain, delves into the critical role of the 'harness' in LLM agents, explaining how it orchestrates models and context. He covers customizing harnesses with middleware and sub-agents, the trade-offs between off-the-shelf and custom solutions for in- and out-of-distribution tasks, and the importance of evaluations and observability for continuous agent improvement. The discussion culminates in the 'data flywheel' concept and the `LangSmith Engine` for automating agent intelligence through iterative refinement.

Aug 11, 2026

How Harvey Built a Research Lab on a Budget | Gabe Pereyra

Gabe Pereyra of Harvey details a playbook for application companies to compete with frontier AI labs by leveraging the ecosystem. Key strategies include building specialized benchmarks like Legal Agent Bench, using domain experts for synthetic data generation to overcome sensitive client data issues, partnering with multiple 'neo labs' for post-training, and developing robust model serving and evaluation infrastructure. He emphasizes open-sourcing data for validation and the 'Moneyball' philosophy for success, addressing challenges like talent acquisition and long-context management in the Q&A.

Aug 02, 2026

When Will The Benchmaxxing Plague End? — Nick Heiner, Surge AI

Nick Heiner explores the phenomenon of "benchmaxing" in AI, where models are optimized for benchmark scores rather than real-world utility. He exposes common antipatterns in benchmark creation, such as contamination, reward hacking, and misaligned verifiers, and critiques labs' tactics like gaming leaderboards. Heiner advocates for a higher standard, emphasizing the need for human expertise, high-fidelity data, and rigorous alignment in evaluation to ensure benchmarks genuinely reflect AI's value.

Jul 26, 2026

DeepSWE: A Contamination-Resistant Coding Benchmark — James Shi, Datacurve

DeepSWE is a novel benchmark for evaluating AI models on software engineering tasks. It features 113 original, long-horizon problems from real open-source repositories, designed to resist contamination and provide clearer differentiation between models. The benchmark highlights unique failure modes and strengths of models like Claude and GPT, emphasizes realistic prompt design, and uses program-based verifiers focusing on observable behavior.