Tokenless

How KV Cache Speeds Up LLMs for Faster AI Models on GPUs

Jun 30, 2026

Interactive discovery

Explore the topic map

Follow the connections between themes, people, and ideas across the Tokenless archive in an interactive topic modeling map.

Open the map Browse posts

Machine Learning

View All

Jun 29, 2026

Frontier results, on device - RL Nabors, Arize

RL Nabors discusses the significant costs associated with using frontier AI models, covering security, latency, and financial implications. She introduces a framework for right-sizing AI solutions by leveraging smaller, task-specific models and Small Language Models (SLMs). The framework details how to prove task feasibility, establish success criteria with golden datasets, conduct capability evaluations (using tools like Phoenix), and select the most appropriate "Small And Good Enough" (SAGE) model. Nabors further demonstrates how prompt engineering, particularly few-shot prompting, and post-processing can close performance gaps with larger models, while advocating for continuous regression evaluations to maintain performance integrity. The overarching message is to "prototype big, deploy small" to optimize AI deployments.

Jun 28, 2026

Research to Reality: Bringing Frontier ML Research to Production - Vaidas Razgaitis, Higharc

Vaidas Razgaitis, Senior Research Engineer at Higharc, shares three tactical tips to accelerate the transition of novel AI/ML research into production-ready features. He emphasizes addressing the critical handoff challenge between ML researchers and software engineers through structured documentation (Research Prototype Taxonomy Document), a well-organized monorepo utilizing decoupled microservices, and a systematic approach to code decomposition and PR review. These strategies aim to improve legibility, maintainability, and delivery speed for ML-driven products.

Jun 12, 2026

Uncertainty-Guided Data Augmentation for Engineers | Deep Dive - Yongmin Kwon

This session details a data-efficient method for training engineering surrogate models by using uncertainty quantification (UQ) to guide geometric data augmentation. Instead of random deformations, the approach lets the deep ensemble model identify its own knowledge gaps (epistemic uncertainty), then uses Free-Form Deformation (FFD) to generate new shapes specifically in those uncertain regions. This ensures every expensive simulation run yields maximally informative data, significantly improving model accuracy for a fixed computational budget across domains like structural mechanics and aerodynamics.

Artificial Intelligence

View All

Jun 30, 2026

How KV Cache Speeds Up LLMs for Faster AI Models on GPUs

LLMs often slow down under heavy traffic due to inefficient GPU memory management during inference. This overview explains how KV cache and Paged Attention, implemented in VLLM, optimize memory usage across prefill and decode phases, significantly boosting LLM throughput, reducing latency, and improving GPU utilization through advanced context handling and specific tuning techniques like prefix caching and speculative decoding.

Jun 29, 2026

The AI Agents Helping Home Services Book More Jobs

Avoca (YC W23) has achieved eight-figure revenue and a $1 billion valuation by building an AI workforce for home services, turning missed calls into revenue. Founders Apurva Shrivastava and Tyson Chen explain how AI expands software's market share beyond 1% by automating labor and operational costs, leading to a 15x larger opportunity. They emphasize that their AI agents augment human workers, reducing attrition in challenging CSR roles and creating new positions for training AI, driven by a deep customer obsession learned at YC.

Jun 29, 2026

What Is AI Code Review? Fixing Slow PRs & Broken Workflows with AI

Anna Gutowska explains how AI code review enhances software development by addressing the slowness and inconsistency of traditional methods. The video delves into the benefits of AI in accelerating reviews, improving code quality, fostering developer learning, and reducing technical debt. It covers the underlying technologies like static/dynamic analysis and LLMs, discusses critical considerations such as over-reliance and context, and provides best practices for integrating AI while emphasizing the indispensable role of human oversight.

Technology

View All

Jun 29, 2026

Platforms: Build Abstractions, not Illusions • Gregor Hohpe • GOTO 2025

Gregor Hohpe explains the critical role of platforms in managing the growing cognitive load on developers due to complex distributed systems. He contrasts platforms, driven by "economies of speed" and fostering innovation through diversity, with traditional IT services and oversimplified abstractions that create dangerous illusions. Hohpe emphasizes building platforms that provide intuitive, domain-specific abstractions to solve real business problems, rather than just repackaging existing cloud services.

Jun 25, 2026

Full Stack Greenfield Projects : Are they still relevant?

Bharat Goenka, co-founder of Tally, discusses the company's unconventional approach to software development through "Full Stack Greenfield" projects. He explains why building every component from scratch, despite being a high-risk strategy, has been crucial for Tally's success in serving the SMB market, fostering extreme customer loyalty, and aspiring to connect 200 million businesses. The talk delves into the historical context, the philosophy of questioning and choosing constraints, and the distinction between product and custom engineering.

Jun 21, 2026

3‑2‑1 Backup Rule Explained: Protect Your Data from Disaster

Jeff Crume outlines essential data resiliency strategies, starting with the 3-2-1 backup rule—three copies, two media types, one offsite—and expanding to include immutable or air-gapped backups, rigorous testing, and encryption. He emphasizes these principles for robust disaster recovery, ransomware protection, and minimizing costly downtime, highlighting the trade-offs in achieving high availability.

Recent Post

Apr 23, 2026

SAP: Bringing the ‘Operating System’ of a Company into the AI Era with CTO Philipp Herzig

SAP CTO Philipp Herzig discusses the company's AI-driven transformation, focusing on three core pillars: generative UI, AI-native business processes, and a unified data layer. He explores the primary challenges to enterprise AI adoption—scale, data fragmentation, and security—while emphasizing the critical role of verifiability and "agent mining" in creating reliable, compounding value. Herzig also details the limitations of LLMs for predictive analytics on tabular data and introduces SAP's alternative, Relational Pre-trained Transformers (RPT1).

Apr 23, 2026

Why Most Robot Demos Are Fake

Changan Chen, co-founder of Rhoda AI, discusses their vision-first approach to building foundation models for robotics. The conversation covers their unique training pipeline, the distinction between policy and world models, and the path to deploying data-efficient, reliable robots in real-world industrial settings.

Apr 22, 2026

CI/CD Breaks at AI Speed: Tangle, Graphite Stacks, Pro-Model PR Review — Mikhail Parakhin, Shopify

Shopify CTO Mikhail Parakhin details their aggressive internal AI adoption, revealing the internal stack—Tangle, Tangent, and SimGym—that powers reproducible ML, auto-research, and customer simulation. He also discusses how AI shifts the software bottleneck to PR review and CI/CD and explains their use of Liquid AI for low-latency, non-transformer workloads.

Apr 22, 2026

Marc Andreessen on how the internet changed news, politics, and outrage | The a16z Show

Marc Andreessen discusses the launch of "Monitoring the Situation" (MTS), a new media network on X. He explores the concept of the “current thing,” how the internet has reinvented media consumption through viral outrage cycles, and why today's internet-native media landscape is reshaping politics, culture, and our collective attention, drawing parallels from the history of CNN to the theories of Marshall McLuhan.

Apr 22, 2026

Agents need more than a chat - Jacob Lauritzen, CTO Legora

Jacob Lauritzen, CTO of Legora, argues that as AI agents tackle more complex work, the bottleneck shifts from task execution to planning and review. He proposes a framework for human-agent collaboration based on increasing 'trust' and 'control', and advocates for moving beyond simple chat interfaces to high-bandwidth, domain-specific artifacts like documents and structured reviews for more effective collaboration.

Apr 22, 2026

How Stripe Built Their New Website

Stripe's Head of Design, Katie Dill, provides a behind-the-scenes look at their recent homepage redesign. She discusses the strategic shift from a product-list to a narrative-driven approach, the intricate design process behind key elements, and how AI is fundamentally changing their design workflow and empowering their team to fight the "gravitational pull to mediocrity".

← Previous Next →

Stay In The Loop! Subscribe to Our Newsletter.

Get updates straight to your inbox. No spam, just useful content.