No Priors Ep. 126 | With Cloudfare CEO Matthew Prince
Matthew Prince, CEO of Cloudflare, discusses the internet's architectural and economic shift from a search-driven model to an AI-native one. He outlines the existential threat to content creators as AI consumes content without providing traffic, and proposes a new marketplace where creators are compensated for providing value and filling knowledge gaps, rather than generating clicks.
The internet’s dominant value creation model of the last 30 years, driven by search, is fundamentally breaking. The new interface for the web is shifting from search engines to AI assistants. This creates a critical problem: users consume derivative content generated by AI, rather than visiting the original sources. As a result, the traffic that has historically been the primary proxy for value for content creators is disappearing.
Data shows it is now 10 times harder to get a click from Google for a piece of content than it was a decade ago, due to answer boxes and AI overviews. For AI-native platforms like OpenAI or Anthropic, it’s 750 to 30,000 times harder, respectively. If creators can no longer generate value—whether through direct sales, advertising revenue, or even the ego boost of having an audience—the incentive to create content will evaporate, starving not only the open web but also the very AI models that rely on that content for fuel.
The Flawed “Traffic-as-Value” Model
A core issue with the modern web can be traced back to Google’s influence, which trained creators to worship the “deity which is traffic.” This single-minded focus on generating clicks led to the rise of clickbait, rage-bait, and low-quality, derivative content from publishers like the Huffington Post, Buzzfeed, and Demand Media, who optimized for cortisol responses rather than advancing human knowledge.
The new AI-driven paradigm offers a chance to correct this. AI models, in their aggregate, represent a vast block of “Swiss cheese”—mostly solid knowledge, but with significant holes. A new economic model should reward creators not for generating the most outrage, but for filling in these holes and providing novel information that adds incremental value to the sum of human knowledge.
Building a New Marketplace for Content
While many creators are looking to copyright law for protection, this may be a flawed strategy, as derivative works created by AI are often likely to be considered fair use. A more robust solution is to architect a new business model and marketplace. To have a market, you first need scarcity.
- Create Scarcity: Cloudflare has taken the first step with its “Content Independence Day” initiative, blocking AI training crawlers by default for all customers. This prevents the free, unilateral consumption of content and establishes control for the publisher.
- Level the Playing Field: The next step is to ensure all players operate under the same rules. This involves evolving standards like
robots.txtfor more fine-grained control and, crucially, forcing Google to abandon its “special snowflake” status. Google can no longer be allowed to use content for its AI products (like Gemini and AI overviews) for free under the old “we send you traffic” deal, while competitors are expected to pay. As regulatory and legal pressure mounts on Google, this playing field will likely be leveled. - Establish the Market: With scarcity and fair rules, a market can emerge. Initially, this will involve direct deals between large publishers (e.g., Reddit, New York Times) and large AI labs. For the long tail of creators and AI companies, an intermediary is needed. Cloudflare aims to help establish scalable models, potentially acting as a clearinghouse that negotiates rates and distributes payments, much like Spotify does for the music industry.
The Future of Infrastructure: Inference and Agents
As the web’s architecture evolves, the nature of compute changes from serving static pages to running inference.
- Inference at the Edge: While some models will run on-device, larger, resource-intensive models are best run at the network edge for low latency. Cloudflare is investing heavily in this, providing GPUs at the edge. Unlike hyperscalers who rent GPUs by time, Cloudflare’s pay-for-what-you-use model incentivizes a relentless drive for efficiency.
- Efficiency is the Unlock: The industry is just scratching the surface of model efficiency. Significant gains in model compression, pruning, and hardware performance (e.g., tensor units per watt) are on the horizon. The goal is to move beyond the current paradigm where more GPU power requires a “mothball nuclear power facility.” Soon, models with the power of today’s GPT-4 could run on a standard smartphone.
- Agentic Infrastructure: The next major wave will be agentic infrastructure. This requires new standards for agent-to-agent communication (like MCP), frameworks for orchestrating complex tasks, and robust identity and payment rails. The micropayment systems developed for the content marketplace could become the foundation for agent-to-agent commerce. Technologies like blockchain and Zero-Knowledge (ZK) proofs may finally find their killer application in providing the scalable, secure identity and permissioning layer required for this new agentic web.