For more than two decades, the economics of the web depended on a simple exchange. Search engines sent users to websites. Websites monetized those visitors through advertising. Content creators were indirectly compensated through traffic.

Artificial intelligence is disrupting that model.

Increasingly, users no longer navigate to information through search results. Instead, they receive synthesized answers directly from AI systems. The query remains, but the navigation layer disappears. That seemingly small behavioral change has significant consequences for the economics of the internet.

The Collapse of the Traffic Exchange

The traditional relationship between search engines and publishers was imperfect but functional. Search engines crawled content and, in return, sent human visitors back to websites. Those visitors generated advertising revenue that funded future content creation.

AI systems alter that exchange. Large language models consume vast quantities of online content but return only a fraction of the traffic historically generated by search engines. The result is a growing imbalance between content extraction and economic compensation.

This is not merely a publishing problem. It is an infrastructure problem. The open web relies on a continuous supply of original, human-created content. If the economic incentives supporting that content weaken, the broader ecosystem becomes less sustainable.

Content Is Becoming the Scarce Resource

For most of the internet era, distribution was scarce and content was abundant. Platforms that controlled distribution captured the majority of value.

AI reverses this dynamic.

Distribution is increasingly commoditized. What becomes scarce is original, trustworthy, and authoritative content. AI systems can summarize and recombine information, but they cannot independently generate the firsthand expertise, reporting, research, and judgment that make information valuable in the first place.

The AI industry depends on a continuous stream of new human-generated content. Yet the mechanisms that historically financed that content are under pressure from the same technologies consuming it.

Infrastructure Is Responding

One of the most significant developments came in 2025 when Cloudflare introduced Pay-Per-Crawl and began blocking AI crawlers by default for new domains.

The principle is straightforward: if AI systems derive value from content, access to that content should be governed through permission and compensation rather than unrestricted extraction.

The technical implementation may evolve, but the broader significance is clear. For the first time, infrastructure providers are creating mechanisms that allow content access to become a commercial transaction rather than an implicit assumption.

This represents the beginning of a new economic framework for AI-era content acquisition.

Why This Matters Beyond Publishers

The sustainability challenge extends beyond media organizations.

AI crawlers generate substantial traffic across ISP, hosting, CDN, and network infrastructure. As AI-driven content acquisition scales, questions emerge around who bears the costs and who captures the value.

A durable ecosystem requires incentives for all participants:

• Content creators who produce original material.
• Infrastructure providers that carry and deliver traffic.
• AI companies that rely on high-quality data.
• Users who expect accurate and trustworthy information.

If any of these groups are excluded from the value chain, the model becomes increasingly fragile.

What Comes Next

Advertising is unlikely to disappear, but its role is changing. The attention-interception model that dominated the web era depends on users navigating toward information. AI increasingly removes that navigation layer.

As a result, value is shifting away from traffic volume and toward content quality, expertise, licensing, and direct relationships between creators, platforms, and users.

The internet’s next economic model has not yet been fully defined. What is becoming clear, however, is that the assumptions that supported the advertising-driven web are weakening. The challenge for the industry is not simply adapting AI technologies but establishing sustainable economic relationships between those who create content, those who transport it, and those who use it.

The future of the open web may depend on getting that balance right.

This article is part of a bigger opinion piece. Read the full text on the IPXO blog.