TECHNOLOGYBUSINESS INSIDER
Cloudflare's latest AI rankings expose the web's biggest free rider
Cloudflare's data reveals Anthropic's AI bots have the highest crawl-to-refer ratio, scraping webpages 2,800 times for every referral, far exceeding OpenAI and others. DuckDuckGo shows a more balanced ratio of 3 scrapes per referral, while Anthropic's practices raise concerns about the web's economic model as AI companies prioritize content extraction over traffic generation.
Mentioned
Related Signal
Adjacent reporting
- Free Apps Are Quietly Turning Smart TVs Into Web-Scraping Proxies for AI
- Cloudflare puts prices on AI scraping
- Google is cannibalizing the web to feed AI
- Internet traffic from AI agents and other bots now surpasses that of people
- AI and bot internet activity outpaces human use for first time in history: ‘On the other side now’