Increase of AI bots on the Internet sparks arms race - Ars Technica
Summary
The rise of AI bots on the Internet is leading to an arms race between publishers and bot developers, as AI traffic surges and sophisticated scraping techniques challenge existing content protections.
Why It Matters
As AI bots increasingly dominate web traffic, understanding their impact on content accessibility and copyright issues is crucial for businesses and publishers. This shift could redefine online interactions and the economic landscape of the Internet, necessitating new strategies for content protection and monetization.
Key Takeaways
- AI bots now account for a significant portion of web traffic, with predictions of increased dominance in the future.
- Sophisticated scraping techniques are emerging, allowing bots to bypass traditional content protections like robots.txt.
- Publishers are responding with aggressive defenses, including legal actions against AI companies for copyright infringement.
- The arms race between AI bots and content providers will shape the future functionality and business models of the web.
- A new programmatic exchange of value between AI scrapers and content owners may be necessary to address these challenges.
Text settings Story text Size Small Standard Large Width * Standard Wide Links Standard Orange * Subscribers only Learn more Minimize to nav The viral virtual assistant OpenClaw—formerly known as Moltbot, and before that Clawdbot—is a symbol of a broader revolution underway that could fundamentally alter how the Internet functions. Instead of a place primarily inhabited by humans, the web may very soon be dominated by autonomous AI bots. A new report measuring bot activity on the web, as well as related data shared with WIRED by the Internet infrastructure company Akamai, shows that AI bots already account for a meaningful share of web traffic. The findings also shed light on an increasingly sophisticated arms race unfolding as bots deploy clever tactics to bypass website defenses meant to keep them out. “The majority of the Internet is going to be bot traffic in the future,” says Toshit Pangrahi, cofounder and CEO of TollBit, a company that tracks web-scraping activity and published the new report. “It’s not just a copyright problem, there is a new visitor emerging on the Internet.” Most big websites try to limit what content bots can scrape and feed to AI systems for training purposes. (WIRED’s parent company, Condé Nast, as well as other publishers, are currently suing several AI companies over alleged copyright infringement related to AI training.) But another kind of AI-related website scraping is now on the rise as well. Many chatbots and other AI tools can now ret...
