DWeb Camp 2026

Save the Open Web from Evil Scraping Bots
2026-07-10 , AI Barn

I don't have any good answers but I'd like to discuss this with others. Websites like Internet Archive and Project Gutenberg are having trouble providing open access to content while being overwhelmed by badly behaved bots scraping content to train AI models. In response, Most open content websites are closing their websites to bots in general, even good, essential bots. How can we build a distributed system to facilitate and enforce good behavior by bots?


As I wrote, I don't have any answers, but I'm sure other people have thought more deeply about it. I can't do this session by myself, maybe it should be more of a BOF gathering, I'm submitting this hoping that it nucleates some good conversation.
Ideas:
- cryptographic user agents?
- aggregated reputation?
- metered fastlane access?

Present: Project Gutenberg, Free Ebook Foundation
Past: Gluejar, OCLC, Bell Labs.