Content-type: text/html Downes.ca ~ Stephen's Web ~ Block the Bots that Feed "AI" Models by Scraping Your Website

Stephen Downes

Knowledge, Learning, Community

I won't be using this, since I'l happy to teach AIs about the world and my view of it, but if you're of a different mind, and view AI scraping as some sort of content theft, then this article describes how to block (compliant) AI systems from scraping your website. But honestly, I think people who are objecting to content scraping are very much missing the point here; it is in no sense a 'copying' of the content, it is a transformative use, and a use basically the same as that any reader of your site might undertake. Moreover, by blocking scraping, site owners are ensuring that the only AI will be commercial AI, created by companies that can afford to pay for large volumes of content. If we want open AI, we have to have open content.

Today: 0 Total: 1745 [Direct link] [Share]


Stephen Downes Stephen Downes, Casselman, Canada
stephen@downes.ca

Copyright 2024
Last Updated: Nov 03, 2024 2:55 p.m.

Canadian Flag Creative Commons License.

Force:yes