How to block bots & content scrapers

22Slides allows you to edit your site's robots.txt file, so you can exercise some control over who (or what) accesses your site's content. You can made these changes from your Settings > Robots.txt section of your website's admin area.

Blocking password-protected pages is already handled automatically by our system, but editing your robots.txt file can be useful for things like blocking content scrapers or AI training bots.

You can read more about robots.txt files here: https://moz.com/learn/seo/robotstxt

The default robots.txt content allows all bots to access every page on your site (except password-protected pages, which is automatically handled elsewhere for security reasons):

User-agent: *
Allow: /

If you'd like to do something like disallow OpenAI's ChatGPT scraper from accessing your site, you can add these two lines to the beginning of the file:

User-agent: GPTBot
Disallow: /

User-agent: *
Allow: /

This instructs GPTBot to stay away, but allows all other bots to view your site. More info here: https://platform.openai.com/docs/gptbot

Please keep in mind that robots.txt is only a set of instructions that well-behaving bots will follow. Some bots may not follow the instructions provided, and malicious bots may actually check your robots.txt file for pages you explicitly disallow to look for ways to exploit sensitive content. This is one reason password-protected pages aren't handled by robots.txt files.