Robots.txt Generator
This tool generates a robots.txt file based on your criteria.
Default Rule
This rule applies to all robots unless overridden by specific user agent rules.
Crawl Delay
The number of seconds bots should wait between requests (leave empty for no delay).
Common Restricted Directories
Add Common Search Engines
robots.txt Preview
A robots.txt file tells search engine crawlers which pages or files the crawler can or can't request from your site. This tool helps you create a properly formatted robots.txt file for your website.