robots.txt Generator
Create robots.txt files for search engine crawlers.
About the robots.txt Generator
Control how search engines crawl your site with a properly formatted robots.txt file.
How to use the robots.txt Generator
- Set user-agent and disallow paths.
- Add your sitemap URL.
- Copy the generated robots.txt.
Frequently Asked Questions
- Where does robots.txt go?
- In your site root: https://yoursite.com/robots.txt
About this tool
All processing happens entirely in your browser. No data is uploaded to any server.
Developer Tools