robots.txt Generator Tool

Default for All RobotsAllowedRefused

Crawl DelayNone5 Seconds10 Seconds20 Seconds60 Seconds120 Seconds

Sitemap

GoogleAllowedRefused

Google ImagesAllowedRefused

Google MobileAllowedRefused

MSN SearchAllowedRefused

YahooAllowedRefused

Yahoo MMAllowedRefused

Yahoo BlogsAllowedRefused

Ask/TeeomaAllowedRefused

GigaBlastAllowedRefused

DMOZ CheckerAllowedRefused

NutchAllowedRefused

Alexa / WaybackAllowedRefused

BaiduAllowedRefused

NaverAllowedRefused

MSN PicSearchAllowedRefused

Disallowed Directories

---

Robots.txt Generator

Robots.txt Generator is a useful tool that helps you generate Robots.txt files to handle many types of bot visitors on your website.