Audit changes before publishing robots.txt.
- What you do
- Separates user-agent groups and Allow/Disallow rules.
- What you get
- User-agent tested
Product guide
Use cases
Free robots.txt simulator for Googlebot, GPTBot and other crawlers, with conflicts, winning rule and normalized file.
Practical example
- 01
Audit changes before publishing robots.txt.
- 02
Explain why a route is allowed or blocked.
- 03
Detect out-of-group rules and basic contradictions.
Explain why a route is allowed or blocked.
- What you do
- Select the most specific group and apply the longest path match.
- What you get
- local route
Detect out-of-group rules and basic contradictions.
- What you do
- Displays the winning rule, warnings, and a downloadable normalized robots.txt.
- What you get
- Winning rule
Responsible scope
Sources and scope
robots.txt is not authentication, security, or a deindexing command. The simulator implements a practical subset of RFC 9309 and does not query remote URLs.