☀ ☾

Start here

Prompt engineering, AI tools, and honest looks at where AI agents actually save time.

Check a page before you publish it, free, right in your browser, no sign-up.

Robots.txt Generator Tool

SEO Tool

Robots.txt Generator

Create clean, SEO-friendly robots.txt rules for search engines, crawlers, and sitemap discovery.

🤖

Preset

User-agent

Crawl-delay optional

Disallow paths one per line

Allow paths one per line

Sitemap URLs one per line

Add rules to block popular AI crawlers Include helpful comments

Choose a preset or customize the fields, then generate your robots.txt file.

Live Output

Robots.txt gives instructions to compliant crawlers. It should not be used to protect private or sensitive content.