robots.txt and meta tag generator
Write a robots.txt with crawl rules and a sitemap, plus the basic SEO meta tags for a page.
Runs in your browser. Nothing you enter is uploaded.
Meta tags
Meta tags
About this tool
This generator writes a robots.txt with a User-agent line, one Disallow rule per blocked path and an optional Sitemap line, plus the basic meta tags for a page. With no blocked paths it writes Allow: /. The meta output covers the title, description, canonical link, a robots tag and the matching Open Graph tags, with characters escaped for HTML attributes.
- Does Disallow in robots.txt keep a page out of search results?
- Disallow stops crawling, and a blocked URL can still be indexed if other pages link to it. To keep a page out of the index, leave it crawlable and use the noindex option in the meta tags.
- What do the 60 and 160 counters mean?
- They count the characters in the title and description against common length guides. Longer text is still written to the output.