Robots.txt Generator

Generate a simple robots.txt file.

OUTPUT

User-agent: * Allow: / Sitemap: https://example.com/sitemap.xml

Use cases

Create a simple robots.txt with sitemap location before launching a new website.

Draft allow and disallow rules before adding them to production.

Check whether crawler access and sitemap hints are represented clearly.

Does robots.txt guarantee indexing?+

No. It gives crawler instructions but does not guarantee indexing.

Where should it be placed?+

Place it at the root of your site, such as /robots.txt.

Endpoint

POST/api/tools/robots-txt

Request

{
  "input": "hello world",
  "options": {}
}

Response

{
  "result": "processed output",
  "tool": "robots_txt"
}