Robots.txt & Sitemap Generator
Make the two files that tell Google what to crawl: robots.txt and sitemap.xml. Download them ready to upload.
Robots.txt & Sitemap Generator in 3 steps
- Enter your websiteYour main address, like https://yourbusiness.com.au.
- Choose the rulesHide admin, cart and checkout pages, block AI crawlers if you want, and list your pages for the sitemap.
- Download and uploadPut both files in your website’s main folder, then submit the sitemap in Google Search Console.
Not showing up on Google at all?
We fix indexing problems, set up Search Console and do the technical SEO, so Google can find every page.
Common questions
What does robots.txt do?
It tells search engine crawlers which parts of your site they should not visit, like admin or checkout pages. It does not hide pages from people, and it is not a security tool.
What is an XML sitemap?
A list of the pages you want Google to find. It helps new sites and big sites get indexed faster. Submit it in Google Search Console after uploading.
Should I block AI crawlers?
It is your choice. Blocking bots like GPTBot or CCBot asks AI companies not to use your content for training. It does not affect your Google rankings.
Can robots.txt remove a page from Google?
No. A blocked page can still appear if other sites link to it. To remove a page, use a noindex tag or delete it, and use the Removals tool in Search Console.
You might also like
GST Calculator
Add or remove 10% Australian GST from any price in one click.
Open toolQR Code Generator
Make a QR code for your website, Wi-Fi, email or text, in your brand colours, with a logo.
Open toolImage Compressor & Resizer
Shrink JPG, PNG and WebP photos for a faster website. Files never leave your device.
Open tool