Tell Crawlers Your Sitemap Location via robots.txt

Originally posted on Friday, June 8th, 2007 at 2:49 pm in SEO.

A recent announcement from Google and Yahoo! led to their crawling engines to support an additional directive that can be included in the robots.txt file. By using the “Sitemap:” directive, you can tell crawlers the absolute URL of the XML sitemap for your website:

Sitemap: http://www.youdomain.com/sitemap.xml

Of course, you can still tell Yahoo! and Google about your sitemap and get some useful information on your website by using Google Webmaster Tools and Yahoo! Siteexplorer.