Search Engine Optimization Tip #6

Posted by Kristy

June 17, 2007

Use a robots.txt file to Tell Search Engine Spiders Where to Go.

Think of this simple text file as a roadmap of your web site. It should be placed in the root directory of your site and it tells the search engine spiders which directories they should go into and which ones they are not allowed in. It is one of the easiest tactics site owners and webmasters can use to help ensure their site gets crawled. It’s also one of the most overlooked opportunities.

To help understand the proper syntax of this file, visit Robotstxt.org. You should also include the location of your sitemap.xml file.