Knowledgebase: Getting Started
How to use robots.txt
Posted by Daniel Scope on 04 December 2011 10:59 AM

What is the purpose of the robots file?
This file tells search engines what they should and should not index (make available as links).

You can simply create a blank file and name it robots.txt. (This will reduce site errors and allow all search engines to rank you if they want.)

If you want to stop search engines from ranking you, use this code:
#Code to not allow any search engines!
User-agent: *
Disallow: /

See more codes here:
http://www.robotstxt.org/robotstxt.html

(0 vote(s))
Helpful
Not helpful

Comments (0)