> OK, I have a minute so I was wondering what is robots.txt and what > is it's purpose and/or uses? > It's a way for web sites to tell bots/spiders/crawlers how to behave (e.g. what they can and cannot view), though it is of course entirely up to the bot to comply. http://www.robotstxt.org/wc/norobots.html