Google’s New Robots.txt Tester Tool for Webmasters
Robots.txt file is usually used to prevent the crawling of some specific pages. There are many tools available in market to do it. However, Google has announced a new robots.txt Tester tool to test the accuracy of your robots file.
We recommend you to use Google’s robots.txt tester tool even if you think the robots.txt file is okay. To find the testing tool go to:
Webmaster Tools> Crawl> Robots.txt Tester
Never be over confident, at-least test once.