Of course not.Robots.txt I hope that most people know what the robots.txt file is for, but in case you don't, the robots.txt file is a file you place at the root of your website to tell web crawlers which paths on your websites they should crawl and which they should ignore.