http://gadgetopia.com/post/5784?utm_source=rss&utm_medium=rss&utm_campaign=Robots.txt%3A+A+Cracker%27s+Best+Friend Geek and you shall find... Tue, 21 May 2013 22:01:41 +0000 hourly 1 http://wordpress.org/?v=3.3.2 http://gadgetopia.com/post/5784#comment-5268 putanerd Wed, 15 Aug 2007 15:54:00 +0000 http://gadgetopia.local/?p=5784#comment-5268 <p>I like the way your think Michael. I will be doing the same.</p> I like the way your think Michael. I will be doing the same.

]]> http://gadgetopia.com/post/5784#comment-6072 Michael Sun, 25 Feb 2007 12:29:00 +0000 http://gadgetopia.local/?p=5784#comment-6072 <p>I'm using this as a spammer and bad-robot bait for years. There's a line in the robots.txt of a directory which is not linked from anywhere else. Every IP which accesses that directory can only have found out by looking at the robots.txt first. Works pretty nice.</p> I’m using this as a spammer and bad-robot bait for years. There’s a line in the robots.txt of a directory which is not linked from anywhere else. Every IP which accesses that directory can only have found out by looking at the robots.txt first. Works pretty nice.

]]> http://gadgetopia.com/post/5784#comment-6083 Brian Thu, 22 Feb 2007 04:21:00 +0000 http://gadgetopia.local/?p=5784#comment-6083 <p>I would say that anyone... "creative" enough to host install files and license keys in the root of their web server without any directory-level security probably deserves to have their stuff messed with. Have they never heard of secure FTP? Integrated authentication? I wouldn't think too many sane people would even consider "protecting" anything meaningful with robots.txt.</p> I would say that anyone… “creative” enough to host install files and license keys in the root of their web server without any directory-level security probably deserves to have their stuff messed with. Have they never heard of secure FTP? Integrated authentication? I wouldn’t think too many sane people would even consider “protecting” anything meaningful with robots.txt.

]]> http://gadgetopia.com/post/5784#comment-6093 Craig Fleming Wed, 21 Feb 2007 12:37:00 +0000 http://gadgetopia.local/?p=5784#comment-6093 <p>Most search engines today (Google for instance) completely ignore this directive anyhow.</p> Most search engines today (Google for instance) completely ignore this directive anyhow.

]]>