FBI Robots.txt

Home

Cryptome points us to this link: the robots.txt file at the FBI’s site. I’ll save you the click:


User-agent: *
Disallow: /

This isn’t the first time the government has shown some nervousness about their Web site content: The White House Vs. Search Spiders.

Deane | September 9, 2004
See Also: robots.txt

What This Links To

The following posts are linked to from this post.
  • The White House vs. Search Spiders
    Enabling historical revisionism: Very interesting note from the Democratic National Committee: Sometime between April 2003 and October 2003, someone at the…
    – October 27, 2003

Add a Comment

Name:
URL:

Comment:
Markdown is enabled in comments. If you have to use HTML, we allow A HREF, B, BR, P, STRONG, EM, UL, LI, and BLOCKQUOTE.
 
Comments on this site are subject to our Terms of Service