#This is a file retrieved by webwalkers a.k.a. spiders that conform to a defacto standard. # # JR initial usage at least from 10/15/2001, some refreshing 04/24/2006 # # Information Sources: # http://www.w3.org/TR/html4/appendix/notes.html#h-B.4.1 (w3.org see section B.4.1) # http://www.robotstxt.org/wc/robots.html # # File Name robots.txt (yes, lower case) # # The webmaster for this site is: # John Ream #
JohnOurRobotTxtFile at SensorySIMS.com
# #Format is: # User-agent: # Disallow: | # #----------------------------------------------------------------------------- # following prevents access to /misc/ for spiders # #User-Agent: Openfind data gatherer #Disallow: / #User-Agent: Openfind #Disallow: / User-Agent: * Disallow: /backup/ Disallow: /cgi-bin/ Disallow: /myimages/ Disallow: /webalbums/