# Robot-Manager was used to generate this file. # Copyright (c) 2001 by Sophtware.com, Inc. All Rights Reserved. # http://www.websitemanagementtools.com/ # PARTIAL access (Alta Vista) User-agent: Scooter Disallow: /oldsite/ Disallow: /testbed/ Disallow: /cgi-bin/ # PARTIAL access (Entire Web) User-agent: speedy Disallow: /oldsite/ Disallow: /testbed/ Disallow: /cgi-bin/ # PARTIAL access (EuroSeek) User-agent: Freecrawl Disallow: /oldsite/ Disallow: /testbed/ Disallow: /cgi-bin/ # PARTIAL access (Excite) User-agent: ArchitextSpider Disallow: /oldsite/ Disallow: /testbed/ Disallow: /cgi-bin/ # PARTIAL access (Google) User-agent: Googlebot Disallow: /oldsite/ Disallow: /testbed/ Disallow: /cgi-bin/ # PARTIAL access (Inktomi) User-agent: Slurp Disallow: /oldsite/ Disallow: /testbed/ Disallow: /cgi-bin/ # PARTIAL access (Lycos) User-agent: Lycos_Spider_(T-Rex) Disallow: /oldsite/ Disallow: /testbed/ Disallow: /cgi-bin/ # PARTIAL access (Mirago) User-agent: mirago Disallow: /oldsite/ Disallow: /testbed/ Disallow: /cgi-bin/ # PARTIAL access (Northern Light) User-agent: Gulliver Disallow: /oldsite/ Disallow: /testbed/ Disallow: /cgi-bin/ # PARTIAL access (SearchUK) User-agent: MegaSheep Disallow: /oldsite/ Disallow: /testbed/ Disallow: /cgi-bin/ # PARTIAL access (whatUseek) User-agent: winona Disallow: /oldsite/ Disallow: /testbed/ Disallow: /cgi-bin/ # PARTIAL access (All Spiders) User-agent: * Disallow: /400.shtml Disallow: /401.shtml Disallow: /403.shtml Disallow: /404.shtml Disallow: /500.shtml Disallow: /oldsite/ Disallow: /testbed/ Disallow: /cgi-bin/