# --------------------------------------------------------------------- # # robots.txt for our site # # Please note: There are a lot of pages on this site, and there are # some misbehaved spiders out there that goes way too fast. # If you are irresponsible, your access to the site may be blocked. # # --------------------------------------------------------------------- # Updated: # --------------------------------------------------------------------- User-agent: Slurp Crawl-delay: 0.5 User-agent: AboutUsBot Disallow: / User-agent: * Disallow: /test/ Disallow: /css/ Disallow: /picture_library/ Disallow: /plesk-stat/