# /robots.txt file for http://peter.tuna.fi/ # Block MSNBot: Crawls too often & uses too much bandwidth User-agent: msnbot Disallow: / # Often spam bots: User-agent: Java Disallow: / # Exclude some locations User-agent: * Disallow: /tmp/ Disallow: /demo/