# This robots.txt file is generated based on the IP Address and
# User Agent requesting it. We grant broad access to certain
# IPs and Agents -- mostly known search engine spiders. All
# others are sent the default rules denying access to the entire
# site. If you have a need to access our site via robots.txt,
# please send an explain by email to our webmaster and we will
# consider granting your IP Address and User Agent an appropriate
# level of access.
#
# Thank you for your interest.
#
User-agent: *
Disallow: /
Also robots.txt cloaking

Hat aber den Vorteil, dass man eine Whitelist führen kann, statt einer Blacklist, die ständig aktualisiert werden muss.
Gruss Gunnar