Servus,
da ich erst seit kurzem ein Countersystem nutze welches auch Bots erkennt wundere ich mich doch so manches mal was da vorbei kommt.
Unter anderem kommt jener Omni Explorer reichlich oft auf allen meinen Seiten vorbei. Und wenn man dann liest welch informationen hier zusammen getragen wurden macht man sich doch ein wenig Gedanken.
Aktuell ist folgende Rule einzubauen um zu sperren!
Code:
RewriteCond %{HTTP_USER_AGENT} OmniExplorer [NC,OR]
RewriteCond %{REMOTE_ADDR} 65.19.150.(19[3-9]|25[0-4]|2[0-4][0-9]) [OR]
Zitat:
Omni-Explorer.
Omni-Explorer is a stealth-mode venture-backed startup based in Silicon Valley. Stay tuned to this site; we plan on launching shortly.
The Omni-Crawler
If you have found this page because of the Omni-crawler, please bear with us. We hope to be able to point many more users to the valuable content on your site shortly.
If you are finding our crawler overly burdensome on your site or prefer not to be included, you can exclude our crawler. Simply insert the following into your robots.txt file (if you don't know what one is, see the Robotstxt.org site).
The Omni-Explorer agent is: OmniExplorer_Bot/1.09. To prevent it from crawling your site, please put the following in your robots.txt file:
User-Agent: OmniExplorer_Bot
Disallow: /
We will also obey the delay directive (in seconds) for how long to wait between page views on your site:
Crawl-delay: 2
Some have written in to inquire if other crawlers they see are also ours; our IP address range is 65.19.150.193 - 65.19.150.254.
We know some of you got hit by an earlier version of our crawler that was particularly...well, hungry. For that, we sincerely apologize.
Please feel free to send us feedback. If you feel that the crawlers are not matching the behavior stated on this page, please include the HTTP log file lines and your robots.txt file (or site) so we can verify the issue. Thank you.
Page last modified 7/2/05
MfG
FB
Leider konnte ich nun in diesem posting nicht erkennen ob es DIE Lösung ist das ich einfach die robots.txt mit den entsprechenden Daten füttere, da man auch schrieb das 'böse' Bots die eh ignorieren.
Wäre daher sehr freundlich wenn man diesbezüglich eine Schlussantwort geben könnte.
MfG
Mindoro