Content Klau - HTTP_USER_AGENT Liste ?!

Beitrag von **Jarez** » 19.01.2006, 20:04

Hallo zusammen,

ich beschäftige mich im Moment mit dem Content Klau und wie ich das auf meinen Seiten verhindern oder zumindest eindämmen kann. Es scheint ja vielen Webmastern so zu gehen, das Webseiten (fast) komplett kopiert werden ... die Texte 1:1 übernommen werden und meist auch direkt die Bilder mit übernommen werden.

Programme wie Mister Pix, Website Copier usw. in allen Ausführungen ...

Eindämmen kann man das Problem ja mittels der .htaccess und
...
RewriteCond %{HTTP_USER_AGENT} ^Mister\ PiX [OR]
...

Wenn sich die Leute Bilder zum "Eigenbedarf" runterladen und viel Traffic verursachen ist es ja schon ärgerlich aber wenn andere Webmaster auf diesem Wege schnell den Content abgrasen finde ich es viel ätzender.

Gibt es denn eine Liste von wichtige HTTP_USER_AGENT's die man getrost sperren kann bzw. sperren sollte, damit man seinen Content nicht nächste Woche auf allen andere Seiten im Web sieht?

Eine sehr umfangreiche Liste ist ja hier:
https://www.abakus-internet-marketing.d ... agent.html

Sind das so in etwa die wichtigsten?!

Die "vernünftigen" SuMa Bots (wie Google usw.) beeinträchtigt so etwas ja nicht, oder??

Der Mozilla Kram kann doch raus, oder??

Gruss
Jarez

von **Anzeige von ABAKUS** »

Beitrag von **Airport1** » 19.01.2006, 21:08

> Gibt es denn eine Liste von wichtige HTTP_USER_AGENT's die man getrost sperren kann bzw. sperren sollte,

Solange niemand mit anderen zusammenarbeiten will wird das nie was werden. Es gab da zwar schon Ansaetze wie auf disallow.de geschildert... anscheinend wird das Rad lieber laufend neu erfunden oder jeder broedelt was vor sich hin..

Wollte sowas auf https://www.bot-trap.de/ anbieten, aber da fehlt mir momentan ein Design und auch etwas die Zeit.. man muesste sich clonen koennen

Beitrag von **JR-EWING** » 20.01.2006, 00:53

Solange niemand mit anderen zusammenarbeiten will

Von meiner Seite schon.

von **Anzeige von ABAKUS** »

Content Erstellung von ABAKUS Internet Marketing
Ihre Vorteile:

einzigartige Texte
suchmaschinenoptimierte Inhalte
eine sinnvolle Content-Strategie
Beratung und Umsetzung

Jetzt anfragen: 0511 / 300325-0

Beitrag von **Hobby-SEO** » 20.01.2006, 12:03

Jarez hat geschrieben:Der Mozilla Kram kann doch raus, oder??

Hallo Jarez,

was meinst Du damit? Du willst nicht im Ernst die User-agents sperren, die mit "Mozilla" anfangen, oder? Damit würdest Du nicht zuletzt fast alle menschlichen Besucher Deiner Website ausschließen.

https://www.joergkrusesweb.de/internet/ ... agent.html

LG, Hobby-SEO

Beitrag von **DZVO** » 20.01.2006, 12:32

https://www.spider-trap.de/ schau dir das projekt mal an vllt ist das schon deine lösung

Beitrag von **Jarez** » 20.01.2006, 12:55

Hallo Hobby-Seo,

die User-Agents die für Browser stehen (mit denen der User auf die Webseiten kommt) sollen natürlich nicht gesperrt werden. Es wäre ja sonst Blödsinn und ich bräuchte keine Webseite ins Netz stellen.

Ich meinte damit dass die Einträge aus der .htaccess raus können.

@DZVO

dein Projekt sieht auf den ersten Blick recht vielversprechend aus. Ich werde mir das Ganze mal in Ruhe ansehen.

@all

Dieses Projekt hier: https://www.web4free.at/Antispam.Htaccess
finde ich recht interessant, dort kann man sich die User-Agents direkt nach Kategorie ansehen. Weiss jemand wie gut bzw. aktuell diese sind?

Gibt es eigentlich für "HTTrack Website Copier" einen wirksamen Schutz um das Teil auszusperren. Dort kann man ja in den Einstellungen einen anderen User-Agent vorgaugeln und schon versagt der .htaccess Schutz auch wenn dort HTTrack gesperrt ist.

Gruss

Beitrag von **Jarez** » 20.01.2006, 13:13

Wichtige User-Agents (wie von SuMa Spidern und den Browsern mit denen die Webseiten aufgerufen werden) möchte ich natürlich nicht sperren.

Da es hier doch schon einige Experten auf dem Gebiet gibt, schmeiss ich gleich mal die nächste Frage ins Forum. Bisher habe ich mir diese kleine Sammlung zusammengestellt.

Fehlen dort wichtige Agents? Welche sollte ich vielleicht lieber raus nehmen?

RewriteCond %{HTTP_USER_AGENT} ^Anarchie/3.5.1D [OR]
RewriteCond %{HTTP_USER_AGENT} ^Anarchie/3.5J2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Anarchie/3.6.2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^DA\ 3.5\ (www.lidan.com) [OR]
RewriteCond %{HTTP_USER_AGENT} ^DA\ 3.5\ (www.lidan.com)\ WebWasher/1.0.2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^DA\ 4.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^DA\ 5.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^EliteSys\ SuperBot/2.0\ (Win32) [OR]
RewriteCond %{HTTP_USER_AGENT} ^FlashGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/3.2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/3.3 [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/3.3.3 [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/3.3.4 [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/3.3J [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/4.1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/4.1.1 [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/4.1.2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/4.2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/4.2b [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/4.2c [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/4.3 [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/4.5 [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/4.5b3 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla\ 3.3 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla\ 3.5 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Gozilla/6.5\ en [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mercator-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mercator-1.1 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mercator-1.2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mercator-2.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mercator-Scrub-1.1 [OR]
RewriteCond %{HTTP_USER_AGENT} ^MyGetRight/0.9.04 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Net\ Vampire/3.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Net\ Vampire/3.3 [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetAnts/1.00 [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetAnts/1.10 [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetZip-Downloader/1.0.58\ (Win32;\ Dec\ 21998) [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer/1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer/1.1 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer/1.2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer/1.3 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer/1.4 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer/1.5 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer/1.6 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer/1.8 [OR]
RewriteCond %{HTTP_USER_AGENT} ^pavuk-0.9pl17\ i686-pc-linux-gnu [OR]
RewriteCond %{HTTP_USER_AGENT} ^pavuk/0.9pl23\ i686-pc-linux-gnu [OR]
RewriteCond %{HTTP_USER_AGENT} ^pavuk/0.9pl24\ sparc-sun-solaris2.7 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Scooter2_Mercator_1-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Scooter2_Mercator_2-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Scooter2_Mercator_3-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Scooter2_Mercator_4-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Scooter2_Mercator_6-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Scooter2_Mercator_lca-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Scooter2_Mercator_lcb-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Scooter2_Mercator_url-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Scooter_Mercator_1-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^SQ\ Webscanner [OR]
RewriteCond %{HTTP_USER_AGENT} ^Stamina/1.4 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro\ 1.29 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro/1.28 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro/1.29 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro/1.29.1590 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro/1.29.1616 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_0-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_1-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_2-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_3-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_n0b-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_n1b-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_n2b-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_n2s0_A-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_n2s0_B-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_n2s1_A-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_n2s1_B-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_n3b-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_4-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_5-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_5b-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_6-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_7-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_dmb-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_fx4-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_fx6-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_fx7-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_fxa-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_hc-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_n2s2_A-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_n2s2_B-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_n2s3_A-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_n2s3_B-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_rdm-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_4444-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_5-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_6-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_7-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n0c-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n0d-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n2c-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n2d-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n2s4_A-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n2s4_B-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n2s5_A-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n2s5_B-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n3c-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n3d-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n5c-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n5d-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_10-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_11-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_8-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_9-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_f5-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_n1c-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_n1d-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_n2s6_A-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_n2s6_B-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_n2s7_A-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_n2s7_B-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_n6b-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_n6c-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_n6d-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_n7b-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_n7c-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier\ Session\ 2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier\ Session\ 4 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier\ Session\ 9 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier\ v.2.2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper\ v8.1 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper\ v9.1 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper\ v9.3 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper\ v9.4 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper\ vWebReaperv7.2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper\ vWebReaperv7.3 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper\ info@webreaper.net [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper\ webreaper@otway.com [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebSauger\ 1.20b [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper/1.05 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper/1.1 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper/1.22 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper/1.38 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper/1.40 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper/2.0\ Beta2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper/2.02 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/2.42 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/3.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/3.03 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/3.06 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/3.07 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/3.5 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/3.6 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/3.66 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/3.7 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/3.71 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/3.80 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/4.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget/1.4.5 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget/1.5.2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget/1.5.3 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget/1.5.3.1 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget/1.6 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE/2.2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE/2.4 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE/2.5
RewriteCond %{HTTP_USER_AGENT} ^Alexibot [OR]
RewriteCond %{HTTP_USER_AGENT} ^asterias [OR]
RewriteCond %{HTTP_USER_AGENT} ^BackDoorBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^Black.Hole [OR]
RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR]
RewriteCond %{HTTP_USER_AGENT} ^BlowFish [OR]
RewriteCond %{HTTP_USER_AGENT} ^BotALot [OR]
RewriteCond %{HTTP_USER_AGENT} ^BuiltBotTough [OR]
RewriteCond %{HTTP_USER_AGENT} ^Bullseye [OR]
RewriteCond %{HTTP_USER_AGENT} ^BunnySlippers [OR]
RewriteCond %{HTTP_USER_AGENT} ^Cegbfeieh [OR]
RewriteCond %{HTTP_USER_AGENT} ^CheeseBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^CherryPicker [OR]
RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR]
RewriteCond %{HTTP_USER_AGENT} ^CopyRightCheck [OR]
RewriteCond %{HTTP_USER_AGENT} ^cosmos [OR]
RewriteCond %{HTTP_USER_AGENT} ^Crescent [OR]
RewriteCond %{HTTP_USER_AGENT} ^Custo [OR]
RewriteCond %{HTTP_USER_AGENT} ^DISCo [OR]
RewriteCond %{HTTP_USER_AGENT} ^DittoSpyder [OR]
RewriteCond %{HTTP_USER_AGENT} ^Download\ Demon [OR]
RewriteCond %{HTTP_USER_AGENT} ^eCatch [OR]
RewriteCond %{HTTP_USER_AGENT} ^EirGrabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailCollector [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailWolf\ 1.00 [OR]
RewriteCond %{HTTP_USER_AGENT} ^EroCrawler [OR]
RewriteCond %{HTTP_USER_AGENT} ^Express\ WebPictures [OR]
RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [OR]
RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE [OR]
RewriteCond %{HTTP_USER_AGENT} ^FlashGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^Foobot [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetWeb! [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It [OR]
RewriteCond %{HTTP_USER_AGENT} ^GrabNet [OR]
RewriteCond %{HTTP_USER_AGENT} ^Grafula [OR]
RewriteCond %{HTTP_USER_AGENT} ^Harvest [OR]
RewriteCond %{HTTP_USER_AGENT} ^hloader [OR]
RewriteCond %{HTTP_USER_AGENT} ^HMView [OR]
RewriteCond %{HTTP_USER_AGENT} ^httplib [OR]
RewriteCond %{HTTP_USER_AGENT} ^HTTrack [OR]
RewriteCond %{HTTP_USER_AGENT} ^humanlinks [OR]
RewriteCond %{HTTP_USER_AGENT} ^ia_archiver [OR]
RewriteCond %{HTTP_USER_AGENT} ^Image\ Stripper [OR]
RewriteCond %{HTTP_USER_AGENT} ^Image\ Sucker [OR]
RewriteCond %{HTTP_USER_AGENT} ^Indy\ Library [OR]
RewriteCond %{HTTP_USER_AGENT} ^InfoNaviRobot [OR]
RewriteCond %{HTTP_USER_AGENT} ^InterGET [OR]
RewriteCond %{HTTP_USER_AGENT} ^Internet\ Ninja [OR]
RewriteCond %{HTTP_USER_AGENT} ^JennyBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^JetCar [OR]
RewriteCond %{HTTP_USER_AGENT} ^JOC\ Web\ Spider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Kenjin.Spider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Keyword.Density [OR]
RewriteCond %{HTTP_USER_AGENT} ^larbin [OR]
RewriteCond %{HTTP_USER_AGENT} ^LeechFTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^LexiBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^libWeb/clsHTTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^LinkextractorPro [OR]
RewriteCond %{HTTP_USER_AGENT} ^LinkScan/8.1a.Unix [OR]
RewriteCond %{HTTP_USER_AGENT} ^LinkWalker [OR]
RewriteCond %{HTTP_USER_AGENT} ^lwp-trivial [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mass\ Downloader [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mata.Hari [OR]
RewriteCond %{HTTP_USER_AGENT} ^Microsoft.URL [OR]
RewriteCond %{HTTP_USER_AGENT} ^MIDown\ tool [OR]
RewriteCond %{HTTP_USER_AGENT} ^MIIxpc [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mister\ PiX [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mister.PiX [OR]
RewriteCond %{HTTP_USER_AGENT} ^moget [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mozilla.*NEWT [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/3.Mozilla/2.01 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Navroad [OR]
RewriteCond %{HTTP_USER_AGENT} ^NearSite [OR]
RewriteCond %{HTTP_USER_AGENT} ^Net\ Vampire [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetAnts [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetMechanic [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetZIP [OR]
RewriteCond %{HTTP_USER_AGENT} ^NICErsPRO [OR]
RewriteCond %{HTTP_USER_AGENT} ^NPBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^Octopus [OR]
RewriteCond %{HTTP_USER_AGENT} ^Openfind [OR]
RewriteCond %{HTTP_USER_AGENT} ^PageGrabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^Papa\ Foto [OR]
RewriteCond %{HTTP_USER_AGENT} ^pavuk [OR]
RewriteCond %{HTTP_USER_AGENT} ^pcBrowser [OR]
RewriteCond %{HTTP_USER_AGENT} ^ProPowerBot/2.14 [OR]
RewriteCond %{HTTP_USER_AGENT} ^ProWebWalker [OR]
RewriteCond %{HTTP_USER_AGENT} ^ProWebWalker [OR]
RewriteCond %{HTTP_USER_AGENT} ^QueryN.Metasearch [OR]
RewriteCond %{HTTP_USER_AGENT} ^ReGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^RepoMonkey [OR]
RewriteCond %{HTTP_USER_AGENT} ^RMA [OR]
RewriteCond %{HTTP_USER_AGENT} ^SiteSnagger [OR]
RewriteCond %{HTTP_USER_AGENT} ^SlySearch [OR]
RewriteCond %{HTTP_USER_AGENT} ^SmartDownload [OR]
RewriteCond %{HTTP_USER_AGENT} ^SpankBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^spanner [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Surfbot [OR]
RewriteCond %{HTTP_USER_AGENT} ^suzuran [OR]
RewriteCond %{HTTP_USER_AGENT} ^tAkeOut [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro [OR]
RewriteCond %{HTTP_USER_AGENT} ^Telesoft [OR]
RewriteCond %{HTTP_USER_AGENT} ^The.Intraformant [OR]
RewriteCond %{HTTP_USER_AGENT} ^TheNomad [OR]
RewriteCond %{HTTP_USER_AGENT} ^TightTwatBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^Titan [OR]
RewriteCond %{HTTP_USER_AGENT} ^toCrawl/UrlDispatcher [OR]
RewriteCond %{HTTP_USER_AGENT} ^True_Robot [OR]
RewriteCond %{HTTP_USER_AGENT} ^turingos [OR]
RewriteCond %{HTTP_USER_AGENT} ^TurnitinBot/1.5 [OR]
RewriteCond %{HTTP_USER_AGENT} ^URLy.Warning [OR]
RewriteCond %{HTTP_USER_AGENT} ^VCI [OR]
RewriteCond %{HTTP_USER_AGENT} ^VoidEYE [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\ Image\ Collector [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\ Sucker [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web.Image.Collector [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebAuto [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebBandit [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebEMailExtrac.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebEnhancer [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebFetch [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebGo\ IS [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebLeacher [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebmasterWorldForumBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebSauger [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website\ eXtractor [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website\ Quester [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website.Quester [OR]
RewriteCond %{HTTP_USER_AGENT} ^Webster.Pro [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebWhacker [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZip [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget [OR]
RewriteCond %{HTTP_USER_AGENT} ^Widow [OR]
RewriteCond %{HTTP_USER_AGENT} ^WWW-Collector-E [OR]
RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE [OR]
RewriteCond %{HTTP_USER_AGENT} ^Xaldon\ WebSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Xenu's [OR]
RewriteCond %{HTTP_USER_AGENT} ^Zeus

Danke im voraus.

Beitrag von **Hobby-SEO** » 20.01.2006, 13:17

Jarez hat geschrieben:Ich meinte damit dass die Einträge aus der .htaccess raus können.

Bin mir nicht sicher, ob ich Dich jetzt verstehe. Es gibt aber (mangelhaft) gefälschte User-agents, die nur so "tun", als wären sie "normale" Browser. Deshalb haben manche "Mozilla"-Einträge in der .htaccess schon Sinn.

Jarez hat geschrieben:Gibt es eigentlich für "HTTrack Website Copier" einen wirksamen Schutz um das Teil auszusperren. Dort kann man ja in den Einstellungen einen anderen User-Agent vorgaugeln und schon versagt der .htaccess Schutz auch wenn dort HTTrack gesperrt ist.

Ich fürchte, dass die Schutzmaßnahmen hier an ihre Grenzen stoßen (würde mich aber SEHR gerne eines Besseren belehren lassen). Man kann nur zusätzlich noch IPs sperren und eine Spider-Trap einbauen.

Wenn es aber jemand ganz gezielt (und nicht automatisiert) auf Dich abgesehen hat, hast Du gegen Content-Klau wahrscheinlich kaum Chancen - zumindest nicht prophylaktisch. Und auch diesbezüglich würde ich mich gerne eines Besseren belehren lassen.

LG, Hobby-SEO

Beitrag von **Hobby-SEO** » 20.01.2006, 13:20

Ahoi,

Du kannst die Liste zumindest schon einmal stark verkürzen. Ein Beispiel:

Code: Alles auswählen

RewriteCond %&#123;HTTP_USER_AGENT&#125; ^GetRight/3.2 &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^GetRight/3.3 &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^GetRight/3.3.3 &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^GetRight/3.3.4 &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^GetRight/3.3J &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^GetRight/4.1.0 &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^GetRight/4.1.1 &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^GetRight/4.1.2 &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^GetRight/4.2 &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^GetRight/4.2b &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^GetRight/4.2c &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^GetRight/4.3 &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^GetRight/4.5 &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^GetRight/4.5b3 &#91;OR&#93;

ist identisch mit:

Code: Alles auswählen

RewriteCond %&#123;HTTP_USER_AGENT&#125; ^GetRight &#91;OR&#93;

Und aus

Code: Alles auswählen

RewriteCond %&#123;HTTP_USER_AGENT&#125; ^Web\ Image\ Collector &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^Web\ Sucker &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^Web.Image.Collector &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^WebAuto &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^WebBandit &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^WebCopier &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^WebEMailExtrac.* &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^WebEnhancer &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^WebFetch &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^WebGo\ IS &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^WebLeacher &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^WebmasterWorldForumBot &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^WebReaper &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^WebSauger &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^Website\ eXtractor &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^Website\ Quester &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^Website.Quester &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^Webster.Pro &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^WebStripper &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^WebWhacker &#91;OR&#93; 
RewriteCond %&#123;HTTP_USER_AGENT&#125; ^WebZip &#91;OR&#93;

wird einfach

Code: Alles auswählen

RewriteCond %&#123;HTTP_USER_AGENT&#125; ^Web &#91;OR&#93;

Und so weiter und so fort.

LG, Hobby-SEO

Beitrag von **Jarez** » 20.01.2006, 13:27

@Hobby-SEO

vielen Dank für die Hinweise mit dem kürzen. Werde mal durchschauen, was da so raus fliegen kann.

Welche "Mozilla" Einträge würden denn Sinn machen ... also welche sollte in die .htaccess mit rein um sie zu sperren? Gibt es da eine kleine Übersicht?

Ich möchte halt nicht User mit ihrem Browser aussperren.

Beitrag von **Jarez** » 20.01.2006, 13:32

In HTTrack kann man solche User-Agents vorgaukeln:

Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.1) Gecko/20020826
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)

usw. ... das sind doch aber fast die gleichen wie hier: https://www.joergkrusesweb.de/internet/ ... agent.html

Da stösst doch der .htaccess Schutz doch schon an seine Grenzen, weil man viel zu schnell Gefahr läuft die "richtigen" User auszusperren?!! Oder?!?!

Beitrag von **easycool** » 14.04.2006, 18:57

Hallo.

Habe ja schon in dem anderen Thread von neuen Gästen berichtet.
Siehe hier:
https://www.abakus-internet-marketing.d ... 26429.html

Mich würde interessieren, ob man diese Liste hier mit gutem Gewissen übernehmen kann - ist ja jetzt schon wieder einige Zeit ins Land gegangen.

RewriteCond %{HTTP_USER_AGENT} ^Anarchie/3.5.1D [OR]
RewriteCond %{HTTP_USER_AGENT} ^Anarchie/3.5J2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Anarchie/3.6.2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^DA\ 3.5\ (www.lidan.com) [OR]
RewriteCond %{HTTP_USER_AGENT} ^DA\ 3.5\ (www.lidan.com)\ WebWasher/1.0.2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^DA\ 4.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^DA\ 5.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^EliteSys\ SuperBot/2.0\ (Win32) [OR]
RewriteCond %{HTTP_USER_AGENT} ^FlashGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/3.2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/3.3 [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/3.3.3 [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/3.3.4 [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/3.3J [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/4.1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/4.1.1 [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/4.1.2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/4.2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/4.2b [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/4.2c [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/4.3 [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/4.5 [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight/4.5b3 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla\ 3.3 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla\ 3.5 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Gozilla/6.5\ en [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mercator-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mercator-1.1 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mercator-1.2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mercator-2.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mercator-Scrub-1.1 [OR]
RewriteCond %{HTTP_USER_AGENT} ^MyGetRight/0.9.04 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Net\ Vampire/3.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Net\ Vampire/3.3 [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetAnts/1.00 [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetAnts/1.10 [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetZip-Downloader/1.0.58\ (Win32;\ Dec\ 21998) [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer/1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer/1.1 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer/1.2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer/1.3 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer/1.4 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer/1.5 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer/1.6 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer/1.8 [OR]
RewriteCond %{HTTP_USER_AGENT} ^pavuk-0.9pl17\ i686-pc-linux-gnu [OR]
RewriteCond %{HTTP_USER_AGENT} ^pavuk/0.9pl23\ i686-pc-linux-gnu [OR]
RewriteCond %{HTTP_USER_AGENT} ^pavuk/0.9pl24\ sparc-sun-solaris2.7 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Scooter2_Mercator_1-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Scooter2_Mercator_2-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Scooter2_Mercator_3-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Scooter2_Mercator_4-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Scooter2_Mercator_6-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Scooter2_Mercator_lca-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Scooter2_Mercator_lcb-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Scooter2_Mercator_url-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Scooter_Mercator_1-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^SQ\ Webscanner [OR]
RewriteCond %{HTTP_USER_AGENT} ^Stamina/1.4 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro\ 1.29 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro/1.28 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro/1.29 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro/1.29.1590 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro/1.29.1616 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_0-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_1-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_2-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_3-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_n0b-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_n1b-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_n2b-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_n2s0_A-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_n2s0_B-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_n2s1_A-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_n2s1_B-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator_n3b-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_4-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_5-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_5b-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_6-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_7-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_dmb-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_fx4-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_fx6-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_fx7-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_fxa-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_hc-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_n2s2_A-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_n2s2_B-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_n2s3_A-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_n2s3_B-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV34_Mercator_rdm-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_4444-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_5-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_6-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_7-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n0c-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n0d-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n2c-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n2d-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n2s4_A-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n2s4_B-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n2s5_A-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n2s5_B-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n3c-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n3d-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n5c-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV35_Mercator_n5d-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_10-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_11-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_8-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_9-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_f5-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_n1c-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_n1d-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_n2s6_A-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_n2s6_B-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_n2s7_A-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_n2s7_B-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_n6b-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_n6c-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_n6d-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_n7b-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV36_Mercator_n7c-1.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier\ Session\ 2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier\ Session\ 4 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier\ Session\ 9 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier\ v.2.2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper\ v8.1 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper\ v9.1 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper\ v9.3 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper\ v9.4 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper\ vWebReaperv7.2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper\ vWebReaperv7.3 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper\ info@webreaper.net [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper\ webreaper@otway.com [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebSauger\ 1.20b [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper/1.05 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper/1.1 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper/1.22 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper/1.38 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper/1.40 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper/2.0\ Beta2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper/2.02 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/2.42 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/3.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/3.03 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/3.06 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/3.07 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/3.5 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/3.6 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/3.66 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/3.7 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/3.71 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/3.80 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP/4.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget/1.4.5 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget/1.5.2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget/1.5.3 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget/1.5.3.1 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget/1.6 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE/2.2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE/2.4 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE/2.5
RewriteCond %{HTTP_USER_AGENT} ^Alexibot [OR]
RewriteCond %{HTTP_USER_AGENT} ^asterias [OR]
RewriteCond %{HTTP_USER_AGENT} ^BackDoorBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^Black.Hole [OR]
RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR]
RewriteCond %{HTTP_USER_AGENT} ^BlowFish [OR]
RewriteCond %{HTTP_USER_AGENT} ^BotALot [OR]
RewriteCond %{HTTP_USER_AGENT} ^BuiltBotTough [OR]
RewriteCond %{HTTP_USER_AGENT} ^Bullseye [OR]
RewriteCond %{HTTP_USER_AGENT} ^BunnySlippers [OR]
RewriteCond %{HTTP_USER_AGENT} ^Cegbfeieh [OR]
RewriteCond %{HTTP_USER_AGENT} ^CheeseBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^CherryPicker [OR]
RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR]
RewriteCond %{HTTP_USER_AGENT} ^CopyRightCheck [OR]
RewriteCond %{HTTP_USER_AGENT} ^cosmos [OR]
RewriteCond %{HTTP_USER_AGENT} ^Crescent [OR]
RewriteCond %{HTTP_USER_AGENT} ^Custo [OR]
RewriteCond %{HTTP_USER_AGENT} ^DISCo [OR]
RewriteCond %{HTTP_USER_AGENT} ^DittoSpyder [OR]
RewriteCond %{HTTP_USER_AGENT} ^Download\ Demon [OR]
RewriteCond %{HTTP_USER_AGENT} ^eCatch [OR]
RewriteCond %{HTTP_USER_AGENT} ^EirGrabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailCollector [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailWolf\ 1.00 [OR]
RewriteCond %{HTTP_USER_AGENT} ^EroCrawler [OR]
RewriteCond %{HTTP_USER_AGENT} ^Express\ WebPictures [OR]
RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [OR]
RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE [OR]
RewriteCond %{HTTP_USER_AGENT} ^FlashGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^Foobot [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetWeb! [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It [OR]
RewriteCond %{HTTP_USER_AGENT} ^GrabNet [OR]
RewriteCond %{HTTP_USER_AGENT} ^Grafula [OR]
RewriteCond %{HTTP_USER_AGENT} ^Harvest [OR]
RewriteCond %{HTTP_USER_AGENT} ^hloader [OR]
RewriteCond %{HTTP_USER_AGENT} ^HMView [OR]
RewriteCond %{HTTP_USER_AGENT} ^httplib [OR]
RewriteCond %{HTTP_USER_AGENT} ^HTTrack [OR]
RewriteCond %{HTTP_USER_AGENT} ^humanlinks [OR]
RewriteCond %{HTTP_USER_AGENT} ^ia_archiver [OR]
RewriteCond %{HTTP_USER_AGENT} ^Image\ Stripper [OR]
RewriteCond %{HTTP_USER_AGENT} ^Image\ Sucker [OR]
RewriteCond %{HTTP_USER_AGENT} ^Indy\ Library [OR]
RewriteCond %{HTTP_USER_AGENT} ^InfoNaviRobot [OR]
RewriteCond %{HTTP_USER_AGENT} ^InterGET [OR]
RewriteCond %{HTTP_USER_AGENT} ^Internet\ Ninja [OR]
RewriteCond %{HTTP_USER_AGENT} ^JennyBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^JetCar [OR]
RewriteCond %{HTTP_USER_AGENT} ^JOC\ Web\ Spider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Kenjin.Spider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Keyword.Density [OR]
RewriteCond %{HTTP_USER_AGENT} ^larbin [OR]
RewriteCond %{HTTP_USER_AGENT} ^LeechFTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^LexiBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^libWeb/clsHTTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^LinkextractorPro [OR]
RewriteCond %{HTTP_USER_AGENT} ^LinkScan/8.1a.Unix [OR]
RewriteCond %{HTTP_USER_AGENT} ^LinkWalker [OR]
RewriteCond %{HTTP_USER_AGENT} ^lwp-trivial [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mass\ Downloader [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mata.Hari [OR]
RewriteCond %{HTTP_USER_AGENT} ^Microsoft.URL [OR]
RewriteCond %{HTTP_USER_AGENT} ^MIDown\ tool [OR]
RewriteCond %{HTTP_USER_AGENT} ^MIIxpc [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mister\ PiX [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mister.PiX [OR]
RewriteCond %{HTTP_USER_AGENT} ^moget [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mozilla.*NEWT [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/2 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/3.Mozilla/2.01 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Navroad [OR]
RewriteCond %{HTTP_USER_AGENT} ^NearSite [OR]
RewriteCond %{HTTP_USER_AGENT} ^Net\ Vampire [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetAnts [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetMechanic [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetZIP [OR]
RewriteCond %{HTTP_USER_AGENT} ^NICErsPRO [OR]
RewriteCond %{HTTP_USER_AGENT} ^NPBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^Octopus [OR]
RewriteCond %{HTTP_USER_AGENT} ^Openfind [OR]
RewriteCond %{HTTP_USER_AGENT} ^PageGrabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^Papa\ Foto [OR]
RewriteCond %{HTTP_USER_AGENT} ^pavuk [OR]
RewriteCond %{HTTP_USER_AGENT} ^pcBrowser [OR]
RewriteCond %{HTTP_USER_AGENT} ^ProPowerBot/2.14 [OR]
RewriteCond %{HTTP_USER_AGENT} ^ProWebWalker [OR]
RewriteCond %{HTTP_USER_AGENT} ^ProWebWalker [OR]
RewriteCond %{HTTP_USER_AGENT} ^QueryN.Metasearch [OR]
RewriteCond %{HTTP_USER_AGENT} ^ReGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^RepoMonkey [OR]
RewriteCond %{HTTP_USER_AGENT} ^RMA [OR]
RewriteCond %{HTTP_USER_AGENT} ^SiteSnagger [OR]
RewriteCond %{HTTP_USER_AGENT} ^SlySearch [OR]
RewriteCond %{HTTP_USER_AGENT} ^SmartDownload [OR]
RewriteCond %{HTTP_USER_AGENT} ^SpankBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^spanner [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Surfbot [OR]
RewriteCond %{HTTP_USER_AGENT} ^suzuran [OR]
RewriteCond %{HTTP_USER_AGENT} ^tAkeOut [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro [OR]
RewriteCond %{HTTP_USER_AGENT} ^Telesoft [OR]
RewriteCond %{HTTP_USER_AGENT} ^The.Intraformant [OR]
RewriteCond %{HTTP_USER_AGENT} ^TheNomad [OR]
RewriteCond %{HTTP_USER_AGENT} ^TightTwatBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^Titan [OR]
RewriteCond %{HTTP_USER_AGENT} ^toCrawl/UrlDispatcher [OR]
RewriteCond %{HTTP_USER_AGENT} ^True_Robot [OR]
RewriteCond %{HTTP_USER_AGENT} ^turingos [OR]
RewriteCond %{HTTP_USER_AGENT} ^TurnitinBot/1.5 [OR]
RewriteCond %{HTTP_USER_AGENT} ^URLy.Warning [OR]
RewriteCond %{HTTP_USER_AGENT} ^VCI [OR]
RewriteCond %{HTTP_USER_AGENT} ^VoidEYE [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\ Image\ Collector [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\ Sucker [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web.Image.Collector [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebAuto [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebBandit [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebEMailExtrac.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebEnhancer [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebFetch [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebGo\ IS [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebLeacher [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebmasterWorldForumBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebSauger [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website\ eXtractor [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website\ Quester [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website.Quester [OR]
RewriteCond %{HTTP_USER_AGENT} ^Webster.Pro [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebWhacker [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZip [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget [OR]
RewriteCond %{HTTP_USER_AGENT} ^Widow [OR]
RewriteCond %{HTTP_USER_AGENT} ^WWW-Collector-E [OR]
RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE [OR]
RewriteCond %{HTTP_USER_AGENT} ^Xaldon\ WebSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Xenu's [OR]
RewriteCond %{HTTP_USER_AGENT} ^Zeus

Danke für Eure Hilfe.

Tschau Easy

Beitrag von **thetrasher** » 14.04.2006, 19:14

User-Agent: Contentklau/1.0

Die User-Agent-Angabe ist äußerst leicht modifizierbar. Sobald ein Spammer/Kopierer seine Botkennung auf Deiner hier veröffentlichten Liste findet, wird er die Zeichenkette ändern. "Gute" Contentdiebe kommen heutzutage mit einer "echten" Browserkennung.

Dein Ansatz hilft nur gegen Anfänger, ist aber (leider) sinnlos.

Beitrag von **easycool** » 14.04.2006, 19:18

Also kann man nur zusehen oder gibt es Möglichkeiten dagegen?

Blockt mir den diese Liste wenigsten nicht die Guten Bots.

Beitrag von **robo** » 14.04.2006, 19:39

Du könntest prüfen, ob die vom Client gesendeten HTTP-Header zum User-Agent passen ...

cu, Robo