Bots vs Browsers - database of 6,036,537 user agents and growing
Browsing Category "Nutch Bots"
Variations of the Nutch bot User Agents
658 User Agents Found.
"Mozilla/31.0"/Nutch-1.8
"Mozilla/4.0"/Nutch-1.19-SNAPSHOT
*/Nutch-0.9
*/Nutch-0.9-dev
abc/Nutch-0.9-dev (abc; http://abc#11.us; abc at abc dot com)
abond/Nutch-1.9
Abortion.sg/Nutch-1.1 (www.Abortion.sg; crawler@Abortion.sg)
ACME Corporation/Nutch-1.0 (ACME Spider; http://www.acme.com; test123@spam.la)
Acorn/Nutch-0.9 (Non-Profit Search Engine; acorn.isara.org; acorn at isara dot org)
Acorn/Nutch-0.9 (Non-Profit Search Engine; acorn.isara.org; acorn at isara dot org)1194220892509
Acorn/Nutch-0.9 (Non-Profit Search Engine; acorn.isara.org; acorn at isara dot org)1194260582878
Affectv Robot v1.0/Nutch-1.6 (http://www.affectv.co.uk; austin at affectv dot co dot uk)
Affectv Robot v1.0/Nutch-2.1
africa/Nutch-0.9 (africa; localhost; localhost)
agentname/Nutch-1.0-dev
Aghaven/Nutch-1.2
Aghaven/Nutch-1.2 (www.aghaven.com)
aisou/Nutch-1.8 (aisou; 21.156.80.131)
AJCrawler/Nutch-1.7
AlexionResearchBot/Nutch-1.3
amd-source-bot/Nutch-1.0
Amit Singh/Nutch-0.9 (Amit Singh; www.cse.iitb.ac.in/~amitsingh; amitsingh@gmail.com)
ant.com/Nutch-1.7 (http://ant.com)
Ant/Ant-Nutch-1.1 (Ant Nutch Crawler; http://www.ant.com; crawler@ant.com)
AntBot/Ant-Nutch-1.1 (Ant Nutch Crawler; http://www.ant.com; crawler@ant.com)
AOL_Daniel_Clark_Spider/Nutch-0.9 (AOL Search; danielaclark1@aol.com)
Apache Sites Search Facet - Big Data Drupal/Nutch-1.6
apache.org/Nutch-0.9 (apache; http://www.apache.org; users@apache.org)
apache.org/Nutch-0.9 (apache; http://www.apache.org; users@apache.org)1610718322060
apache.org/Nutch-0.9 (apache; http://www.apache.org; users@apache.org)2114384368396
Applied-Technologies-Inc-Spider/Nutch-1.4
aramabeta.com - search engine beta version/Nutch-2.3-SNAPSHOT
arjun/Nutch-1.4
AskAboutOil/0.06-rcp (Nutch; http://www.nutch.org/docs/en/bot.html; nutch-agent@askaboutoil.com)
asked/Nutch-0.8 (web crawler; http://asked.jp; epicurus at gmail dot com)
Attentio/Nutch-0.9-dev (Attentio's beta blog crawler; www.attentio.com; info@attentio.com)
Attributor/Nutch-1.0-dev (Test crawler; http://www.attributor.com; info at attributor com)
avi/Nutch-1.7
Ayna/Nutch-0.9 (Ayna Search Engine Crawler; http://www.ayna.com/; search at aynacorp dot com)
baidu/Nutch-1.0
Balihoo/Nutch-1.0-dev (Crawler for Balihoo.com search engine - obeys robots.txt and robots meta tags ; http://balihoo.com/index.aspx; robot at balihoo dot com)
bd/Nutch-1.7
beambot/Nutch-1.14-SNAPSHOT (db data collector; info at address to follow dot com)
beast/Nutch-0.9 (agentspider; beast@mail.com)
becomex /Nutch-0.9
becomex /Nutch-1.0
bender/Nutch-0.8.1 (myd@cs.stanford.edu)
betasearch/Nutch-2.1 (Academic Beta Search; http://www.aramabeta.com; info@aramabeta.com)
betasearch/Nutch-2.2.1 (Academic Beta Search; http://www.aramabeta.com; info@aramabeta.com)
Bigsearch.ca/Nutch-0.9-dev (Bigsearch.ca Internet Spider; http://www.bigsearch.ca/; info@enhancededge.com)
Bigsearch.ca/Nutch-1.0-dev (Bigsearch.ca Internet Spider; http://www.bigsearch.ca/; info@enhancededge.com)
Bigsearch.ca/Nutch-x.x-dev (Bigsearch.ca Internet Spider; http://www.bigsearch.ca/; info@enhancededge.com)
BilgiBetaBot/0.8-dev (bilgi.com (Beta) ; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org)
blackcrawl/Nutch-0.9 (crawl for fun; www.zipfelchappe.com; zipfelchappe@localhost)
blah/Nutch-1.9
bldbCrawler/Nutch-1.4
blender.cs.qc.cuny.edu Spider (research purposes only)/Nutch-1.2 (This spider intends to collect individual webpages (not whole websites) for research in Natural Language Processing. Please contact us
Bloodhound/Nutch-0.9 (Testing Crawler for Research - obeys robots.txt and robots meta tags ; http://balihoo.com/index.aspx; robot at balihoo dot com)
Boardreader.com Test Crawl/Nutch-1.9
bob/Nutch-0.9 (bob; http://www.google.com; x@y)
BobCrawl/Nutch-0.9 (Test/Development crawler; http://notavalable.com; notavailable@notavailable.com)
boo/Nutch-1.0
boo/Nutch-1.0 (boo)
boosker/Nutch-1.2
botrobin/Nutch-1.13-SNAPSHOT (http://smarter.codes/bot-robin/; botrobin@smarter.codes)
BpeerSpider/Nutch-1.9
Cabot/Nutch-0.9 (Amfibi's web-crawling robot; http://www.amfibi.com/cabot/; agent@amfibi.com)
Cabot/Nutch-1.0-dev (Amfibi's web-crawling robot; http://www.amfibi.com/cabot/; agent@amfibi.com)
Cabot/Nutch-1.0-dev (Amfibi's web-crawling robot; http://www.amfibi.com/cabot/; help at amfibi dot com)
Cabot/Nutch-1.2 (Amfibi's webcrawler robot; http://www.amfibi.com/cabot; cabot@amfibi.com)
caizhi/infomation/Nutch-0.8.1
cancho/Nutch-1.0 (crawl test; http://asdf.net/; asdf@asdf.net)
Cazoodle/Nutch-0.9-dev
Cazoodle/Nutch-0.9-dev (Cazoodle Nutch Crawler; http://www.cazoodle.com; mqbot@cazoodle.com)
CazoodleBot/Nutch-0.9-dev (CazoodleBot Crawler; http://www.cazoodle.com/cazoodlebot; cazoodlebot@cazoodle.com)
CazoodleBot/Nutch-0.9-dev (CazoodleBot Crawler; http://www.cazoodle.com; mqbot@cazoodle.com)
CB/Nutch-1.11
CB/Nutch-1.7
CCResearchBot/1.0 commoncrawl.org/research//Nutch-1.7-SNAPSHOT
Chen Li/Nutch-1.0 (Nutch spiderman; http://chenli.com.cn; chenlibiti@163.com)
Chickety China (the Chinese Chicken) / Nutch-0.9
Chou Xeon Spider/Nutch-1.10
Chrome/44.0.2403.155/Nutch-2.4-SNAPSHOT
cierzo-development/Nutch-1.1-dev
CjxSearch/Nutch-1.4
clark-crawler2/Nutch-1.19-SNAPSHOT
CloudACL/Nutch-1.4
COMODOspider/Nutch-1.0
COMODOSpider/Nutch-1.2
ComodoSpider/Nutch-2.2.1
Companyspot/Nutch-1.8 (Companyspot spider; http://www.companyspot.co.uk/)
complex_network_group/Nutch-0.9-dev (discovering the structure of the world-wide-web; http://cantor.ee.ucla.edu/~networks/crawl; nimakhaj@gmail.com)
Comrite/0.7.1 (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org)
CorporateNewsSearchEngine/Nutch-1.7 (http://pibs.co/news-search-engine)
coruscan/Nutch-1.4
Covario Amazon Nutch Crawler/Nutch-1.2
Covario IDS/Nutch-1.2
crawl test/Nutch-1.0-dev
crawler/Nutch-1.7
crawler/Nutch-1.9
crawling by taiil/Nutch-1.8
Crawling for Cows/Nutch-1.6
CreativeCommons/0.06-dev (Nutch; http://www.nutch.org/docs/en/bot.html; nutch-agent@lists.sourceforge.net)
CRIM Crawler/Nutch-2.3 (Crawler du Centre de Recherche Informatique de Montréal (CRIM))
cronquest/Nutch-2.2 (cronquest; http://cronquest.com; info at cronquest dot com)
DataPatrol/Nutch-1.0 (DataPatrol indexer from Garlik; http://www.garlik.com/products.php; crawler at garlik dot com)
DERIbot/Nutch-1.0-dev (DERIbot; http://deri.org ; info@deri.ie)
disco/Nutch-0.9 (experimental crawler ... please email imagine@gmail.com if problems observed; imagine@gmail.com)
disco/Nutch-0.9 (experimental crawler ... please email imagine@gmail.com if problems observed; nedrocks@gmail.com)
disco/Nutch-0.9 (experimental crawler; nedrocks@gmail.com)
disco/Nutch-0.9 (experimental crawler; www.discoveryengine.com; disco-crawl@discoveryengine.com)
disco/Nutch-1.0-dev (experimental crawler; www.discoveryengine.com/robot.html; disco-crawl@discoveryengine.com)
disco/Nutch-1.0-dev (experimental crawler; www.discoveryengine.com; disco-crawl@discoveryengine.com)
DiscoverEd/Nutch-1.7 (OER search crawler; http://wiki.creativecommons.org/DiscoverEd; webmaster@creativecommons.org)
dmis_crawler/Nutch-1.4
dnt Nutch Spider/20100101 Firefox/27.0
Domnutch-Bot/Nutch-1.0 (Domnutch; http://www.Nutch.de/)
dpdev/Nutch-1.0 (datapatrol from garlik.com; http://www.garlik.com/crawler; crawler at garlik dot com)
dpdev/Nutch-1.0 (datapatrol from garlik.com; http://www.garlik.com/products.php; crawler at garlik dot com)
dpdev/Nutch-1.0-dev (datapatrol from garlik.com; http://www.garlik.com/crawler; crawler at garlik dot com)
ealbum/Nutch-1.0
easy crawl/Nutch-1.10
ecxi/Nutch-1.0 (esCERT-UPC-ecxi; http://escert.upc.edu/; admin escert edu)
ecxi/Nutch-1.0-dev (esCERT-UPC-ecxi; http://escert.upc.edu/; admin escert edu)
education portal/Nutch-0.9 (Please do not forbid, its for your benefit)
EIC Nutch Spider/Nutch-1.7 (EIC bot agent; http://www.eic.com; devmaster@eic.com)
Element/Nutch-2.0-dev
enlle punto com/Nutch-1.9
Eric Osgood/Nutch-1.0 (Nutch spiderman; http://www.calpoly.edu/~eosgood ; MyEmail)
Eurobot/Nutch-1.0-dev (1.0)
ExactSeek Crawler (http://www.exactseek.com/)/Nutch-1.4
ExactSeek Crawler (nutch 1.4)/Nutch-1.4
ExactSeek Crawler (nutch 1.4)/Nutch-1.4 (ExactSeek Crawler; http://www.exactseek.com)
Facet Engine Spider/Nutch-1.2 (Internet Crawler; spider@facetengine.com)
fetch/Nutch-1.0 (TCGfetch; http://fetch.thecyberguardian.com; TCGEmail)
Filangy/0.01-beta (Filangy; http://www.nutch.org/docs/en/bot.html; filangy-agent@filangy.com)
Filangy/1.0x (Filangy; http://www.nutch.org/docs/en/bot.html; filangy-agent@filangy.com)
foobar/Nutch-1.0-dev (foobar; foobar.com; foo@bar.com)
FreeNutch/Nutch-1.2
fujilabolx1 Spider/Nutch-1.10
fujilabolx2 Spider/Nutch-1.10
fujilabolx4 Spider/Nutch-1.10
fujilabolx5 Spider/Nutch-1.10
GentilBot/Nutch-1.0
GeoHasher/Nutch-1.0 (GeoHasher Web Search Engine; geohasher.gotdns.org; geo_hasher at yahoo * com)
gh-index-bot/Nutch-1.0 (GH Web Search.; lucene.apache.org; gh_email at someplace dot com)
Gimme60/Nutch-1.8 (page refresh: please visit gimme60.com; gimme60.com; support _AT__SIGN_ gimmie60 _DOT_COM_)
Googlebot/Nutch-1.0
googlepages/Nutch-0.9 (googlepages; http://www.googlepages.com; info@googlepages.com)
graydonCrawler/Nutch-1.9 (Graydon crawler, for testing purposes only; info@graydon.nl)
guoming/Nutch-1.6
HD nutch agent/Nutch-1.1 (Think)
healia/Nutch-0.9 (the personalized health search engine.; http://www.healia.com; mikes@healia.com)
HealRWorld/Nutch-1.10
Heeii/Nutch-0.8.1 (Heeii; www.heeii.com; info@heeii.com)
Heeii/Nutch-0.9 (Heeii; www.heeii.com; info@heeii.com)
hiva/Nutch-2.0-dev
HouxouCrawler/Nutch-0.8.2-dev (houxou.com's nutch-based crawler which serves special interest on-line communities; http://www.houxou.com/crawler; crawler at houxou dot com)
HouxouCrawler/Nutch-0.9 (houxou.com's nutch-based crawler which serves special interest on-line communities; http://www.houxou.com/crawler; crawler at houxou dot com)
HPI-BI-Crawler/0.1(+http://www.hpi.uni-potsdam.de/meinel/forschung/web_30/blog_intelligence.html) /Nutch-2.0-dev
HPI-BI-Crawler/0.1(+http://www.hpi.uni-potsdam.de/meinel/forschung/web_30/blog_intelligence.html) /Nutch-2.0-dev
HPI-BI-Crawler/0.1(+http://www.hpi.uni-potsdam.de/meinel/forschung/web_30/blog_intelligence.html)/Nutch-2.0-dev
HPL/Nutch-0.9
HTML Analyzer/Nutch-1.12
http://www.163.com/Nutch-1.0 (http://www.163.com)
hunter/Nutch-0.8.1
Iceweasel2.0.0.16/Nutch-1.0 (Webbrowser; http://iceweasel.com; info@iceweasel.com)
Iceweasel2.0.0.16/Nutch-1.1 (Webbrowser; http://iceweasel.com; info@iceweasel.com)
iCrawl/Nutch-1.15
IgnitionOneBot/Nutch-1.9 ( This is the IgnitionOne Company Bot for Web Crawling. IgnitionOne Company Site: http://www.ignitionone.com/ ; rongyao dot huang at ignitionone dot com )
IIT Bombay CFILT NLP Bot/Nutch-1.1 (IITB CFILT Crawler)
IITB-CFILT-Bot/Nutch-1.1 (This is the crawler of IIT Bombay, India. The data will be used for research purposes.; http://www.cfilt.iitb.ac.in/; pb@cse.iitb.ac.in)
ilial/Nutch-0.9 (Ilial, Inc. is a Los Angeles based Internet startup company. For more information please visit http://www.ilial.com/crawler; http://www.ilial.com/crawler; crawl@ilial.com)
ilial/Nutch-0.9 (Ilial, Inc. is a Los Angeles based Internet startup company.; http://www.ilial.com/crawler; crawl@ilial.com)
ilial/Nutch-0.9-dev
InboundScore/Nutch-1.4 (http://InboundScore.com/)
Infoaxe./Nutch-0.9
Infoaxe./Nutch-1.0
informatics/Nutch-1.2
Innovazion Crawler/Nutch-1.7
innoventage/Nutch-1.0 (poc; www.google.com; proof of concept)
innoventage/Nutch-1.0 (poc; www.innoventage.com; proof of concept)
Insideview/Nutch-1.13-SNAPSHOT
InsideView/Nutch-1.5.1
InternetArchive/0.8-dev (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org)
intumit/Nutch-1.1
intumit/Nutch-1.2
intumit/Nutch-1.2 (intumit)
intumit/Nutch-1.3
intumit/Nutch-1.4
IRP_edu_bot/Nutch-0.9
IS Alpha/Nutch-1.0
IS Alpha/Nutch-1.1
istellabot-nutch/Nutch-1.10
istellabot/Nutch-1.10
istellabot/Nutch-1.11
johnhew crawler, johnhew@seas.upenn.edu/Nutch-1.12
jupiter/Nutch-1.2
Kavande Crawler 1.0/Nutch-1.4 ( Iranian National Web Crawler ; kh3rad@gmail.com)
KeywordSearchTool.co/Nutch-1.4 (http://KeywordSearchTool.co/robot)
kindsight/Nutch-1.0 (kscrawler; www.projectrialto.com; crawler@projectrialto.com)
Kiodia Spider/Nutch-1.11
KnowItAll/Nutch-0.9 (Nutch-UW-Crawler; http://cs.washington.edu/homes/mjc/crawler.html; uwcrawler08@gmail.com)
Kraken/Nutch-2.2.1 (Nutch crawler launched by Integral Ad Science, Inc.; TBD; TBD)
Krugle/Krugle,Nutch/0.8+ (Krugle web crawler; http://corp.krugle.com/crawler/info.html; webcrawler@krugle.com)
Krugle/Krugle,Nutch/0.8+ (Krugle web crawler; http://www.krugle.com/crawler/info.html; webcrawler@krugle.com)
KS Crawler/Nutch-1.0 (http://www.kindsight.net/kscrawler; crawler@kindsight.net)
KS Spider/Nutch-0.9
KSCrawler/Nutch-1.0 (http://www.kindsight.net/en/kscrawler; crawler@kindsight.net)
Kusiri/Nutch-2.2.1
LawSolver1/Nutch-1.9
lewismc/Nutch-2.3-SNAPSHOT (Nightly crawl for integration testing of Nutch 2.3-SNAPSHOT, Gora 0.3 and Cassandra 1.1.2; http://nutch.apache.org; lewismc@apache.org)
LijitSpider/Nutch-0.9 (Reports crawler; http://www.lijit.com/; info(a)lijit(d)com)
LijitSpider/Nutch-0.9 (Reports crawler; http://www.lijit.com/robot/crawler; info(a)lijit(d)com)
linguatools-bot/Nutch-1.6 (searching for translated pages; http://www.linguatools.de/linguatoolsbot.html; peter dot kolb at linguatools dot org)
linkdexbot/Nutch-1.0-dev (http://www.linkdex.com/; crawl at linkdex dot com)
Lisboa/Nutch-1.2
LiveYellow/1.0/Nutch-1.2
lmspider/Nutch-0.9-dev (For research purposes.; www.nuance.com; lmspider@nuance.com)
LPbot/Nutch-1.1
LpLinkCheck/Nutch-1.12 (Sends you traffic; http://www.linkpendium.com/)
LTI/LemurProject Nutch Spider/Nutch-1.0-dev (lti crawler for CMU; http://www.lti.cs.cmu.edu; changkuk at cmu dot edu)
LTI/LemurProject Nutch Spider/Nutch-1.0-dev (Research spider using Nutch; http://lucene.apache.org/nutch/bot.html; admin@lemurproject.org)
LTI/LemurProject Nutch Spider/Nutch-1.0-dev (Research spider using Nutch; http://www.lemurproject.org; mhoy@cs.cmu.edu)
LWNutch/Nutch-1.4 (another scientific bot - we accept your robots.txt! )
LWNutch/Nutch-1.4 (another scientific bot - we check your robots.txt! ; contact by mail: abuse _at- languageweaver.com)
ly/1.0/Nutch-1.2
Lynx Spider/Nutch-1.7
male.com.sg/Nutch-1.1 (http://www.male.com.sg; crawler@male.com.sg)
maluuba-crawler/Nutch-1.6
Manav/Nutch-0.9 (1.0; manavraman at yahoo dot com)
Manav/Nutch-1.0-dev (1.0; manavraman at yahoo dot com)
Martin/Nutch-1.0 (Nutch spiderman; MyEmail)
MaxPointCrawler/Nutch-1.1
MaxPointCrawler/Nutch-1.1 (maxpoint.crawler at maxpointinteractive dot com)
MaxPointCrawler/Nutch-1.1 (MaxPoint.Crawler@maxpointinteractive.com)
MaxPointCrawler/Nutch-1.10 (maxpoint.crawler at maxpointinteractive dot com)
MaxPointCrawler/Nutch-1.14 (maxpoint.crawler at maxpointinteractive dot com)
MaxPointCrawler/Nutch-1.17 (maxpoint.crawler at maxpointinteractive dot com)
MaxPointCrawler/Nutch-1.17 (valassis.crawler at valassis dot com)
MaxPointCrawler/Nutch-1.6 (maxpoint.crawler at maxpointinteractive dot com)
mercury/Nutch-1.2
Misterbot-Nutch/0.7.1 (Misterbot-Nutch; http://www.misterbot.fr; admin@misterbot.fr)
mmcrawler/Nutch-1.0 (MM Robots; http://; lindaoi1@hotmail.com)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)/Nutch-1.0
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)/Nutch-1.0-dev
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; en) Opera 8.01/Nutch-0.8.1 (http://lucene.apache.org/nutch/about.html; http://lucene.apache.org/nutch/bot.html; mail@dev.null)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; en) Opera 8.01/Nutch-0.9 (http://lucene.apache.org/nutch/about.html; http://lucene.apache.org/nutch/bot.html; mail@dev.null)
Mozilla/4.0 (compatible; MSIE 6.1; Windows XP; .NET CLR 1.1.4322; .NET CLR 2.0.50727)/Nutch-1.3
Mozilla/4.0 (compatible; MSIE 6.1; Windows XP; .NET CLR 1.1.4322; .NET CLR 2.0.50727)/Nutch-1.5
Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0)/Nutch-1.7
Mozilla/4.0/Nutch-0.9
Mozilla/4.0/Nutch-1.0-dev (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)
Mozilla/5.0 (Android; Mobile; rv:21.0) Gecko/21.0 Firefox/21.0 commoncrawl.org/research//Nutch-1.7-SNAPSHOT
Mozilla/5.0 (compatible; ADIR Research Project; http://bradipo.net/mark) /Nutch-0.9
Mozilla/5.0 (compatible; Advisorbot/1.0)/Nutch-1.2
Mozilla/5.0 (compatible; MJ12bot/v1.4.7; http://www.majestic12.co.uk/bot.php?+)/Nutch-1.13
Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0) commoncrawl.org/research//Nutch-1.7-SNAPSHOT
Mozilla/5.0 (compatible; OpenindexDeepSpider/Nutch-1.5-dev; +http://openindex.io/spider.html; systemsATopenindexDOTio)
Mozilla/5.0 (compatible; OpenindexDeepSpider/Nutch-1.5-dev; +http://www.openindex.io/en/webmasters/spider.html)
Mozilla/5.0 (compatible; OpenindexDeepSpider/Nutch-1.5-dev; +http://www.openindex.io/en/webmasters/spider.html; systemsATopenindexDOTio)
Mozilla/5.0 (compatible; OpenindexShallowSpider/Nutch-1.5-dev; +http://openindex.io/spider.html; systemsATopenindexDOTio)
Mozilla/5.0 (compatible; OpenindexShallowSpider/Nutch-1.5-dev; +http://www.openindex.io/en/webmasters/spider.html)
Mozilla/5.0 (compatible; OpenindexShallowSpider/Nutch-1.5-dev; +http://www.openindex.io/en/webmasters/spider.html; systemsATopenindexDOTio)
Mozilla/5.0 (compatible; OpenindexSpider/Nutch-1.5-dev; +http://openindex.io/spider.html; systemsATopenindexDOTio)
Mozilla/5.0 (compatible; OpenindexSpider/Nutch-1.5-dev; +http://www.openindex.io/en/webmasters/spider.html)
Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_0 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8A293 Safari/6531.22.7 commoncrawl.org/research//Nutch-1.7-SNAPSHOT
Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_0 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8A293 Safari/6531.22.7/Nutch-1.0
Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_0 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8A293 Safari/6531.22.7/Nutch-1.3
Mozilla/5.0 (Linux; Android 4.4.2; SMART_3.5_BY_NUTCHA Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36
Mozilla/5.0 (Linux; U; Android 4.0; en-us; Tuna Build/IFK77E) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.67 commoncrawl.org/research//Nutch-1.7-SNAPSHOT
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36/Nutch-1.13
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.61 Safari/537.36/Nutch-1.18
Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.5; en-US; rv:1.9.1.9) Gecko/20100315 Firefox/3.5.9/Nutch-1.0
Mozilla/5.0 (Mobile; rv:18.0) Gecko/18.0 Firefox/18.0 commoncrawl.org/research//Nutch-1.7-SNAPSHOT
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36/Nutch-1.13
Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.125 Safari/537.36/Nutch-1.6
Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36 QIHU 360SE/Nutch-1.13
Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36/Nutch-2.3
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1 /Nutch-1.2
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1/Nutch-1.2
Mozilla/5.0 (Windows; N; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727)/Nutch-1.0 (Crawler; lucene.apache.org/nutch/; a@b.net)
Mozilla/5.0 (Windows; N; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727)/Nutch-1.1 (Crawler; lucene.apache.org/nutch/; a@b.net)
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4/Nutch-0.9
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.9/Nutch-0.9
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.224 Safari/534.10/Nutch-1.2
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0/Nutch-0.9
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.36 (KHTML, like Gecko) Chrome/13.0.766.0 Safari/534.36/Nutch-1.4 (yanjunshi@comodo.com)
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.75 Safari/535.7/Nutch-1.4
Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20100101 Firefox/10.0/Nutch-1.9
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.12) Gecko/20080207 Ubuntu/7.10 (gutsy) Firefox/2.0.0.12/Nutch-0.9
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.19) Gecko/20081202 Firefox (Debian-2.0.0.19-0etch1)/Nutch-1.0
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.19) Gecko/20081202 Firefox (Debian-2.0.0.19-0etch1)/Nutch-1.0 UNTRUSTED/1.0
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.12) Gecko/2009070811 Ubuntu/9.04 (jaunty) Firefox/3.0.12/Nutch-1.0-dev (imcs; http://imcs.ro; admin@imcs.ro)
Mozilla/5.0 (X11; U; OpenBSD i386; en-US; rv:1.9.2.8) Gecko/20101230 Firefox/3.6.8/Nutch-1.0
Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:17.0) Gecko/20100101 Firefox/17.0/Nutch-1.4
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:16.0) Gecko/20100101 Firefox/16.0/Nutch-1.7
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:55.0) Gecko/20100101 Firefox/55.0/Nutch-1.12
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.95 Safari/537.36/Nutch-1.13
Mozilla/5.0+(compatible;)/Nutch-1.6
Mozilla/5.0+(compatible;+PiplBot;+http://www.pipl.com/bot/)/Nutch-1.14-SNAPSHOT
Mozilla/5.0/nutch (contactbigdatafr at gmail.com)
Mozilla/5.0/Nutch-1.1 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13)
Mozilla/Nutch-1.0
Mozilla/Nutch-1.1
Mozilla/Nutch-1.2 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.11)
Mozilla/Nutch-1.5.1 (Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20100101 Firefox/15.0)
Mozilla5.0/Nutch-1.6
MQBot/Nutch-0.8-dev (mqbot@cazoodle.com)
MQBOT/Nutch-0.9-dev (MQBOT Crawler; http://falcon.cs.uiuc.edu; mqbot@cs.uiuc.edu)
MQBOT/Nutch-0.9-dev (MQBOT Nutch Crawler; http://falcon.cs.uiuc.edu; mqbot@cs.uiuc.edu)
MQBOT/Nutch-0.9-dev (MQBOT Nutch Crawler; http://vwbot.cs.uiuc.edu; mqbot@cs.uiuc.edu)
MULTIPRISE/Nutch-1.0 (Robot d'indexation; http://www.multiprise.biz; admin@multiprise.fr)
MXT/Nutch-1.10
MXT/Nutch-1.10 (http://t.co/GSRLLKex24; informatique at mixdata dot com)
MXT/Nutch-1.12-SNAPSHOT (http://t.co/GSRLLKex24; informatique at mixdata dot com)
My crawler /Nutch-1.11
my crawler/Nutch-1.6
My Nutch Spider (www.yurichev.com; dennis at yurichev dot com)
My Nutch Spider/Nutch-1.10
My Nutch Spider/Nutch-1.11
My Nutch Spider/Nutch-1.12
My Nutch Spider/Nutch-1.14
My Nutch Spider/Nutch-1.15
My Nutch Spider/Nutch-1.18
My Nutch Spider/Nutch-1.3
My Nutch Spider/Nutch-1.4
My Nutch Spider/Nutch-1.5
My Nutch Spider/Nutch-1.5-SNAPSHOT
My Nutch Spider/Nutch-1.5.1
My Nutch Spider/Nutch-1.6
My Nutch Spider/Nutch-1.7
My Nutch Spider/Nutch-1.7 (http://wiki.creativecommons.org/DiscoverEd)
My Nutch Spider/Nutch-1.8
My Nutch Spider/Nutch-1.9
My Nutch Spider/Nutch-2.2.1
My Nutch Spider/Nutch-2.3.1
My Spider/Nutch-1.0 (My Bot; http://www.intersect.org.au; sridhar.reddapani@intersect.org.au)
my/Nutch-2.1
myagent/Nutch-1.7
mybot/Nutch-1.0 (mybot; http://mybot.com; mybot@mybot.com)
MyCrawl001/Nutch-1.4
myfirsttest/Nutch-0.8.1 (myfirsttest; http://www.science.uva.nl/; xzhang1@science.uva.nl)
myNutch/Nutch-1.2
mynutchcrawler/0.8.1 (nutch 0.8.1; http://localhost:8080; info at mysite dot com)
MyNutchSpider/Nutch-1.7
MyNutchSpider/Nutch-2.1
MyNutchSpider/Nutch-2.2.1
MyNutchTest/Nutch-1.6
MyNutchTest/Nutch-1.7
myse/Nutch-1.11
Mysearch/Nutch-0.9
Mysite/Nutch-2.0
Mysite/Nutch-2.2.1
Neo Lee/Nutch-0.9 (Nutch spiderman; http://lucene.apache.org/nutch/; MyEmail)
Netluchs/Nutch-1.0 ( ; http://www.netluchs.de/; _do_not_spam_me___humans_please_use_info_at_netluchs.de_without_the_dash)
Netluchs/Nutch-1.0-dev ( ; http://www.netluchs.de/; _do_not_spam_me___humans_please_use_info_at_netluchs.de_without_the_dash)
NetSeer/Nutch-0.9 (NetSeer Crawler; http://www.netseer.com; crawler@netseer.com)
NexiSpider/Nutch-1.5.1
NIS Nutch Spider/Nutch-1.7
noopsis Spider/Nutch-1.1 (noopsis crawler)
NRLCorpusBuilder/Nutch-1.9
NSE/Nutch-1.2
nsyght.com/Nutch-0.9 (nsyght.com; Nsyght.com)
nsyght.com/Nutch-0.9 (nsyght.com; search.nsyght.com)
nsyght.com/Nutch-1.0-dev (nsyght.com; Nsyght.com)
nutch
Nutch 1.2/Nutch-1.2 (Facet Engine Nutch Crawler; spider@facetengine.com)
Nutch agent name/Nutch-1.0 (Nutch agent description; http:// MyAgent.googlepages.com ; MyEmail)
Nutch Crawler QH/Nutch-1.7
Nutch crawler/Nutch-0.9 (picapage.com; admin@picapage.com)
Nutch Crawler/Nutch-2.4-SNAPSHOT
Nutch Experimental Crawler/Nutch-1.4
Nutch Experimental Crawler/Nutch-2.0-dev
Nutch Master Test/Dolphin-0.1-Beta
Nutch Master Test/Nutch-1.13-SNAPSHOT
Nutch Spider/Nutch-1.4
Nutch Spider/Nutch-1.5
Nutch Spider/Nutch-1.6
Nutch Spider/Nutch-2.2.1
Nutch test crawler/Nutch-1.4-dev (Crawler testing; crawler/a/t/luminouslabs.com)
nutch test/Nutch-1.0 (nutch test)
nutch-1.3/Nutch-1.3
nutch-1.4/Nutch-1.3
nutch-1.4/Nutch-1.4
Nutch-1.4/Nutch-1.4 (shirdrn@gmail.com)
nutch-1.8/Nutch-1.8
nutch-2.3.1-crawler/Nutch-2.3.1
nutch-crawl/Nutch-1.0-dev (imcs; http://imcs.ro; admin@imcs.ro)
nutch-crawler/Nutch-0.9
nutch-crawler/Nutch-1.2
nutch-solr-integration-test/Nutch-1.2 (MoonValley Web Crawler using Nutch 1.2; http://www.moonvalley.com/; cwoolum@moonvalley.com)
nutch-solr-integration/1.1
nutch-solr-integration/Nutch-1.0
nutch-solr-integration/Nutch-1.3
nutch-solr-integration/Nutch-1.4
nutch-solr/Nutch-1.0-dev
nutch-spider-2.2.1/Nutch-1.12-SNAPSHOT
nutch.biz/Nutch-1.0 (nutch.biz; crawler@nutch.biz)
nutch.us/Nutch-1.0 (nutch.us; crawler@nutch.us)
nutch.us/Nutch-1.0 (www.nutch.us; crawler@nutch.us)
nutch.us/Nutch-1.0-dev (www.nutch.us; crawler@nutch.us)
nutch/1.2 (nutch)
Nutch/1.2/Nutch-1.2
Nutch/2.2.1 (page scorer; http://integralads.com/site-indexing-policy/)
Nutch/Nutch-0.8.1
Nutch/Nutch-0.8.1 (Nutch; Nutch; Nutch)
Nutch/Nutch-0.9
Nutch/Nutch-0.9 (Eurobot; http://www.ayell.eu )
nutch/Nutch-0.9 (nutch)
Nutch/Nutch-0.9 (Nutch; http://lucene.apache.org/nutch/)
Nutch/Nutch-0.9 (Nutch; http://nutch; nutch)
Nutch/Nutch-1.0 (academic purpose; cats.kaist.ac.kr; smiler82@naver.com)
nutch/Nutch-1.0 (nutch)
Nutch/Nutch-1.0-dev (A Nutch-based crawler.; http://lucene.apache.org/nutch/bot.html; nutch-agent AT lucene.apache.org)
nutch/Nutch-1.0-dev (nutch)
Nutch/Nutch-1.15
Nutch/Nutch-1.4 (A semantic crawler for a PhD thesis; http://issel.ee.auth.gr/doku.php/research; kvavliak at issel dot ee dot auth dot gr)
Nutch/Nutch-1.5
Nutch/Nutch-2.0
Nutch/Nutch-2.0 (Nutch Crawler)
Nutch/Nutch-2.1
nutch/Nutch-2.2
Nutch/Nutch-2.3.1
Nutch_Crawler/Nutch-1.3
nutch_princeton/Nutch-1.0-dev (princeton crawler for cass project; http://www.cs.princeton.edu/cass/; zhewang a_t cs ddot princeton dot edu)
Nutch_Spider/Nutch-1.15
nutch_test/Nutch-0.9 (nutch_test; http://www.8dorm.com; webmaster@8dorm.com)
Nutch0.8/Nutch-0.9-dev
nutch0.9/Nutch-0.9-dev
Nutch12/Nutch-1.2
nutch17Agent/Nutch-1.7
NUTCHCRAWLER/Nutch-0.9 (anouar@yatinoo.com)
NutchCrawler/Nutch-1.1
NutchCrawler/Nutch-1.6
NutchCrawler/Nutch-2.2.1
NutchCVS (Nutch; http://nutch.apache.org/bot.html; agent@nutch.apache.org)
NutchCVS/0.05 (Nutch; http://www.nutch.org/docs/en/bot.html; nutch-agent@lists.sourceforge.net)
NutchCVS/0.06-dev (Nutch running at UW; http://www.nutch.org/docs/en/bot.html; sycrawl@cs.washington.edu)
NutchCVS/0.06-dev (Nutch; http://www.nutch.org/docs/en/bot.html; nutch-agent@lists.sourceforge.net)
NutchCVS/0.0x-dev (Nutch; http://www.nutch.org/docs/bot.html; nutch-agent@lists.sourceforge.net)
NutchCVS/0.7 (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org)
NutchCVS/0.7.1 (Nutch running at UW; http://crawlers.cs.washington.edu/; sycrawl@cs.washington.edu)
NutchCVS/0.7.1 (Nutch running at UW; http://www.nutch.org/docs/en/bot.html; sycrawl@cs.washington.edu)
NutchCVS/0.7.1 (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org)
NutchCVS/0.7.1 (Nutch; http://lucene.apache.org/nutch/bot.html; raphael@unterreuth.de)
NutchCVS/0.7.2 (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org)
NutchCVS/0.7.2 (Nutch; http://lucene.apache.org/nutch/bot.html; west@cis.poly.edu)
NutchCVS/0.8-dev (Nutch running at UW; http://www.nutch.org/docs/en/bot.html; sycrawl@cs.washington.edu)
NutchCVS/0.8-dev (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org)
NutchCVS/0.8.1 (http://cis.poly.edu/westlab/; west@poly.edu)
nutchCVS/Nutch-0.8.1 (nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org)
NutchCVS/Nutch-1.0 (http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org)
NutchEC2Test/Nutch-0.9-dev (Testing Nutch on Amazon EC2.; http://lucene.apache.org/nutch/bot.html; ec2test at lucene.com)
NutchNASSV/Nutch-2.2.1
NutchOrg/0.0x-dev (Nutch; http://www.nutch.org/docs/bot.html; nutch-agent@lists.sourceforge.net)
nutchsearch/Nutch-0.9 (Nutch Search 1.0; herceg_novi at yahoo dot com)
NutchSearchEngineCrawler/Nutch-1.7
NutchSearchEngineCrawler/Nutch-2.2.1
NutchTest/Nutch-1.0
NutchVinegarCrawl/Nutch-0.8.1 (Vinegar; http://www.cs.washington.edu; eytanadar at gmail dot com)
nutchwax.com/Nutch-1.0 (nutchwax.com; crawler@nutchwax.com)
Nutraspace/Nutch-1.2 (www.nutraspace.com)
NutraspaceBot/Nutch-2.3-SNAPSHOT
NWBSpider/Nutch-2.0-dev
Openindex Test Spider/Nutch-1.9
OpenPlaces/Nutch-1.0-dev (OpenPlaces Content Crawler; http://www.openplaces.com; dnadeau 64th-ascii-char openplaces c o m)
OpenWebIndex/Nutch-1.5
OpenWebIndex/Nutch-1.6
Opera Nutch Spider/Nutch-1.13
Opera/9.80 (J2ME/MIDP; Opera Mini/Nutch-0.9 (crawl for fun; www.zipfelchappe.com; zipfelchappe@localhost)/19.916; U; en) Presto/2.5.25
Opera/9.80 (J2ME/MIDP; Opera Mini/Nutch-0.9 (crawl for fun; www.zipfelchappe.com; zipfelchappe@localhost)/20.2463; U; en) Presto/2.5.25
Opera/9.80 (J2ME/MIDP; Opera Mini/Nutch-0.9/838; U; en) Presto/2.4.15
Opera/9.80 (J2ME/MIDP; Opera Mini/Nutch-1.0 (Test crawl; lucene.apache.org/20.2477; U; en) Presto/2.5.25
Opera/9.80 (J2ME/MIDP; Opera Mini/Nutch-1.0- dev (Crawler for Balihoo.com search engine - obeys robots.txt and robots meta tags ; http:/19.872; U; en) Presto/2.5.25
Opera/9.80 (J2ME/MIDP; Opera Mini/Nutch-1.0-dev (Sapphire Web Crawler using Nutch; http:/18.684; U; en) Presto/2.4.15
Opera/9.80 (J2ME/MIDP; Opera Mini/Nutch-1.0-dev/838; U; en) Presto/2.4.15
OrangeCrawler/Nutch-1.0 (ldorange.crawler@orange-ftgroup.com)
Orionis Crawler/Nutch-1.15 (Web crawler from Nacely; contact at nacely dot com)
Pangalactic Gargleblaster/Nutch-1.2 (Very intoxicating; http://circuitbomb.com; DigitalMail)
PaqueBot/Nutch-1.12
PaqueBot/Nutch-1.13
PenTest.sg/Nutch-1.1 (www.PenTest.sg; crawler@PenTest.sg)
Peter Wang/Nutch-0.9 (Nutch spiderman; http://peterpuwang.googlepages.com ; MyEmail)
pic2u/Nutch-1.2 (http://www.pic2u.com)
pilican/Nutch-1.9
pilican/Nutch-1.9-SNAPSHOT
Pinky and Brain/Nutch-1.5.1
Pluggd/Nutch-0.9 (Pluggd automated crawler; http://www.pluggd.com; support at pluggd dot com)
PluggedInNode/Nutch-1.4
PR Crawler/Nutch-1.0
PR Crawler/Nutch-1.0 (data mining develpment project; crawler@projectrialto.com)
PRCrawler/Nutch-0.9 (data mining development project)
PRCrawler/Nutch-0.9 (data mining development project; crawler@projectrialto.com)
prg-tst/Nutch-1.9
Primo Web Spider/Nutch-1.4
PrivateSearch/0.1.0 (Polite Nutch Crawler; grierforensics.com)
Punk Spider/Nutch-1.4
QEAVis agent/Nutch-0.9 (http://nlp.uned.es/qeavis/)
QH/Nutch-1.5
QkaSpider/Nutch-2.2.1
QleeQ1/Nutch-1.4
RADaR-Bot/Nutch-1.3 (http://radar-bot.com/)
RADaR-Bot/Nutch-1.4 (http://radar-bot.com/)
Raymond Balmès/Nutch-1.0 (spiderman; http://www.balmes.com ; raymond.balmes@gmail.com)
rdfbot/Nutch-1.0-dev
REAP-crawler Nutch/Nutch-1.0-dev (Reap Project; http://reap.cs.cmu.edu/REAP-crawler/; Reap Project)
REAP-crawler/Nutch-1.0-dev (Sapphire Web Crawler using Nutch; http://boston.lti.cs.cmu.edu/crawler/; mhoy@cs.cmu.edu)
research-scan-bot/Nutch-1.0
rmatie.com spider/Nutch-1.4
RoadRunner/Nutch-1.0 (webmaster@fieldtech.org)
roboo/Nutch-1.0 (roboo; http://wap.roboo.com; winter.pi@roboo.com)
roboobot/Nutch-1.0 (roboobot; http://wap.roboo.com; winter.pi@roboo.com)
robotCazoodleBotCazoodleBot/Nutch-0.9-dev (CazoodleBot Crawler; http://www.cazoodle.com/cazoodlebot; cazoodlebot@cazoodle.com)
Robotgenius crawler/Nutch-1.0-dev (http://robotgenius.net; misc at robotgenius dot net)
robotgenius/Nutch-1.0-dev
rogerbot/1.1 (http://moz.com/help/guides/search-overview/crawl-diagnostics#more-help, rogerbot-crawler+pr4-crawler-15@moz.com)/Nutch-1.13
SafeDNS search bot/Nutch-1.9 (https://www.safedns.com/searchbot; support [at] safedns [dot] com)
sait/Nutch-0.9 (SAIT Research; http://www.samsung.com)
SapphireWebCrawler/1.0 (Sapphire Web Crawler using Nutch; http://boston.lti.cs.cmu.edu/crawler/; mhoy@cs.cmu.edu)
SapphireWebCrawler/Nutch-1.0 (Sapphire Web Crawler using Nutch; http://boston.lti.cs.cmu.edu/crawler/; lezhao+crawl@cs.cmu.edu)
SapphireWebCrawler/Nutch-1.0 (Sapphire Web Crawler using Nutch; http://boston.lti.cs.cmu.edu/crawler/; philgooh@cs.cmu.edu)
SapphireWebCrawler/Nutch-1.0 (Sapphire Web Crawler using Nutch; http://boston.lti.cs.cmu.edu/crawler/; philgooh@cs.cmu.edu)1611093279668
SapphireWebCrawler/Nutch-1.0 (Sapphire Web Crawler using Nutch; http://boston.lti.cs.cmu.edu/crawler/; philgooh@cs.cmu.edu)1611093398146
SapphireWebCrawler/Nutch-1.0-dev (Sapphire Web Crawler using Nutch; http://boston.lti.cs.cmu.edu/crawler/; mhoy@cs.cmu.edu)
SBIder/Nutch-1.0-dev (http://www.sitesell.com/sbider.html)
SCFCrawler/Nutch-1.8 (Image Crawler for StolenCameraFinder.com; http://www.stolencamerafinder.com/; crawler@stolencamerafinder.com)
Schloerbot/Nutch-1.0 (Schloer consulting bot; http://schloerconsulting.com/schloerbot)
ScholarScope/Nutch-1.2 (Scholar Research Engine)
searchdnabot/Nutch-1.0 (SearchDNA bot; http://searchenginedna.com; crawl at searchenginedna dot com)
SearchEngineVerificationCrawler/Nutch-1.0 (The purpose of this crawling is to collect web pages for verifying search engines.; http://www.yama.info.waseda.ac.jp/~takuya/en/aboutSRVC.html; srvc at yama
SEcomp/Nutch-1.6 (SEcomp)
SeekGen/Nutch-0.9 (SeekGenBot; http://www.seekgen.com; Email)
Seengine/1.0/Nutch-2.0-dev
SemrushBot/Nutch-1.5-SNAPSHOT
Server/Nutch-1.2
Setooz/Nutch-1.0 (http://www.setooz.com)
sgcrawler in-the-right-place/Nutch-1.3
sGroup crawler 1/Nutch-1.3
SHC/Nutch-1.0 (SemanticHacker Crawler; http://www.semantichacker.com/crawler-info; abuse@semantichacker.com)
Sigram/Nutch-1.0-dev (Test agent for Nutch development; http://www.sigram.com/bot.html; bot at sigram dot com)
SimilarPages/Nutch-1.0-dev (SimilarPages Nutch Crawler; http://www.similarpages.com; info at similarpages dot com)
SimilarPages/Nutch-1.0-dev (SimilarPages Nutch Crawler; http://www.similarpages.com; info@similarpages.com)
SindiceBot/Nutch-1.0-dev (http://sindice.com/dev?section=bot)
SindiceBot/Nutch-1.0-dev (http://sindice.com/developers/bot)
sky nutch crawler/Nutch-1.9
SlowTestCrawler/0.1/Nutch-2.1 (Experimental)
SlowTestCrawler/Nutch-2.1 (Experimental)
SocialVest Spider/Nutch-1.4
SonyEricssonK750i/R1N Browser/SEMC-Browser/4.2 Profile/MIDP-2.0 Configuration/CLDC-1.1/Nutch-1.0-dev
sphsearch.org/Nutch-1.0 (sphsearch.org; crawler@sphsearch.org)
sphsearch.org/Nutch-1.0-dev (www.sphsearch.org; crawler@sphsearch.org)
spider-lh/Nutch-1.10
Spiderbot/Nutch-1.7
SpiderMan/Nutch-0.9 (Nutch spiderman; http://spiderman.nutch.com ; MyEmail)
spyder/Nutch-2.1
spyder/Nutch-2.1 (just another internet crawler; http://www.paloaltonetworks.com/products/features/url-filtering.html; ghalevy@paloaltonetworks.com)
srmse/Nutch-1.7
ssearch_bot/Nutch-1.0 (sSearch Crawler; http://www.semantissimo.de)
StarhubBot/Nutch-1.10
strascom/Nutch-1.7
SU Nutch Spider/Nutch-1.4
SubhojitTestCrawl/Nutch-1.4
Sufog/Nutch-2.2.1 (www.sufog.com; www.sufog.com)
TANNER Spider/Nutch-1.1
tbot-nutch/Nutch-1.10
TCDBOT/Nutch-0.8 (PhD student research;http://www.tcd.ie; mcgettrs at t c d dot IE)
temaseek.com/Nutch-1.0 (temaseek.com; crawler@temaseek.com)
Teoma/Nutch-1.0 (Mozilla/5.0 (compatible; Ask Jeeves/Teoma); http://about.ask.com/en/docs/about/webmasters.shtml)
Teoma/Nutch-1.2 ( Question and Answer Search; Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://sp.ask.com/docs/about/tech_crawling.html); bot@afarm.com)
Test crawler Nutch/Nutch-1.0-dev (Nutch Test Project; changkuk@cmu.edu)
test search engine/Nutch-1.19-SNAPSHOT
Test Spider/Nutch-1.12
Test Spider/Nutch-2.1
Test-Fetcher-0.1/Nutch-0.9 (Awesomeness)
Test.Buzzz/Nutch-0.8.1 (Test.Buzz; http://test.com; test@test.com)
test/Nutch-0.8.1 (Test robot; http://test.com; info at test.com>)
Test/Nutch-1.12
test/Nutch-1.2
test/unique/Nutch-0.8.1
TestBot/Nutch-1.1
testbot/Nutch-1.9 (testbot 123)
TestCrawler/Nutch-0.9 (Testing Crawler for Research ; http://balihoo.com/index.aspx; tgautier at balihoo dot com)
TestCrawler/Nutch-0.9 (Testing Crawler for Research ; http://chitchit.org/TestCrawler.html; amitjain at spro dot net)
tester/Nutch-1.6 (just a test; www.example.com; mail@example.com)
testnutch/Nutch-1.10
TestNutch/Nutch-1.2 (testing Nutch; http://nutch.apache.org; info@nutch.apache.org)
TestSpider/Nutch-1.0-dev
The Lemur Web Crawler/Nutch-1.3 (Lemur Web Crawler using Nutch 1.3; http://boston.lti.cs.cmu.edu/crawler_12/; admin@lemurproject.org)
The Lemur Web Crawler/Nutch-1.3 (Lemur Web Crawler; http://boston.lti.cs.cmu.edu/crawler_12/; admin@lemurproject.org)
Tipiweb/Nutch-1.0 (http://www.tipiweb.net)
toofaan/Nutch-1.0 (http://www.toofaan.com)
TosCrawler/Nutch-1.4 (http://www.toshiba.co.jp/rdc/about/crawl_info.htm; 'Rdc-crawler at ml dot toshiba dot co dot jp')
TosCrawler/Nutch-1.4 (http://www.toshiba.co.jp/rdc/about/crawl_info_en.htm; 'Rdc-crawler at ml dot toshiba dot co dot jp')
TosCrawler/Nutch-1.5.1 (http://www.toshiba.co.jp/rdc/about/crawl_info.htm; 'Rdc-crawler at ml dot toshiba dot co dot jp')
TosCrawler/Nutch-1.6 (http://www.toshiba.co.jp/rdc/about/crawl_info.htm; 'Rdc-crawler at ml dot toshiba dot co dot jp')
TosCrawler/Nutch-1.6 (http://www.toshiba.co.jp/rdc/about/crawl_info_en.htm; 'Rdc-crawler at ml dot toshiba dot co dot jp')
TosCrawler/Nutch-1.8 (http://www.toshiba.co.jp/rdc/about/crawl_info.htm; 'Rdc-crawler at ml dot toshiba dot co dot jp')
Trailfire-bot/0.7.1 (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org)
Trailfire/0.7.1 (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org)
TsolCrawler/Nutch-1.4 (http://www.toshiba-sol.co.jp/info/140100.htm; 'tsol-itc-crawler at toshiba-sol dot co dot jp')
TUBITAK Crawler/Nutch-1.6
Tycoon Agent/Nutch-1.0-dev
UCY/Nutch-1.2
UKWizz/Nutch-0.8.1 (UKWizz Nutch crawler; http://www.ukwizz.com/)
university/Nutch-1.0 (research)
usyd.schwa.lab.nlp.research/Nutch-1.1 (http://it.usyd.edu.au/~smerity/schwa/crawler/; smerity %AT% it.usyd.edu.au)
uw_cse_xwci uw.crawler@gmail.com http://myst.cs.washington.edu/index.html/Nutch-0.9 (University of Washington Computer Science XWC project crawl; http://myst.cs.washington.edu/index.html; uw dott craw
VeriCiteCrawler/Nutch-1.9
vik-robot/Nutch-1.0 (vikspider; http://vik.com; chenlibiti@163.com)
viz/Nutch-1.7
volverine/Nutch-0.9 (agentspider; beast@mail.com)
Vorboss Web Crawler [crawl@vorboss.net]/Nutch-2.3
Vorboss Web Crawler/Nutch-2.3
VWBOT/Nutch-0.9-dev (VWBOT Nutch Crawler; http://vwbot.cs.uiuc.edu; vwbot@cs.uiuc.edu)
Web archive/Nutch-1.7 (http://web.archive.com)
web2express.org/Nutch-0.9-dev (leveled playing field; http://web2express.org/; info at web2express.org)
Webcrawler/Nutch-1.0 (Test crawl; lucene.apache.org/nutch/; a@b.net)
Webcrawler/Nutch-1.0-dev (Test crawl; lucene.apache.org/nutch/; a@b.net)
WebCrawler/Nutch-1.2 (WebCrawler; WebCrawler)
webcrawler101/Nutch-1.9
webmoney.advisor.bot/Nutch-1.0
webmoney.advisor/Nutch-1.0
webmoney.advisor/Nutch-1.2
WebMoney/1.0 (AdvisorBot 0.1)/Nutch-1.2
Webscope/Nutch-0.9-dev (http://www.cs.washington.edu/homes/mjc/agent.html)
Wegtam Crawler/Nutch-1.10-SNAPSHOT
Wegtam Crawler/Nutch-1.9-SNAPSHOT
weivel/Nutch-0.9 (weive.com - web spider; http://www.weive.com/page/view/weivel; weivel at weive dot com)
WF search/Nutch-1.12
WhelanLabs Search/Nutch-1.4 (WhelanLabs Web Crawler Agent; searchengine@whelanlabs.com; whelanlabs@gmail.com)
whiteiexpres/Nutch-0.9 (whiteiWebBot; whiteiexpress.com; whiteye@kon-x.com)
wiki.do.test/Nutch-1.4 (Nutch testing; http://crawler_test_amazon.com; me@crawler_test_amazon.com)
wiki.do/Nutch-1.4 (WikiDo.com; http://wikido.com; wikidocom@gmail.com)
WikiDo/Nutch-1.4 (http://wikido.com; crawler@wikido.com)
WK/Nutch-1.1 (Nutch spiderman; n/a ; MyEmail)
wminer/Nutch-1.4
wminer/Nutch-1.4 (wminer; wminer.com; wminer)
wminer/Nutch-1.5.1 (wminer; wminer.com; wminer)
wminer/Nutch-1.6 (wminer; wminer.com; wminer)
WocBot/Nutch-1.4 (Wocodi Web Crawler 1.0; http://www.wocodi.com/; crawler@wocodi.com)
Woovi/Nutch-1.0 (http://www.woovi.com/bot)
workload-generator/Nutch-0.9 (web20-setup; https://twiki.hpl.hp.com/bin/view/Main/WebsearchNotes)
www.osaicbt.com/Nutch-2.2.1
Xiao/Nutch-1.8
yggdrasil/Nutch-0.9 (yggdrasil biorelated search engine; www dot biotec dot tu minus dresden do de slash schroeder; heiko dot dietze at biotec dot tu minus dresden dot de)
ylmsch/Nutch-1.11
YottaBot Spider/Nutch-2.3
your-crawler-name/Nutch-2.3.1
YourNutchSpider/Nutch-2.2.1
yourOrganization/Nutch-1.10
ZipppBot/Nutch-0.9 (ZipppBot .02; http://www.zippp.net; crawlteam@zippp.net)
ZipppBot/Nutch-1.0-dev (ZipppBot .02; http://www.zippp.net; crawlteam@zippp.net)
ZoomBot/Nutch-1.12-SNAPSHOT
ZoominfoBot/Nutch-1.12-SNAPSHOT
Zscho.de Crawler/Nutch-1.0-Zscho.de-semantic_patch (Zscho.de Crawler, collecting for machine learning; http://zscho.de/ )
Zscho.de Crawler/Nutch-1.0-Zscho.de-semantic_patch (Zscho.de Crawler, collecting for machine learning; http://zscho.de/)
zschobot/Nutch-0.9-semantic_patch (zschobot indexing; Zscho.de/de/bot.html)
USER AGENT CATEGORIES
BOTS
BROWSERS
MOBILE BROWSERS
OLD SCHOOL
OTHER
SPAM / HACKS