Bots vs Browsers - Public Bot / User Agent Database & Commentary

Bots vs Browsers - database of 16,270,901 user agents and growing


JUL 13

2008

2,000 bots!



We set a new milestone this week for our user agent database - we broke 2,000 bots!

This week's logs were comprised mostly of new versions of bots we know about. There are a few new faces mixed in there too - here's the latest:
  • On the browser side, one new face in our logs is from iGetter. This is a new download utility that looks pretty interesting. related...
  • Thumbshots is another new bot on our site - we witnessed 3 unique bot user agents from them this week: thumbshots-de-bot, KFSW/1.0.0.0.4 thumbshot-de-bot, and KFSW/1.0.0.0.1 ThumbShots-de-bot. related...
  • Exooba crawler/0.5.1 debuted this version this week in our server logs. According to their site, they are a testing crawler that uses gathered data strictly for testing purposes only. Their site also informed us that they fully comply with robots.txt exclusion standards. related...
  • Here's an interesting one - Daruma/1.08 (Windows 98; U) [en]. We were unable to find anything about this user agent in our research. When googling this bot, we are the only result on google as of yet. However, according to Wikipedia, the word "Daruma" is commonly used to describe "Daruma Dolls", also known as dharma dolls. These are hollow, round Japanese wish dolls, modelled after Zen patriarch Bodhidharma. This just goes to show that you can learn alot more than techie facts on this blog. related...
  • CatchBot/1.0 is another new face on the site. related...
  • Metaspinner/0.01 is a bot from MetaSpinner - yet another search engine crawling the web for data. related...
  • OmniExplorer_Bot/1.0x Job Crawler is a new version of the OmniExplorer bot family. OmniExplorer has a long standing pedigree of bots on our site, and here you can see more of them: related...
  • Yacybot is another long-standing bot family on our site, and they turned up a new version this week as well related...
  • We posted about our first OOZBOT sighting last week, and now we have a new user agent from them as well: OOZBOT related...
Mid July is approaching fast, and we've got 187,746 user agents and 2,018 bots - thanks for visiting!

Blog Archives:












APR 18

2009

Where did Palm go?







JUL 13

2008

2,000 bots!





JUN 19

2008

And we're back...


JAN 21

2008

Happy 2nd Birthday!


OCT 25

2007

The Creature Post



JUL 22

2007

Hot Summer Bots




JUN 13

2007

Twiceler Strikes!


JUN 3

2007

Land of 1,000 bots




MAR 24

2007

March Bot Madness


MAR 10

2007

Got Spiders?




FEB 3

2007

February Stars


JAN 12

2007

Happy 1st Birthday!




DEC 12

2006

50,000 User Agents!



OCT 17

2006

2 weeks, 30 bots


OCT 2

2006

4 weeks, 23 bots



AUG 26

2006

500 bots!


AUG 20

2006

5 days, 8 bots











MAY 2

2006

A few new bots









JAN 12

2006

Day One


USER AGENT CATEGORIES

BOTS
BROWSERS
MOBILE BROWSERS
OLD SCHOOL
OTHER
SPAM / HACKS
  • Script Injections (349)
  • SQL Injection (7,701)
  • PHP Injection (1,993)
  • XSS Attacks (66)
  • Link Injection (250,869)