Bots vs Browsers - Public Bot / User Agent Database & Commentary

Bots vs Browsers - database of 16,271,402 user agents and growing


JUL 25

2006

40,000 User Agents!!!



Today, Bots vs Browsers finally broke the big 40,000 user agent benchmark. In the last few weeks leading up to this, we have seen quite a few new bots also (18 to be exact). The most eventful visits were:
  • Gaisbot/3.0 paid us a visit, apparently from university affiliated with Global Area Information Servers.
    They were also kind enough to disclose a URL and an email address in the user agent string, in case anyone has questions / complaints.   related...
  • The Bloglines/3.1 bot came by to read up on our blog.   related...
  • MyFamilyBot/1.0 appeared in our web logs also, apparently looking for family-related content.   related...
  • Crawler/1.0 made it's debut sporting a very creative and original name. Crawler/1.0 comes from the land of http://elibron.com (which is ironically a literature site, yet names are scarce...)   related...
  • PediaSearch.com Crawler crawled by - I can't seem to find any information about this user agent, but I'd say it's safe to assume it's a crawler.   related...
  • MQBot visited our sites also - this is a bot that aids in research and development at the University of Illinois in Urbana-Champaign.   related...
  • The yoono/1.0 web-crawler/1.0 (pronunciation anyone?) web crawler stopped by as well.   related...
  • Both Yahoo! Slurp and Googlebot stepped up with new bots on our sites in the last three weeks.   related...
  • MojeekBot/2.0 crawled by looking for content for it's search engine at Mojeek.com.   related...
  • And, as always, a few new iterations of FindLinks came over. Development efforts over at the FindLinks laboratories must be very fruitful, seeing that versions 1.1.3-beta2, 1.1.3-beta3, and 1.1.3-beta4 have all been introduced this month.
    To see a history of FindLinks bots that have toured our sites, click here   related...
The latest catch puts our database at 40,000 user agents and 436 bots

Blog Archives:












APR 18

2009

Where did Palm go?







JUL 13

2008

2,000 bots!





JUN 19

2008

And we're back...


JAN 21

2008

Happy 2nd Birthday!


OCT 25

2007

The Creature Post



JUL 22

2007

Hot Summer Bots




JUN 13

2007

Twiceler Strikes!


JUN 3

2007

Land of 1,000 bots




MAR 24

2007

March Bot Madness


MAR 10

2007

Got Spiders?




FEB 3

2007

February Stars


JAN 12

2007

Happy 1st Birthday!




DEC 12

2006

50,000 User Agents!



OCT 17

2006

2 weeks, 30 bots


OCT 2

2006

4 weeks, 23 bots



AUG 26

2006

500 bots!


AUG 20

2006

5 days, 8 bots











MAY 2

2006

A few new bots









JAN 12

2006

Day One


USER AGENT CATEGORIES

BOTS
BROWSERS
MOBILE BROWSERS
OLD SCHOOL
OTHER
SPAM / HACKS
  • Script Injections (349)
  • SQL Injection (7,701)
  • PHP Injection (1,993)
  • XSS Attacks (66)
  • Link Injection (250,869)