Bots vs Browsers - Public Bot / User Agent Database & Commentary

Bots vs Browsers - database of 16,271,428 user agents and growing


JUN 23

2008

Another one bites the dust



Just a quick update - we had a few new interesting bots, and another menacing crawl from Wget...
  • Wget/1.11.3 returned with a vengeance today, this time from IP 131.107.0.106. They burned through 66,000 page views in just a few hours. As usual, we banned this IP and life will go on. related...
  • SearchHub Bot is a bot sent from inSynchro, a Malaysian Business Intelligence company established in 1999 that specializes in developing web-based enterprise solutions specifically in the area of Project Management. related...
  • Thanks to the Google "Translate this page" feature, I was able to scout out Semager/1.2. Their site is in German, and from what I can tell in the English translation, the Semager bot feeds the Semager Search Engine. Nothing sinister here - just another search engine bot. We do have a few Semager user agents in stock - related...
Bottom line: 182,499 user agents and 1,959 bots.

Blog Archives:












APR 18

2009

Where did Palm go?







JUL 13

2008

2,000 bots!





JUN 19

2008

And we're back...


JAN 21

2008

Happy 2nd Birthday!


OCT 25

2007

The Creature Post



JUL 22

2007

Hot Summer Bots




JUN 13

2007

Twiceler Strikes!


JUN 3

2007

Land of 1,000 bots




MAR 24

2007

March Bot Madness


MAR 10

2007

Got Spiders?




FEB 3

2007

February Stars


JAN 12

2007

Happy 1st Birthday!




DEC 12

2006

50,000 User Agents!



OCT 17

2006

2 weeks, 30 bots


OCT 2

2006

4 weeks, 23 bots



AUG 26

2006

500 bots!


AUG 20

2006

5 days, 8 bots











MAY 2

2006

A few new bots









JAN 12

2006

Day One


USER AGENT CATEGORIES

BOTS
BROWSERS
MOBILE BROWSERS
OLD SCHOOL
OTHER
SPAM / HACKS
  • Script Injections (349)
  • SQL Injection (7,701)
  • PHP Injection (1,993)
  • XSS Attacks (66)
  • Link Injection (250,869)