Shy Guys at TMA’26

Automated bots now account for roughly half of all web requests, and a growing number spoof their identity to evade detection or to ignore robots.txt. JavaScript challenges and CAPTCHAs are resource-intensive and degrade the user experience, while commercial solutions are costly.

In Shy Guys: A Light-Weight Approach to Detecting Robots on Websites, Rémi Van Boxem proposes a passive approach that runs entirely on standard web server logs, with no client-side interaction. It combines user-agent string analysis with favicon-based heuristics.

On 4.6 million requests with 54,945 unique user-agent strings, collected from websites hosted around the world, it detects 67.7% of bot traffic with a 3% false-positive rate, while the state of the art detects less than 20%. It can serve as a first line of defense, sending only the ambiguous requests to active challenges.

We presented it as a poster at TMA’26 in Delft. This is joint work with Cristel Pelsser and Ramin Sadre.

▶️ DIAL