Gabriel Weinberg, the founder of the DuckDuckGo search engine had an interesting blog post over the weekend, talking about some weird traffic the site was getting. He points to a couple SERPs generated by what he deems to be queries from botnets.
In the comments, Eric Klein says one of them looks like its trying to boost ranking by “making it seem that there are a lot of requests.”
“Could they be trying to game their placement on search pages by hitting the search sites often enough to pretend to be meaningful?” he asks.
Weinberg responds, “Ahh, that’s a very plausible theory.”
“It’s known as ‘referral spam,” adds @TristanPerryIX in the comments. “It spams servers with HTTP referrers for two purposes:
1) Sometimes webmasters will (like you done :)) review the logs, and then click on their website. I assume these spammers are (wrongly!) assuming that this is ‘good’ traffic which might result in a sale/advert click.
2) Some websites have an automatic ‘these websites link to us..’ section or widget. Naturally referral spam could be used to game this section, thus getting a backlink (thus gaming the search engines).”
DuckDuckGo doesn’t save IPs, which is one of its selling points. It blocks them at the firewall level, Weinberg says, noting that this is dissociated from query data. “If we didn’t block the most egregious botnet machines and abusers, our machines would almost instantly be under water,” he adds.
“This discussion now makes me wonder if other search engines include this errant traffic in their query counts,” says Weinberg. “We work hard to keep them completely out because they would overwhelm our real direct queries #s and therefore distort our perception of progress. We also separate out API requests for the same reason, which now also makes me wonder whether everyone else is doing that too.”
It’s an interesting factor to consider, when you think about search market share.
Below, you can view a recent interview we conducted with Weinberg: