Advertising fraud is a serious problem. The Internet Advertising Bureau predicts ad fraud could cost brands as much as £6.5 billion ($10.1 billion) in wasted spend from brands each year. For Google - the biggest seller of online advertising in the world - that's a huge problem. It wants and needs to be trusted that ads being bought through its platform are actually reaching a human audience, not an army of botnets created by criminal gangs exploiting unsuspecting consumers' personal computers, who are taking serious amounts of money out of the advertising ecosystem by generating millions of false clicks on ads (among other techniques.)
And it's because ad fraud is essentially a form of organized crime that many people AdAge encountered during its time at Google's offices in central London asked only to be referred to by their first names. One Russian engineer, Sasha, said: "Because it is part of organized crime, I'm guessing it would not be a friendly environment for the people that speak out against it."
The unit itself is situated behind a "hulking door with a circular vault-like handle," AdAge describes, which just adds to the air of mystery around it. The operation is "one of the most important and best-protected secret units of the web," AdAge writes.
Spider.io
Spider.io was sold to Google early last year.
The man leading Google's botnet-fighting unit can be named: Douglas de Jager. He founded Spider.io, which was sold to Google for an undisclosed amount last year. All seven Spider.io staff members moved over to Google. AdAge says it is the mixture of Spider.io's expertise, plus Google's computing power that has speeded up the fraud-fighting process "dramatically."
But it has also added some restrictions: The bot-fighting unit has to steer clear of Google's sales team to avoid conflicts of interest. AdAge explains:
The sales team, as you might imagine, doesn't stand to gain immediately when inventory is removed from Google's systems. The more ads it sells, the more money it takes in.
The engineers work in what they describe as their "dungeon," scanning malware binary for up to two hour sessions at a time looking for patterns, and raking the forums used by fraudsters for clues as to where the bad actors originate from. But they do regularly emerge from their dungeon: For coffee. After each session, the team "flocked" to Google's famous micro-kitchens, where everything is free, to caffeinate "and forget," AdAge writes.
Ultimately, Google's team are looking through all these clues for "signals" - a type of behavior that is inadvertently created by a fraudster when they program a bot that can help the engineers identify the traffic.
Google's secret weapon to do this is called "Powerdrill":
Powerdrill is a freak computing system. It's capable of processing a half trillion cells of data in a less than five seconds (translation: It's damn fast). And it can spit that data out as charts and other graphical representations that make it possible to spot the irregularities of nonhuman traffic.
AdAge goes into further detail - including a fascinating anecdote about an un-named ad verification service that was responsible for a swathe of non-human traffic on the Google network - about what Google's team are up to and the way it goes about tracking down fraudsters in its article, which you can read in full here.