This expert team is one reason why Facebook is able to move fast without breaking things

Facebook has a target on its back.

Thanks to its sheer scale, there are many potential avenues for hackers to attack.

And thanks to its tremendous global reach, there are also many well-funded bad guys who would take advantage of any chinks in Facebook's armor.

Given that Facebook has a well-known mandate to "move fast and break things," this presents a dilemma.

"Security" and "moving fast" are usually exact opposites. Developers want to move fast, but the security team needs to make sure that the code they're writing isn't introducing new risks into the system.

So Facebook turned the usual security process on its head, so its developers can keep moving fast without breaking anything too serious.

"You still have to move fast, there's now just a cost to moving fast," says Facebook open source software engineer Christine Abernathy.

A crack team

Facebook software and security engineer Ted Reed says that the goal is to make security part of the normal workflow.

If every developer at Facebook came to him whenever they got a suspicious e-mail, that would be ideal. But the next best thing is to lock everything down behind the scenes.

"We put the burden on ourselves," Reed says.

There's a team of security pros at Facebook, Reed says, whose main job is to stand by until a call comes in.

When a developer flags a piece of the code they're working on as requiring review in Facebook's project management tracking tools, a security engineer rushes off to do a review as soon as they possibly can - while the developer is free to keep on hacking away at the code.

In a more proactive sense, Facebook's security squad is always working to protect the underlying infrastructure, making sure that the data that developers are working with is secured on every level. The goal is to make the underlying security completely unnoticeable to the developer.

"It becomes very hard to build insecure things," Reed says.

The security team also has to build a strong relationship with Facebook's developers.

Often, Reed says, a member of the infrastructure security team will join a Facebook project team to help them solve a problem - and end up joining that team permanently.

Facebook encourages that kind of team-jumping flexibility, and the security team loves it - it means that the product team in question now has someone devoted to preventing hacks.

Giving back

Another big way that the security team wins over Facebook developers is by giving them something that they can't get enough of: Data.

Reed's claim to fame at Facebook is leading the development of a tool called "Osquery." It's a clever piece of software that scans every single computer on the Facebook network and catalogs every aspect of it, from which documents are on the hard drive to what programs are running in the background.

That data gets put into an SQL database - the kind that programmers and data analysts are intimately familiar with.

At the most basic level, Osquery can tell when something is immediately awry. If a server somewhere on Facebook's network is running at 500 percent of its normal server capacity, something is wrong.

"So either you made a code change, or someone else did," Reed says.

When there's a major data breach at companies like Target or Experian, Facebook's security team reads the news, gathers as much as it can about how their system was compromised, and then uses Osquery to make sure they're not vulnerable in the same way.

Opening up

And then, the really clever bit is that Reed's team took an unconventional approach with Osquery.

Most security types are "paranoid," Reed says, but he convinced Facebook's powers-that-be to allow them to release Osquery as open source - meaning that developers from all over the world can look into Osquery's source code and, crucially, contribute back.

It's a smash hit: Since its release in mid-2014, Osquery has become the number-one most popular security project on GitHub, the so-called Facebook for programmers.

Developers from big web companies have started using Osquery and contributing back their own data and the searches they routinely run using it, though Reed says those users don't like to discuss it. Again, security people are paranoid.

The end result, though, is that Reed's Osquery team can offer its developers a continually updated, constantly evolving look into the landscape of computers and how people are using them.

That data is especially important given that Facebook has a huge focus on getting people in the developing world online, where they might not be running a laptop running the latest and greatest version of Windows.

"We can give something back to developers," Reed says.

Screenshot

A screenshot of Facebook Osquery

There's an obvious question here: Has Osquery ever actually turned back a hacker attack? Reed says he honestly doesn't know - it's not his department. He just builds the tools.

Sometimes, Reed says, Facebook's dedicated anti-intrusion squad will get an e-mail, jump up from their desks in alarm, and scramble to a conference room. But when Reed looks in, they're just playing Starcraft.

He asks what happened, and they brush him off.

"Don't worry about it," they tell him.