The big Facebook outage offers a behind-the-scenes look at how the social network's AI 'sees' your photos and interprets them for blind users

Photo by Chip Somodevilla/Getty Images

Facebook CEO Mark Zuckerberg. On Wednesday, the company experienced major photos outages.

• An image outage on Facebook, Instagram, and WhatsApp shows how Facebook's AI automatically tags your photos behind the scenes.
• The tags sparked anger and amusement on Twitter, as people shared how their photos were interpreted - or misinterpreted.
• People shared some of their tags on Twitter, which ranged from the mundane "(one person, beard") to the disconcerting ("people standing, hoes, and indoor").

• Visit Business Insider's homepage for more stories.

An image outage across Facebook, Instagram, and WhatsApp gives a behind-the-scenes look at how Facebook's AI sees your photos.

The outage, which started on Wednesday and impacted Facebook's 1.5 billion-plus daily active users and rendered Instagram all but unusable, stopped social media images from loading and left in their place descriptions like: "image may contain: table, plant, flower, and outdoor" and "image may contain: tree, plant, sky."

(Or maybe that's just my feed.)

Screenshot/Matt Weinberger

This is the kind of thing a lot of people saw on Facebook, Instagram, and WhatsApp instead of their actual photos.

These descriptions, or tags, show how Facebook's AI interprets these images.

On Twitter, people shared screenshots of how their photos were tagged by Facebook. "To be fair, 'one person, beard' is pretty much a spot-on description of me," wrote Zack Whittaker, an editor at TechCrunch. 

So, what's going on?

Facebook automatically scans all photos on the social network with facial and image recognition software powered by AI, to detect who or what is being pictured.

This is then used in the company's accessibility efforts to describe photos to people who are blind or otherwise visually impaired, and who are accessing the site via a screen reader.

In short, Facebook uses machine learning to automatically interpret photos, then reads this interpretation aloud to blind users. The photo outage just meant that we got an in-your-face look at those interpretations, too. 

At times, it appeared the AI incorrectly tagged the photo, as in the case of Fortune reporter Danielle Abril's case where the photo description read: "5 people, including Danielle Abril, people smiling, people standing, hoes and indoor." Abril did not immediately respond to a request for comment, but it's perhaps possible that the photo's subjects were holding gardening tools in the photo.

So while there's a benign explanation here, it's also a stark reminder of just how much data Facebook is gathering at all times, even when we don't realize it's happening. Thanks to its increasingly sophisticated AI technology, Facebook can even gather information from something as innocous as a vacation photo.

"Once something is legible, of course, it becomes easy to store, analyze, and extract data from. It's only when the system breaks down, like today, that we realize it's happening at all," wrote James Vincent in the Verge.