Facebook will start scanning 10,000 posts a second to make comments less terrible
Today, we got a look behind the scenes on how AI is helping Facebook sift all that text and improve the Facebook experience with "Deep Text" - a system developed by Facebook's AI labs that scans 10,000 posts every second in 20 languages.
"Every minute, Facebook users make 4,000 new posts," with 180 million comments made on public Facebook posts per day, says Hussein Mehanna, an engineering director with Facebook's Applied Machine Learning team.
Deep Text is "deep learning-based text understanding engine," as Facebook puts it. Its ability to understand text like a human would is already being put to work in Facebook Messenger. And Facebook plans to take it a lot further.
It's the technology behind the scenes that can distinguish the words "I need a ride," in which case Messenger might prompt you to order an Uber, versus "I was going to get a ride," in which case the moment has already passed. Not to mention "I like to ride a donkey," which is a whole different thing entirely.
More intriguingly, Mehanna says Deep Text can be used in busy Facebook comment threads.
Say for example when Mark Zuckerberg does a live Facebook Q&A, and the top few comments are often in another language, off-topic, or both:
"It becomes really hard to find interesting comments," says Mehanna.
Mehanna says that there's a lot of potential for Deep Text to scan those comments and rank them for you, personally. If Facebook's AI has figured out that you speak English and Farsi, it'll prioritize relevant comments in those languages so you see them first - while pushing annoying things down.
It's an elegant solution for Facebook: It maintains the platform's very public commitment to global free speech, since Deep Text isn't actually deleting the comments.
But it also means that you'll see relevant comments in your own language first, making for a better Facebook experience. And if the comments do contain spam or straight-up hate speech, Deep Text could one day be empowered to automatically regulate.
Another cool thing is that Deep Text will be able to understand when your Facebook status is offering an item for sale, and offer to cross-list it automatically to your regional networks.
Going forward, Mehanna says that they want to train Deep Text to understand more of Facebook's 40 officially-supported languages. But he says that it's also almost inevitable: One Facebook engineer took Deep Text's "brains" and trained it how to speak Indonesian in a weekend.
That ability to learn will also behoove Facebook's future artificial intelligence efforts, especially around chatbots. Unlike Google, Facebook's users are speaking the way that real actual humans speak. It'll make for more humanoid bots.
"I think with Facebook data, we can make AI far more human, and social, and conversational," says Mehanna.