Instagram's abuse-fighting AI system has trouble with Kanye West lyrics

In an effort to root out spam and abusive comments from its app, Instagram has been using DeepText - a deep-learning, word categorization system pioneered by its parent company Facebook - to separate good text from harmful text, and eliminate the latter.

And according to a recent Wired profile, Instagram has been testing DeepText's ability to fight "mean comments" by feeding the system a variety of clean and explicit material including, notably, Kanye West lyrics.

Instagram cofounder Kevin Systrom began using DeepText to successfully fight spam posts after the release of the "near-human," text-understanding AI last June.

In June of this year, Systrom turned to the more complex task of eliminating mean comments from the app through DeepText - a formidable undertaking that requires the AI system to assess the context of potentially abusive, and often enigmatic, posts.

So far, according to the report, user response to the tool has been "reasonably positive," if unnoticed. But it does have some notable issues.

For instance, DeepText has had difficulty with words and slang that have different meanings across cultures. And it also has trouble distinguishing Kanye West lyrics from abusive insults when Instagram employees feed it such content, as the report details of West's controversial song "Famous":

"Every line in this sequence got banned when it was put through: 'For my southside n----- that know me best / I feel like me and Taylor still might have sex / Why, I made that b---h famous.'"

Read the full Wired profile here (spotted via Complex).