Microsoft has made a computer that can recognise a dog in a bike basket

Microsoft Research and the Carnegie Mellon University have created a computer that can analyse a situation and answer multi-layered questions, according to a blog post.

A multi-layered question is just that: A question with multiple elements that rely on each other. An example could be "What is in the basket?" In order to understand the question, a computer must understand the query "what is," what a "basket" is, and then analyse what is actually in the basket to successfully answer the question.

Beyond deciding that it is, in fact, a dog in the basket, the research could be used to help cyclists who use a phone. Queries such as "What is in the left side behind me?" and "Are there any runners close to me that I might not see?" can be answered by the computer.

"As humans, we focus on what's needed to answer these and other questions," says Xiaodong He, a senior researcher for Microsoft. "With this system, the image goes through deep neural networks, deciding which regions are relevant to the question, and suppressing irrelevant information."

The system uses natural language processing, which breaks down a scene into simple language that a human would use (e.g. "what is in the left side behind me?").

"This is the technology that couldn't have been imagined a few years ago - modeling human behavior to solve problems," said Li Deng, one of the researchers.