We got to see how IBM's Watson supercomputer could help diagnose a patient

CHICAGO - It's not every day that you get to watch artificial intelligence in action. 

IBM Watson Health and the Radiological Society of North America teamed up on a project called "Eyes of Watson," on display at the RSNA conference this week in Chicago. The demonstration is meant to show radiologists how Watson evaluates X-ray images, from start to finish. 

When I stopped by earlier this week, the area was busy with people interested to compare what they saw in a scan to what Watson saw. 

To start, I was asked to pick which finding I saw in a particular image of a breast. Not having any training in the area, I could only guess. 

Then, I pressed the "Ask Watson" button, and the system got to work. It started by analyzing the image in different ways, and narrowing in on areas it determined were anomalies.

The step-by-step analysis was designed to show radiologists at the conference how Watson arrived at a particular conclusion. "The ask was to show how it thinks, or how did it know that this was the right feature to look at," IBM Research Fellow Tanveer Syeda-Mahmood, who was behind the project, told Business Insider.

Up next, Watson walked through all the possible diagnoses it determined were possible from the breast image.

"It's showing systematically how it eliminated other possibilities so you can be confident that it's not just magic but it is using the same thought process that you would use to eliminate possibilities."

To do this, Watson looked at text from the chart it was given, the shape, orientation, and other characteristics of the image. 

"If you think of knowledge as a graph, then there may not be an answer that directly connects the disease to your findings," she said. "You have to make inferences." That process as illustrated here is simplified from what Watson's working through behind the scenes, which Syeda-Mahmood said was intentional to make it easier to understand.

In the end, the diagnosis Watson picked lit up in green. 

I then had a chance to evaluate Watson's diagnosis.

Watson-powered products will start rolling out in various degrees starting in 2017, thought it might be a couple more years until Watson can diagnose people in your local hospital, since that requires clinical trials and regulatory approval to validate that it's safe and effective to use. 

At the conference, there was a lot of curiosity about Watson and what it can do. Syeda-Mahmood said some people indicated they might use it as a tool to train new radiologists. Other reactions were mixed, with some doctors expressing skepticism and others excitement about the future of artificial intelligence in radiology.