A billionaire hedge fund manager explains what is freaking him out about the art market

Ken Griffin, the billionaire founder of the hedge fund Citadel, was collecting art before it was cool on Wall Street.

Now he's concerned that his peers who have jumped into the art market are doing it for the wrong reasons - they consider it an investment.

"The art market is now viewed as a market," he said Thursday in an interview with CNBC's Kate Kelly. "When I started collecting art 20 years ago, it was really a market dominated by collectors. Today we're seeing far more people pursuing art as an investment. I would have some cause for concern around that."

The problem with seeing art as an investment is that it is an opaque market. There's not enough information out in the public to really understand the future value of an asset.

Two professors at New York University's Stern School of Business, Jianping Mei and Michael Moses, developed their own valuation method - the Mei Moses Art Indexes - after writing a paper called "Art as an Investment and the Underperformance of Masterpieces" in 2002.

The idea behind the indexes was to bring more clarity to price discovery by putting sales data in one place.

"Two major obstacles in analyzing the art market are heterogeneity of artworks and infrequency of trading," the paper said. "The present paper overcomes these problems by constructing a new repeated-sales data set based on auction art price records at the New York Public Library as well as the Watson Library at the Metropolitan Museum of Art."

Sounds great, but it's not enough.

Mei and Moses will tell you that even with the information they have the art market is still super secretive. Their data sets are not close to complete, and they never will be.

What's more, in the past few years sales have been moving away from big auction houses - where there's some transparency -and toward smaller art dealers. That will only make the market more difficult to track.

So what the newbies are doing, according to Griffin, is buying art based on the value of an artist's name, not based on what the buyer loves. That's adding even more price dislocation to the market.

"We're seeing the tier-one artists with their best works setting all-time record highs," Griffin said. "The Modigliani sold last week for $170 million. But we're seeing second-tier artists and second-tier works by the best artists starting to slide down in price.

"I think people should be very hesitant in thinking about art as an investment."