Businesses are rushing to use generative AI. Now comes the messy part.

• A vast majority of US companies are adopting generative AI tools, according to a new Bain survey.
• Most are also not sure where the value of these tools will come from.

Businesses are embracing generative AI at an unusually fast pace. Now comes the messy part: Making money from these big investments.

That's the main takeaway from a new survey of the corporate world by consulting firm Bain & Company.

It surveyed 200 US companies with at least $5 million in revenue. Half were tech companies while the rest were spread across retail and consumer goods, manufacturing, healthcare, and financial services. Here are the highlights:

• 85% of the companies said adopting AI was a top-five priority. 12% listed the technology as their top priority. Only 1% said it was not a priority.
• Language generation and software coding are the two most common AI applications across all types of businesses.
• The companies reported spending $5 million a year on generative AI, on average.
• A fifth of those surveyed said they are spending more than $50 million per year on generative AI.
• The average annual generative AI spend for companies with more than $5 billion in revenue was $13.1 million.
• Companies with $500 million or less annual revenue spent an average of $1.6 million a year on this technology.

The big ROI question

For the most part, the companies told Bain that generative AI has met or exceeded their expectations. However, they also said the business case for significant AI investment is not clear.

It's a crucial question yet to be answered by AI juggernauts, such as Nvidia, Microsoft, OpenAI, and Google. These tech giants are betting on heavy AI usage in the future, but their customers must find value in these new services if the boom is to continue.

Despite the early rush to embrace generative AI, only 11% of the businesses Bain surveyed had a clear vision for how they would use generative AI and how it would add value.

That could be an issue of vision, talent, or it could be an issue with the tools themselves. Though most respondents reported that the AI tech they've tried met their expectations, a significant minority found the tools fell short of expectations.

The top concerns

In the Bain survey, worries about the quality of generative AI outputs tied with data-privacy and security concerns as issues holding businesses back from moving faster to adopt the technology.

There's also a lack of internal expertise. Compared to a similar survey Bain conducted last year, expertise is a rising concern, while performance and security are waning worries.

Gene Rapoport, who leads Generative AI initiatives for Bain's Private Equity practice, said CEOs need to take more ownership of the implementation of AI tools. That's in part because most companies expect generative AI to bolster revenue and increase the productivity and efficiency of the employees they already have, but far fewer have a full understanding of how that will happen.

Driving costs down

Those in the trenches building AI tools may be less concerned, citing the power of the underlying tech and the path of past technological breakthroughs.

"There's a natural cycle where you invest in new technology and then you expect pay-off," said Oren Etzioni, an AI investor and professor emeritus at the University of Washington in Seattle. "I'm very optimistic about the ROI coming."

ROI can and must improve in two ways, he emphasized. An observable contribution to revenue is one, but costs coming down is another.

"As a field, computer scientists are so good at driving costs down. Even in the 19 months since it started, the cost per query has been documented to go down significantly and training is getting more efficient," Etzioni said.

Nvidia CEO Jensen Huang echoed this urgency in his Computex keynote speech in Taiwan earlier this month, decrying "computation inflation," wherein computing costs grow faster than AI model performance improves.

"This of course cannot continue," he said.

Got a tip or an insight to share? Contact Senior Reporter Emma Cosgrove at ecosgrove@businessinsider.com or use the secure messaging app Signal: 443-333-9088