- ServiceNow CIO Chris Bedi says embracing AI has sometimes been hard for businesses where managers and staff may resist or even reject these tools.
- While AI is now widely used in the consumer market, adoption is slower in enterprises, even though many companies recognize the benefits of the technology.
- This has led to the push for "explainable AI;" tools that help people who use AI systems understand how they work.
- "We all intuitively have trusted algorithms in our consumer lives," Bedi told Business Insider. "However, in the enterprise, one of my observations has been that blind trust in algorithms isn't there. Humans have a strong need to understand the 'why' behind an algorithm."
- Click here for more BI Prime stories.
Sign up here to receive updates on all things Innovation Inc.
From virtual assistants to navigation apps, AI has become a trusted tool used daily by millions of consumers. But ServiceNow CIO Chris Bedi has this cautionary tip for businesses thinking of introducing AI to its workforce: go slow.
ServiceNow, an enterprise cloud company valued at about $58 billion, uses AI itself to automate different business processes, such as the onboarding of clients or new employees, or the handling of customer queries and complaints.
While consumers now routinely rely on services like Amazon's Alexa or Google's Waze in their day-to-day lives, AI tools can be a tougher sell to business organizations where many managers and employees may distrust and even reject them.
This trend, which Bedi said he saw from conversations with the cloud software giant's customers, has underscored a need to strip away the "black box" nature of artificial intelligence systems and to emphasize what has been described as "explainable" AI that managers and rank-and-file staff can embrace more readily.
"We all intuitively have trusted algorithms in our consumer lives," Bedi told Business Insider. "However, in the enterprise, one of my observations has been that blind trust in algorithms isn't there. Humans have a strong and need to understand the 'why' behind an algorithm."
Trust is a big hurdle for AI in enterprises
The last few years saw the rise of AI-powered tools geared to helping businesses perform a wide range of tasks faster and more efficiently, from identifying sales leads, processing invoices, hiring, to developing marketing strategies.
A 2019 IDC survey found that half of global organizations considered embracing AI a priority, and two-thirds stressed the need for an "AI First" culture.
But trust remains a major hurdle for rolling out AI tools in enterprises, Bedi said. One problem is that managers and employees are not readily convinced that they will actually help them do their jobs more effectively.
"If they just get the output of the algorithm, without understanding the 'why,' that trust isn't there," Bedi said. "There can be a tendency to reject it."
A 2018 Gartner report on AI stressed the importance of building "trust," in rolling out AI tools to people who may not have the technical background to understand the nitty-gritty of the technology.
"Business stakeholders are not data scientists, so they need to understand the data that is driving AI model decisions in nontechnical terms," the report said.
Resentment toward AI
In some ways, the rejection is based on resentment, Bedi said. Introducing a new tool that purportedly could perform specific tasks faster and even better may alienate managers and staff who've been doing those tasks for years.
A typical reaction, Bedi said, could go something like this: "I've been doing my job for a year and I've reached a certain position because I'm really, really good at my job. Now, you're telling me this algorithm knows a little better?"
"That's threatening to people," he added. That is also why "explainable AI is a concept in the industry now for a while," he added.
Explainable AI stresses the need for tools or systems that help managers and employees understand how an AI tool works and how it comes up with its conclusions and outcomes.
"Sort of like an AI on top of AI," Bedi said.
In fact, new startups focused on explainable AI have emerged. One of them, Fiddler Labs, also aims to help businesses ensure that their AI tools are leading to biased outcomes, that may even violate regulations, especially in regulated industries, such as health care and financial services.
"There are regulations coming up on AI which means companies need to build this transparency for their customers and regulators," Fiddler Labs founder Krishna Gade told Business Insider. "Increasingly, machine learning and AI are seen as some sort of business risk…you need to make sure that it doesn't have bias in it, or doesn't have concerns that could cause customers to mistrust your products."
In fact, this push for explainable AI can create a dilemma for some businesses, Bedi said. That's because the more sophisticated and powerful an AI tool is, the more complicated is the algorithm behind it, and the harder it is to explain to managers and staff.
That's why businesses may at times find themselves with a tough choice, Bedi said. They can opt for "a good enough model which people can in their brains internalize which can lead to adoption. Or they can embrace a "great model" that's harder for people to understand and leads to "no adoption" and end up being "useless because people don't trust it."
Got a tip about ServiceNow or another tech company? Contact this reporter via email at bpimentel@businessinsider.com, message him on Twitter @benpimentel or send him a secure message through Signal at (510) 731-8429. You can also contact Business Insider securely via SecureDrop.