Nvidia has a stranglehold on AI companies as a chip shortage begins to bite. This rival CEO is eager to offer an alternative.

• Nvidia's GPUs have become a vital resource in the race to develop AI models like GPT-4.
• Huge demand for GPUs has created a shortage that risks stunting AI development.

If there was one company to rule them all in the generative AI boom of 2023, that company might well be Nvidia.

Few exert as much power and influence as the $1 trillion chip behemoth because of its graphics processing units (GPUs), which have proven to be the vital source of the compute power needed to train the large language models behind apps like OpenAI's ChatGPT and Google's Bard.

Demand for Nvidia's GPUs, like the $40,000 H100, have been so great that CEO Jensen Huang is reportedly preparing to triple production next year in the face of shortages. Nation states have been scrambling to secure supplies, while black markets for the processors have sprung up in recent months.

In other words, Nvidia has a stranglehold on companies looking to accelerate the development of their AI models in ways that could revolutionize how the entire economy operates.

One rival is desperate to offer an alternative to those feeling the squeeze.

Nigel Toon, CEO of semiconductor firm Graphcore, told Insider in an interview that his company was "at that tipping point now" in the development of its AI-ready processors, to the extent that he feels Graphcore can start to be "seen as a real alternative to Nvidia."

Graphcore's fundamental proposition is a whole new type of processor called an IPU, short for intelligent processing unit. Toon described it as a piece of technology that massively boosts the number-crunching power involved in handling data fed into AI models.

"An IPU does multiple instructions on multiple pieces of data all in parallel and orchestrates how those come together as a complete compute product," he said. "So that's how we end up being different from a GPU."

Having a more advanced bit of hardware isn't enough to pry companies away from Nvidia. Toon said Nvidia's biggest selling point is its CUDA software, which works as a simple plug-and-play system for companies looking to use their technology.

Graphcore is at an "expensive" incubation phase as it tests its processors with select customers. It's also working on its own software, known as Poplar, to offer that same kind of plug-and-play usability as Nvidia. Losses totaled $184.5 million in 2021, the most recent filings showed.

If successful, Toon sees 2024 as an opportune moment to drive forward with a big sales push of Graphcore's IPUs, particularly as he sees GPU shortages continuing to grip the AI market through next year.

"We're at that point where the technology's working, the technology's maturing, and we'll be able to start broadening the customers quite rapidly through next year," he said. "Customers are still going to be short next year, and so there's a massive opportunity for us."

How successful he will be in prying away customers from Nvidia remains unclear. Insider first reported that venture capital giant Sequoia had written down its stake in Graphcore last year, while regulatory concerns over sales to China persist.

Toon also acknowledged that big tech firms were trying to bypass the likes of Graphcore by making their own chips in-house.

But he believes these companies would face difficulties given the complexity of building such specialized technology, forcing a large number of them into a desperate rethink.

"I think a lot of them will fall into that camp and they'll actually come back and say 'gosh, we're facing the prospect of spending billions with Nvidia, and we need an alternative'," Toon said. "That's where we come in."