- SemiAnalysis, a respected research firm, just divided the tech world into 2 groups.
- One group is "GPU-poor," with limited access to Nvidia's latest AI chips.
Nvidia makes the GPUs that are needed to train the most powerful AI models. If you can get lots of these chips, you've got a head start on most other companies. If you don't have serious supply, you're behind from the beginning.
It's no longer about saying "AI" as many times as possible on an earning conference call. You now actually have to have the tech components, other infrastructure, and a smart plan to deploy this incredibly expensive gear. Being at the front of the line for Nvidia GPUs and in the good graces of that company are table stakes now.
Dylan Patel and Daniel Nishball, at research firm SemiAnalysis, captured this situation in a compelling post this weekend. Much of this article is behind a paywall, so anyone who is serious about AI should subscribe to their great newsletter. I'm just sharing my thoughts on the free parts of the post here.
The GPU-Poor
Patel and Nishball divide the tech industry into 2 main groups: The "GPU-poor" and the "GPU-rich." The GPU-poor are mostly startups and open-source researchers who are struggling a limited supply of GPUs.
First up are European startups and government backed supercomputers such as Jules Verne, which the SemiAnalysis writers describe as "completely uncompetitive."
Then there are well-known AI firms, such as Hugging Face, Databricks, and Together, that are also GPU-poor.
There's a small in-between group, which has been buying loads of GPUs from Nvidia, but not making their money back: Cohere, Saudi Arabia, and UAE, according to SemiAnalysis.
The GPU-Rich
Patel and Nishball then list a handful of firms with more than 20,000 A100 and/or H100 GPUs from Nvidia.
The main leaders here are OpenAI, Google, Anthropic, Inflection, Elon Musk's X, and Meta, "who will have the highest ratios of compute resources to researchers," the authors wrote.
A few of those companies, as well as multiple Chinese firms, will have secured more than 100,000 GPUs by the end of 2024 next year. Meta will rank second in the world based on its number of H100 GPUs, they noted.
Who's the top GPU-rich company?
It's Google, according to SemiAnalysis. The internet giant is "the Most Compute Rich Firm In The World," with "unbeatably efficient architecture," Patel and Nishall wrote.
The company, which announced itself an AI-first enterprise years ago, will soon roll out its next huge AI model called Gemini, and is already training the next iteration. OpenAI watch out.
"Google has woken up, and they are iterating on a pace that will smash GPT-4 total pre-training FLOPS by 5x before the end of the year," the analysts wrote. "The path is clear to 20x by the end of next year given their current infrastructure buildout. Whether Google has the stomach to put these models out publicly without neutering their creativity or their existing business model is a different discussion."