In September, Amazon said it would invest up to $4 billion in Anthropic, a San Francisco startup working on artificial intelligence.
Shortly after, an Amazon executive sent a private message to an executive at another company. He said Anthropic had won the deal because it agreed to build its ai using specialized computer chips designed by Amazon.
Amazon, he wrote, wanted to create a viable competitor to chipmaker Nvidia, a key partner and kingmaker in the all-important field of artificial intelligence.
The rise of generative ai over the past year exposed how dependent big tech companies had become on Nvidia. They can't build chatbots and other ai systems without a special type of chip that Nvidia has mastered in recent years. They've spent billions of dollars on Nvidia systems and the chipmaker hasn't been able to keep up with demand.
That's why Amazon and other industry giants, including Google, Meta and Microsoft, are building their own ai chips. With these chips, technology giants could control their own destiny. They could control costs, eliminate chip shortages, and eventually sell access to their chips to companies that use their cloud services.
While Nvidia sold 2.5 million chips last year, Google spent between $2 billion and $3 billion building about a million of its own ai chips, said Pierre Ferragu, an analyst at New Street Research. Amazon spent $200 million on 100,000 chips last year, he estimated. Microsoft said it had begun testing its first ai chip.
But this job is a balancing act between competing with Nvidia while also working closely with the chipmaker and its increasingly powerful CEO, Jensen Huang.
Mr. Huang's company accounts for more than 70 percent of ai chip sales, according to research firm Omdia. It provides an even larger percentage of the systems used in creating generative ai. Nvidia's sales have soared 206 percent over the past year and the company has added about $1 trillion in market value.
What is revenue for Nvidia is costs for the technology giants. Orders from Microsoft and Meta accounted for about a quarter of Nvidia's sales in the last two full quarters, said Gil Luria, an analyst at investment bank DA Davidson.
Nvidia sells its chips for about $15,000 each, while Google spends an average of just $2,000 to $3,000 on each, according to Ferragu.
“When they encountered a vendor holding them over a barrel, they reacted very strongly,” Luria said.
Companies constantly court Mr. Huang, competing to be at the front of the line to receive his tokens. He regularly appears on event stages with his CEOs, and companies are quick to say they remain committed to their partnerships with Nvidia. They all plan to continue offering their chips along with yours.
While big tech companies are getting into Nvidia's business, Nvidia is getting into theirs. Last year, Nvidia launched its own cloud service where businesses can use its chips, and it is funneling chips to a new wave of cloud providers, such as CoreWeave, that compete with the big three: Amazon, Google and Microsoft.
“The tensions here are a thousand times greater than the usual jockeying between customers and suppliers,” said Charles Fitzgerald, a technology consultant and investor.
Nvidia declined to comment.
The ai chip market is expected to more than double by 2027, reaching approximately $140 billion, according to research firm Gartner. Venerable chipmakers like AMD and Intel are also building specialized ai chips, as are startups like Cerebras and SambaNova. But Amazon and other tech giants can do things that smaller competitors can't.
“In theory, if they can reach high enough volume and can reduce their costs, these companies should be able to offer something that is even better than Nvidia,” said Naveen Rao, who founded one of the first ai chip startups. oops and then sold it to Intel.
Nvidia builds what it calls graphics processing units, or GPUs, which it originally designed to help render images for video games. But a decade ago, academic researchers realized that these chips were also very good at building the systems, called neural networks, that now power generative ai.
As this technology took off, Huang quickly began modifying Nvidia chips and related software for ai, and they became the de facto standard. Most of the software systems used to train ai technologies were designed to work with Nvidia chips.
“Nvidia has great chips and, more importantly, it has an incredible ecosystem,” said Dave Brown, who heads Amazon's chip efforts. That makes getting customers to use a new type of ai chip “very, very challenging,” he said.
Rewriting software code to use a new chip is so difficult and time-consuming that many companies don't even try, said Mike Schroepfer, an advisor and former chief technology officer at Meta. “The problem with technological development is that much of it dies before it even begins,” he said.
Rani Borkar, who oversees Microsoft's hardware infrastructure, said Microsoft and its peers needed customers to be able to move between chips from different companies in a “seamless” manner.
Amazon, Brown said, is working to make switching between chips “as simple as possible.”
Some tech giants have had success making their own chips. Apple designs the silicon in iPhones and Macs, and Amazon has deployed more than two million of its own traditional server chips in its cloud computing data centers. But achievements like these require years of hardware and software development.
Google has the biggest advantage in ai chip development. In 2017, it introduced its tensor processing unit, or TPU, named after a type of calculation vital for building artificial intelligence. Google used tens of thousands of TPUs to create artificial intelligence products, including its online chatbot, Google Bard. And other companies have used the chip through Google's cloud service to create similar technologies, including high-profile startup Cohere.
Amazon is now in the second generation of Trainium, its chip for building ai systems, and has a second chip built exclusively for delivering ai models to customers. In May, Meta announced plans to work on an ai chip tailored to its needs, although it is not yet in use. In November, Microsoft announced its first ai chip, Maia, which will initially focus on running Microsoft's own ai products.
“If Microsoft builds its own chips, it builds exactly what it needs at the lowest possible cost,” Luria said.
Nvidia's rivals have used their investments in high-profile artificial intelligence startups to boost use of its chips. Microsoft has committed $13 billion to OpenAI, the maker of the ChatGPT chatbot, and its Maia chip will serve OpenAI technologies to Microsoft customers. Like Amazon, Google has invested billions in Anthropic and is also using Google's artificial intelligence chips.
Anthropic, which has used chips from Nvidia and Google, is among a handful of companies working to build ai using as many specialized chips as they can get their hands on. Amazon said that if companies like Anthropic were to use Amazon chips on an increasing scale and even help design future chips, doing so could reduce the cost and improve the performance of these processors. Anthropic declined to comment.
But none of these companies will surpass Nvidia anytime soon. Their chips may be expensive, but they are among the fastest on the market. And the company will continue to improve its speed.
Rao said his company, Databricks, trained some experimental ai systems using Amazon's ai chips, but built its largest and most important systems using Nvidia chips because they provided higher performance and worked well with a wider range of software.
“We have many years of hard innovation ahead of us,” said Amazon's Brown. “Nvidia will not sit still.”
