Microsoft is reportedly working on its own AI chips that can be used to train large language models and avoid a costly reliance on Nvidia. The Information reports that Microsoft has been developing the chips in secret since 2019, and some Microsoft and OpenAI employees already have access to them to test how well they perform for the latest large language models like GPT-4.

Nvidia is the key supplier of AI server chips right now, with companies racing to buy up these chips and estimates suggesting OpenAI will need more than 30,000 of Nvidias A100 GPUs for the commercialization of ChatGPT. Nvidias latest H100 GPUs are selling for more than $40,000 on eBay, illustrating the demand for high-end chips that can help deploy AI software.

While Nvidia races to build as many as possible to meet demand, Microsoft is reportedly looking in-house and hoping it can save money on its AI push. Microsoft has reportedly accelerated its work on codename Athena, a project to build its own AI chips. While its not clear if Microsoft will ever make these chips available to its Azure cloud customers, the software maker is reportedly planning to make its AI chips available more broadly inside Microsoft and OpenAI as early as next year. Microsoft also reportedly has a road map for the chips that includes multiple future generations.

Microsofts custom SQ1 processor. Photo by Amelia Holowaty Krales / The Verge

Microsofts own AI chips arent said to be direct replacements for Nvidias, but the in-house efforts could cut costs significantly as Microsoft continues its push to roll out AI-powered features in Bing, Office apps, GitHub, and elsewhere.

Microsoft has also been working on its own ARM-based chips for several years. Bloomberg reported in late 2020 that Microsoft was looking at designing its own ARM-based processors for servers and possibly even a future Surface device. We havent seen those ARM chips emerge yet, but Microsoft hasworked with AMD and Qualcommfor custom chips for its Surface Laptop and Surface Pro X devices.

If Microsoft is working on its own AI chips, it would be the latest in a line of tech giants. Amazon, Google, and Meta also have their own in-house chips for AI, but many companies are still relying on Nvidia chips to power the latest large language models.

More here:

Microsoft reportedly working on its own AI chips that may rival Nvidia's - The Verge