Home / News / Technology / OpenAI’s Push for Custom AI Chip — Strategy To Reduce Soaring Cloud Costs?
Technology
3 min read

OpenAI’s Push for Custom AI Chip — Strategy To Reduce Soaring Cloud Costs?

Published
James Morales
Published

Key Takeaways

  • OpenAI is reportedly developing a custom AI chip in partnership with Broadcom.
  • The proposed hardware would handle AI inference workloads.
  • In 2024, OpenAI will spend $4 billion running its AI models on Microsoft’s cloud.

OpenAI is reportedly exploring the prospect of developing its own inference chip, joining a host of chip makers and other Big Tech players doubling down on the technology.

Amid surging demand for AI processing capacity, the move could help ease the burden on the firm’s skyrocketing cloud costs, which are projected to reach as much as  $7 billion in 2024.

OpenAI and Broadcom Exploring AI Chip Possibilities

On Tuesday, Oct. 29, Reuters reported  that OpenAI is working with Broadcom to design a new AI chip.

The proposed solution wouldn’t replace Nvidia GPUs, which OpenAI and its peers depend on for AI training, but would be used for inference—the process of running pre-trained AI models like the ones that generate ChatGPT outputs.

Although discussions are still in the early stages, the report said the two firms have already consulted with the Taiwan Semiconductor Manufacturing Company (TSMC) about a potential foundry partnership. (Broadcom is a “fabless” manufacturer that outsources production to foundries like TSMC.)

Meeting Infrastructure Demands

With the demand for AI inference rising, OpenAI isn’t the only company looking to build up its capacity with new semiconductor solutions. The Big Three cloud providers—Amazon Web Services (AWS), Google, and Microsoft—have also developed custom inference chips, as has Meta.

For the hyperscalers, the move is a logical response to an evolving cloud market that will help them adapt to customers’ changing requirements in the age of AI. 

Meanwhile, AI developers like Meta and OpenAI could benefit from reduced cloud costs by building custom data center solutions and realigning their relationships with infrastructure providers.

OpenAI’s $7 Billion Cloud Bill

In 2024, OpenAI is reportedly  on track to run up a $4 billion cloud bill from inference workloads alone, with another $3 billion in AI training costs on top of that.

The massive cloud expenses, which are nearly double the firm’s estimated revenues  for the year, are being channeled straight into Microsoft’s pocket. As part of their expansive relationship , Microsoft is OpenAI’s exclusive cloud provider, granting the AI company discounted rates on its Azure usage.

Was this Article helpful? Yes No

James Morales

Although his background is in crypto and FinTech news, these days, James likes to roam across CCN’s editorial breadth, focusing mostly on digital technology. Having always been fascinated by the latest innovations, he uses his platform as a journalist to explore how new technologies work, why they matter and how they might shape our future.
See more