Key Takeaways
OpenAI is reportedly exploring the prospect of developing its own inference chip, joining a host of chip makers and other Big Tech players doubling down on the technology.
Amid surging demand for AI processing capacity, the move could help ease the burden on the firm’s skyrocketing cloud costs, which are projected to reach as much as $7 billion in 2024.
On Tuesday, Oct. 29, Reuters reported that OpenAI is working with Broadcom to design a new AI chip.
The proposed solution wouldn’t replace Nvidia GPUs, which OpenAI and its peers depend on for AI training, but would be used for inference—the process of running pre-trained AI models like the ones that generate ChatGPT outputs.
Although discussions are still in the early stages, the report said the two firms have already consulted with the Taiwan Semiconductor Manufacturing Company (TSMC) about a potential foundry partnership. (Broadcom is a “fabless” manufacturer that outsources production to foundries like TSMC.)
With the demand for AI inference rising, OpenAI isn’t the only company looking to build up its capacity with new semiconductor solutions. The Big Three cloud providers—Amazon Web Services (AWS), Google, and Microsoft—have also developed custom inference chips, as has Meta.
For the hyperscalers, the move is a logical response to an evolving cloud market that will help them adapt to customers’ changing requirements in the age of AI.
Meanwhile, AI developers like Meta and OpenAI could benefit from reduced cloud costs by building custom data center solutions and realigning their relationships with infrastructure providers.
In 2024, OpenAI is reportedly on track to run up a $4 billion cloud bill from inference workloads alone, with another $3 billion in AI training costs on top of that.
The massive cloud expenses, which are nearly double the firm’s estimated revenues for the year, are being channeled straight into Microsoft’s pocket. As part of their expansive relationship , Microsoft is OpenAI’s exclusive cloud provider, granting the AI company discounted rates on its Azure usage.