Elon Musk: GPU Chip Shortages And Electricity Supply Critical Factor in AI Development
Are chip shortage and electricity supply slowing AI growth? l Source: Chesnot/Getty Images
Key Takeaways
- Training advanced AI models like Grok 2 requires significant computing power, with the lack of advanced chips a major hurdle.
- The increasing power demands of AI will soon outpace the current electricity supply, according to Elon Musk
- The Tesla CEO predicts artificial general intelligence (AGI) will exceed human intelligence within the next couple of years.
During an interview on X Elon Musk spoke with Norway’s wealth fund CEO Nicolai Tangen, touching upon how the availability of electricity impedes the progress of artificial intelligence (AI).
These factors directly impact Musk, as the billionaire encounters challenges in advancing Grok 2 amid the ongoing chip shortage.
Chip And Electricity Supply Issues
Elon Musk said that the scarcity of advanced chips has posed a significant challenge in training Grok’s version 2 model, requiring approximately 20,000 NVIDIA H100 GPUs.
In addition to chip shortages, Musk highlighted electricity supply as a major bottleneck in AI development. He stressed that while chip availability has been a prominent issue, the adequacy of electricity supply will become increasingly critical within the next year or two.
Last year, during a summit hosted by Pacific Gas & Electric in July 2023, Musk urged utilities and power system stakeholders to revise their expectations for electricity demand in a decarbonized future. Musk asserted that current demand predictions are likely too conservative and advocated for a greater sense of urgency in bringing projects online to meet the forthcoming surge in electricity demand.
While the US has experienced relatively stable electricity demand in recent years, Musk cautioned that as demand rises, there’s a pressing need to expedite the deployment of new projects. His primary concern lies in the lack of urgency and widespread misunderstanding regarding the magnitude of future electricity demand.
Development Of Grok
Elon Musk disclosed that the shortage of advanced chips has posed a significant hurdle in training Grok’s version 2 model. However, looking ahead, Musk anticipates that future models will demand even more resources. “The Grok 3 model and beyond will require 100,000 Nvidia H100 chips,” he revealed during the interview.
According to Oracle’s Larry Ellison, Elon Musk is among those highly interested in Nvidia’s semiconductors. Ellison revealed that Musk has chosen to utilize Oracle’s servers to support the operations of his xAI’s newly launched chatbot, Grok. Although the cloud infrastructure giant managed to supply enough Nvidia chips to power the initial version of Grok, Ellison noted that they fell short of Musk’s extensive requirements.
Last year, Musk founded xAI as a competitor to OpenAI, which he has sued for allegedly deviating from its original mission to develop AI for the betterment of humanity rather than for profit. OpenAI has refuted these allegations.
Musk sued OpenAI, accusing the company of becoming overly secretive. He subsequently announced his intention to make his Grok chatbot openly accessible. This move aims to distinguish xAI from OpenAI, which offers a basic version of ChatGPT and the underlying linguistic model for free but retains the code as proprietary.
AI To Become Smarter Then Humans By 2026
During his recent interview, Elon Musk predicted that artificial general intelligence (AGI) would likely surpass the intelligence of the smartest human soon. According to him, this will happen within the next year or by 2026 at the latest. He stated: “If you define AGI as smarter than the smartest human, I think it’s probably next year, within two years.”
Musk emphasized the unparalleled advancement of AI technology, remarking, “AI is the fastest-advancing technology I’ve seen of any kind, and I’ve seen a lot of technology.” He highlighted the exponential growth in AI hardware and dedicated computing resources, which are scaling at a rate of tenfold annually or even faster.
Expressing his confidence in his prediction, Musk reiterated, “My guess is we’ll have AI smarter than any one human around the end of next year.”
In response to a graph shared by X host Mario Nawfal illustrating the exponential growth of AI computational capacity since 1998, Musk affirmed: “The AI compute growth chart says it all.”
Musk had advocated for establishing a neutral oversight body tasked with monitoring AI development and signaling potential risks when necessary at Britain’s AI Safety Summit the previous year.
Elon Musk Gifts Blue Checkmark to Thousands of X Accounts: Confused Users Ask Why?
TSMC Secures $11.6B for US Chip Plants: Who’s Getting Remaining $18.7B?
