Why it matters: The crypto boom caused a severe spike in PC graphics card prices throughout 2020 and 2021, and prices still aren’t where many users would like them. The current AI craze could have a similar effect as some companies report shortages, but Nvidia says the supply chain is more complex than many assume.
Recent statements from AI companies indicate that the AI boom is increasing the demand for GPUs to the point where some report shortages. Nvidia claims its supply chains haven’t totally failed to meet demand but are encountering minor snags, somewhat allaying fears.
Microsoft, OpenAI, and other companies mention steps they’ve had to take to mitigate shortages in Nvidia H100 and A100 enterprise GPUs, which are used for AI workloads. Microsoft is rate-limiting employees and the CEO of Quora says the hardware shortage is hiding the real potential of AI applications. Elon Musk joked that enterprise GPUs are currently harder to acquire than drugs.
AMD (unlike Qualcomm) is a joy to buy chips from. 7.38 PFLOPS of compute was delivered today, with exaflops more to come.
Nice to see them more active on the ROCm GitHub, we are working on the layer above @__tinygrad__ pic.twitter.com/5yE11TVA5F
– George Hotz �’ (@realGeorgeHotz) August 1, 2023
Nvidia explained that oftentimes, the manufacturing hangup with GPUs doesn’t involve the main processing unit but rather some other component of the add-in board. In the current case, the problem is packaging, which the company currently handles with TSMC’s 3D stacking chip-on-wafer-on-substrate (CoWoS) technology. Nvidia promises that supply will be much stronger in the second half of 2023.
It’s unclear how much the situation could spill over into the consumer GPU market, but the AI boom has already caused Nvidia to divert some production resources from GeForce RTX 4090 graphics cards toward H100s. While comments from AI companies imply that shifting attention toward AMD hardware would take time, a post from prolific hacker George Hotz indicates that his company, comma, is purchasing Radeon 7900 XTX cards for that purpose.
The Green Team is currently dominant in AI, but TechSpot’s analysis suggests AMD can certainly make significant moves in the sector. The company’s primary advantage in AI is its lead in Windows PC CPUs that incorporate AI blocs.
Earlier this year, groups that had previously used GPUs for mining began attempts to pivot their infrastructure toward AI workloads to recoup investments after the crypto winter. Although the shift may prove fruitful for some, it will likely be more challenging than mining due to competition from major players like Microsoft, Amazon Web Services, and OpenAI.
