It looks like we might see another crypto-like GPU shortage of gaming GPUs, only this time it will be caused by the growing demand for AI. The shortage of AI GPUs in China has resulted in companies setting their sight on gaming GPUs, and things can take a turn here.
The Acquisition of AI-specific GPUs Has Become a Nuisance For Tech Startups, Interest Shifting Towards Gaming GPUs
We previously reported that the shortages of GPUs, such as NVIDIA's H100, are rapidly increasing. NVIDIA cannot fulfill the immense demand from its market, due to which its AI GPUs are being sold at tremendous costs, reaching almost USD $70,000. While this has caused hindrances to acquiring GPUs, some companies have found another way to tackle the shortage: getting consumer graphics cards.
While there isn't much evidence behind the shift in interest, a recent tweet from the CEO of Comma AI has revealed that the company has brought AMD gaming GPUs
AMD (unlike Qualcomm) is a joy to buy chips from. 7.38 PFLOPS of compute was delivered today, with exaflops more to come.
Nice to see them more active on the ROCm GitHub, we are working on the layer above @__tinygrad__ pic.twitter.com/5yE11TVA5F
— George Hotz
(@realGeorgeHotz) August 1, 2023
The image shared above reveals a large batch of AMD GPUs and upon zooming the image, the model is the Radeon RX 7900 XTX from XFX "MERC 310". While the exact number of GPUs brought isn't specified, based on the image and depending on the metric used to calculate performance, it should either be 60 GPUs if FP16 was used and 120 GPUs if FP32 was used. Since AI is mostly FP16 dependent, we can say that there were a total of 60 GPUs, delivering a total of 7.38 PTFLOPS ( as George said). The Radeon RX 7900 XTX retails for around USD $900-1000, hence the total cost reaches up to $60,000. George says that there are Exaflops more to come.
You might wonder about the choice of AMD gaming GPUs. Well, there are two main reasons, the first being high VRAM. AI workloads are required to have significant memory capacities, and since the Radeon RX 7900 XTX is the highest-end offering from AMD, it is seen as the best option. The 24 GB VRAM comes in at a much lower price compared to NVIDIA's top offering which retails for $1599 US and above.
Secondly, as stated by George himself ("AMD (unlike Qualcomm) is a joy to buy chips from"), acquiring AI GPUs has been a pain for tech startups since only the industry players such as OpenAI and Microsoft have exclusive access to NVIDIA's inventory. AMD gaming GPUs haven't gained as much sales traction as NVIDIA's Ada Lovelace lineup; hence high-end offerings from AMD are easier to access, and who knows, maybe George got a good discount too.
An argument about " Will AI cause GPU shortage" isn't feasible right now, but we have reported several times about order backlogs and time delays for AI GPUs. This has created the opportunity for companies to look at gaming GPUs, and as well all know, when NVIDIA and AMD see such an opportunity, they run at it as fast as light. We might never know; it may be a possibility that we might see another "LHR" and "NON-LHR" category, but this time, for AI computing.
PugetSystems released benchmarks of AMD and NVIDIA consumer-tier GPUs within Stable Diffusion and the results are very interesting, You can see that the latest Radeon & GeForce graphics cards for gaming offer exceptional performance which gives AI companies even more incentive to go after gaming GPUs that offer better value than the Pro and HPC parts while being readily available on retail shelves.
In the end, the fact is that the PC market is at rock bottom right now so anything companies including NVIDIA and AMD will take anything that can push more sales of the hardware. Who knows even the RTX 4060 Ti 16 GB may become a success in the AI market (while it's not so much in the gaming segment).
Refference- https://wccftech.com
0 Comments