Prepare for a seismic shift in the AI realm as Nvidia introduces the HGX H200, a cutting-edge chip poised to redefine the boundaries of AI capabilities. This powerhouse chip, set to launch in the second quarter of 2024, boasts a remarkable 1.4 times more memory bandwidth and 1.8 times more memory capacity compared to its predecessor, the highly sought-after H100.
What sets the H200 apart is its groundbreaking use of HBM3e, a faster memory specification. This innovation elevates the GPU's memory bandwidth to an impressive 4.8 terabytes per second, up from the H100's 3.35 terabytes per second, and expands its total memory capacity to a staggering 141GB, a significant leap from the 80GB of the previous model.
Ian Buck, Nvidia’s VP of high-performance computing products, emphasized the impact of this upgrade, stating, "The integration of faster and more extensive HBM memory serves to accelerate performance across computationally demanding tasks, including generative AI models and high-performance computing applications, while optimizing GPU utilization and efficiency."
Crucially, the H200 maintains compatibility with systems supporting H100s, ensuring a seamless transition for users. Nvidia is collaborating with global system manufacturers and cloud service providers to make the H200 widely accessible. Amazon, Google, Microsoft, and Oracle are among the first to integrate these chips into their cloud services.
While the exact pricing remains undisclosed, previous-generation H100s were estimated to cost between $25,000 to $40,000 each. The demand for Nvidia's chips, driven by the exponential growth of generative AI, has led companies to use them as collateral for loans. The H200's arrival signals a promising future for GPU buyers, with Nvidia aiming to triple the production of the H100 in 2024, reaching up to 2 million units. As AI continues its explosive growth, the HGX H200 is poised to be a game-changer in the world of AI processing.
