Nvidia Brings Back Dual GPU for Data Centers

Nvidia introduced a brand new dual-GPU product, the H100 NVL, at its GTC Spring 2023 keynote. This would possibly not deliver again SLI or multi-GPU gaming, and it will not be top-of-the-line graphics playing cards for gaming, as an alternative concentrating on the rising AI market. Based on data and pictures launched by Nvidia, the H100 NVL (H100 NVLink) could have three NVLink connectors on the prime, with two adjoining playing cards positioned in separate PCIe slots.

An fascinating pace change that appears to focus extra on inference efficiency than coaching for internet hosting servers that do not help Nvidia’s SXM possibility. NVLink connections ought to assist make up for the lacking bandwidth that NVSwitch supplies in SXM options, and there are another key variations.

Get the specs. Earlier H100 options – each SXM and PCIe – got here with 80GB of HBM3 reminiscence, however the precise bundle consists of six stacks, every with 16GB of reminiscence. It is not clear if a stack was disabled fully or if it was for ECC or another function. What we do know is that the H100 NVL will include 94GB per GPU and a complete of 188GB of HBM3. We assume the “lacking” 2GB per GPU is now for ECC.

Energy is barely larger than the H100 PCIe, 350–400 watts per GPU (configurable), a rise of fifty W. The general efficiency, in the meantime, is successfully double that of the H100 SXM: 134 teraflops FP64, 1,979 teraflops TF32 and seven,916 teraflops FP8 (plus 7,916 teraflops INT8).

Mainly this appears to be like like the identical core design of the H100 PCIe, which additionally helps NVLink, however probably now has extra GPU cores enabled and has 17.5% extra reminiscence. Reminiscence bandwidth can also be barely larger than H100 PCIe (2TB/s for H100 PCIe and three.35TB/s for H100 SXM), with 3.9TB/s per GPU and seven.8TB/s mixed.

Since it is a twin card answer, every card occupies an space of ​​2 slots, Nvidia solely helps 2 to 4 twin H100 NVL playing cards for frequent and authorized methods. How a lot does a single pair value and may or not it’s bought individually? Though typically a single H100 PCIe may be discovered, this has but to be seen. about $28,000 (opens in new tab). So $80,000 for a pair of H100 NVLs is out of the query.