Nvidia: H100 AI Performance Improved Up To 54 Percent With Software Optimizations

Nvidia has launched some new efficiency numbers for its H100 compute GPU. MLPerf 3.0, the newest launch of a key benchmark for deep studying workloads. The Hopper H100 processor not solely outperforms its predecessor A100 in coaching time measurements, but additionally positive factors efficiency due to software program optimizations. Additionally, Nvidia revealed early efficiency comparisons of the compact L4 compact computing GPU with its predecessor, the T4 GPU.

Nvidia first revealed the outcomes of the H100 take a look at. MLPerf 2.1 comparison In September 2022, it was revealed that its flagship compute GPU might beat its predecessor, the A100, by as much as 4.3-4.4x at varied inference workloads. The newly launched efficiency figures from MLPerf 3.0 not solely verify (no shock) that Nvidia’s H100 is quicker than the A100, nevertheless it’s additionally higher than Intel’s lately launched Xeon Platinum 8480+ (Sapphire Rapids) processor and NeuChips It reaffirms that it’s also tangibly sooner than . ReccAccel N3000 and Qualcomm’s Cloud AI 100 options throughout a variety of workloads

These workloads embody picture classification (ResNet 50 v1.5), pure language processing (BERT Giant), speech recognition (RNN-T), medical imaging (3D U-Internet), object detection (RetinaNet), and advice (DLRM). will get. Nvidia states that not solely are their GPUs sooner, however in addition they have higher assist within the machine studying business – some workloads have failed competing options.

There’s something incorrect with the numbers launched by Nvidia, although. Sellers have the choice to submit MLPerf ends in two classes: closed and open. Within the closed class, all distributors should run mathematically equal neural networks, whereas within the open class, they will swap networks to optimize the efficiency of their {hardware}. Nvidia’s figures solely replicate the closed class, so any optimizations that Intel or different distributors can apply to optimize the efficiency of their {hardware} usually are not mirrored in these group outcomes.

As Nvidia’s personal instance demonstrates, software program optimizations can convey big advantages to trendy AI {hardware}. The corporate’s H100 elevated from 7% on advice workloads to 54% on object detection workloads in MLPerf 2.1 versus MLPerf 3.0, which is a fairly large efficiency enhance.

Referring to the explosion of ChatGPT and related providers, Dave Salvator, Director of Synthetic Intelligence, Benchmark and Cloud at Nvidia, writes in a weblog publish: “On this iPhone second of AI, inference efficiency is important… Deep studying is now deployed virtually all over the place, and the manufacturing unit led to an insatiable want for inference efficiency, from flooring to on-line advice techniques.”

Along with reaffirming that the H100 is the king of extraction efficiency in MLPerf 3.0, the corporate additionally introduced that the lately launched AD104 based L4 compute GPU (opens in new tab). This Ada Lovelace-powered computing GPU card is available in a single-slot, low-profile kind issue to slot in any server, but delivers fairly superior efficiency: as much as 30.3 FP32 TFLOPS and as much as 485 FP8 for basic computing. TFLOPS (not often) ).

Nvidia solely in contrast its L4 to one in all its different compact knowledge heart GPUs, the T4. The latter is predicated on the 2018 Turing structure TU104 GPU, so relying on the workload it’s not stunning that the brand new GPU is 2.2–3.1 occasions sooner than its predecessor in MLPerf 3.0.

“Along with stellar AI efficiency, L4 GPUs supply as much as 10x sooner picture decoding, as much as 3.2x sooner video processing, and over 4x sooner graphics and real-time rendering efficiency,” wrote Salvator.

The benchmark outcomes for Nvidia’s H100 and L4 compute GPUs already supplied by main system producers and cloud service suppliers look undoubtedly spectacular. Observe, although, that we’re coping with benchmark numbers revealed by Nvidia itself, somewhat than impartial checks.