| Precision | Peak (PFLOPS) | Per GPU (TFLOPS) | TFLOPS/W |
|---|---|---|---|
| FP64 | 2.7 | 37.0 | 27.00 |
| FP32 | 5.4 | 75.0 | 54.00 |
| TF32 | 79.2 | 1100.0 | 1500.00 |
| FP16 | 162.0 | 2250.0 | 2700.00 |
| BF16 | 162.0 | 2250.0 | 2700.00 |
| FP8 | 324.0 | 4500.0 | 5400.00 |
| FP4 | 648.0 | 9000.0 | 10800.00 |
| INT8 | 324.0 | 4500.0 | 5400.00 |
All figures are dense. Vendors commonly headline the sparse number, which is twice the dense one.
NVLink
36 GB200 superchips (72 B200 GPUs + 36 Grace CPUs) in a liquid-cooled rack-scale system