| Precision | Peak (PFLOPS) | Per GPU (TFLOPS) | TFLOPS/W |
|---|---|---|---|
| FP64 | 1.28 | 40 | 27.00 |
| FP32 | 2.56 | 80 | 54.00 |
| TF32 | 40.0 | 1,250 | 1501.00 |
| FP16 | 80.0 | 2,500 | 2701.00 |
| BF16 | 80.0 | 2,500 | 2701.00 |
| FP8 | 160 | 5,000 | 5403.00 |
| FP4 | 320 | 10,000 | 10806.00 |
| INT8 | 160 | 5,000 | 5403.00 |
All figures are dense. Vendors commonly headline the sparse number, which is twice the dense one.
NVLink
16 GB200 superchips (32 B200 GPUs + 16 Grace CPUs) in a rack-scale system