How to Calculate TFLOPS

The formula

One TFLOPS is one trillion floating-point operations per second. Peak throughput for a single GPU is the number of cores, multiplied by the clock they run at, multiplied by the operations each core completes per cycle:

Every term has to describe the same hardware. Count the cores that actually execute the precision you are asking about, use the boost clock the vendor quotes for that part, and remember that the result is a ceiling the silicon can reach, not a number any real workload sustains.

A worked example

The NVIDIA A100 SXM4 has 6,912 FP32 CUDA cores and a 1,410 MHz boost clock. Both figures are on our A100 spec page, so you can check the inputs before you trust the output.

NVIDIA publishes 19.5 TFLOPS of FP32 for this part, and the arithmetic lands on the same number. That is the check worth doing: if your result and the vendor's disagree by more than a rounding error, one of your three inputs describes different hardware.

Why the answer changes with precision

The formula above describes the shader cores. Modern datacenter GPUs also carry tensor units that process whole matrix tiles per instruction rather than single values, so their throughput is not derived from the core count at all. Halving the precision roughly doubles the rate, because twice as many values fit through the same datapath.

This is why a hand-calculated FP32 figure looks nothing like the headline number on a product page. The headline is almost always a tensor-core precision, and increasingly an 8-bit or 4-bit one. When you compare two accelerators, compare the same precision on the same basis, or the comparison means nothing.

Peak is not what you get

The measure that matters is model FLOPS utilisation: the share of peak a real workload actually sustains once memory bandwidth, interconnect and kernel efficiency have taken their cut. Large-model training typically lands between 40 and 60 percent. Inference varies far more widely, and small batch sizes can fall below 10 percent.

So a cluster quoted at 8 petaFLOPS of BF16 might deliver 3.5 in practice. Size against the utilisation you expect to achieve, not against the peak, and treat any vendor comparison that quotes only peak as a starting point rather than an answer.

Scaling to a cluster

Aggregate peak is simply the per-GPU figure multiplied by the accelerator count. Eight H100 SXM5 give 8 × 990 = 7,920 TFLOPS of dense BF16. Apply your expected utilisation to that, and add the power figure, because at datacenter scale power is the constraint that binds before compute does.

Related