Intel Gaudi 2 96GB vs NVIDIA A100 SXM4 80GB

The Gaudi 2 delivers 38% more FP16 throughput than the A100 (432 vs 312 TFLOPS dense). The Gaudi 2 also carries 16 GB more memory (96 GB vs 80 GB).

Gaudi 2: Gaudi 2, 2022 A100: Ampere, 2020

FP16 dense lead

+38%

Gaudi 2: 432 vs 312 TFLOPS

Memory

96 vs 80 GB

16 GB more for the Gaudi 2

Bandwidth

2.5 TB/s vs 2.0 TB/s

Gaudi 2 moves data faster

TDP

600 W vs 400 W

A100 draws 200 W less

See rental pricing from $0.55/hr Download comparison sheet Printable version →

Intel Gaudi 2 96GB

Leads on FP16 Full specs →

What is the Intel Gaudi 2?

The Gaudi 2 is an ASIC from Intel on the Gaudi 2 architecture, an accelerator built for AI workloads. It is an OAM module, the open accelerator format used in multi-GPU servers. Gaudi 2 full specifications →

Can I rent the Gaudi 2 in the cloud?

Yes. Cloud providers tracked by Flopper rent the Gaudi 2 by the hour, and the pricing page compares their current rates and the price history side by side. Gaudi 2 rental prices →

How does the Gaudi 2 compare with other accelerators?

This page sets the Gaudi 2 against the A100. The Gaudi 2 is the newer generation of the two. This is a cross-vendor matchup, Intel against NVIDIA; the Gaudi 2 is an OAM module while the A100 is an SXM module. The compare tool builds the same comparison for any two parts in the catalogue. All GPU comparisons →

Where do the Gaudi 2 figures come from?

Every Gaudi 2 figure on this page traces to Intel's own datasheet or specification table. How Flopper verifies specifications →

NVIDIA A100 SXM4 80GB

Full specs →

What is the NVIDIA A100?

The A100 is a GPU from NVIDIA on the Ampere architecture, an accelerator for AI training, inference and high-performance computing. It is an SXM module, the mezzanine format used in dense multi-GPU servers. A100 full specifications →

Can I rent the A100 in the cloud?

Yes. Cloud providers tracked by Flopper rent the A100 by the hour, and the pricing page compares their current rates and the price history side by side. A100 rental prices →

How does the A100 compare with other accelerators?

This page sets the A100 against the Gaudi 2. The A100 is the older generation of the two. This is a cross-vendor matchup, NVIDIA against Intel; the A100 is an SXM module while the Gaudi 2 is an OAM module. Flopper also compares it head to head with the H100 SXM5 80GB, B200 SXM 180GB and A100 PCIe 40GB, across every published precision, memory and power. A100 vs H100 SXM5 80GB →

Where do the A100 figures come from?

Every A100 figure on this page traces to NVIDIA's own datasheet or specification table. How Flopper verifies specifications →

Specifications

Gaudi 2

A100

Architecture

Gaudi 2Ampere

Launch Year

20222020

Form Factor

OAMSXM

Memory

96 GB80 GB

Memory Bandwidth

2.5 TB/s2.0 TB/s

TDP

600 W400 W

Process Node

7nm7nm

Spec

Gaudi 2

A100

Architecture Gaudi 2Ampere

Launch Year 20222020

Form Factor OAMSXM

Memory 96 GB80 GB

Memory Bandwidth 2.5 TB/s2.0 TB/s

TDP 600 W400 W

Process Node 7nm7nm

Performance (TFLOPS)

Gaudi 2

A100

9.7 TFLOPS

45 TFLOPS 20 TFLOPS

362 TFLOPS 156 TFLOPS

312 TFLOPS sparse

432 TFLOPS 312 TFLOPS

624 TFLOPS sparse

432 TFLOPS 312 TFLOPS

624 TFLOPS sparse

865 TFLOPS

FP6

865 TOPS 624 TOPS

1,248 TOPS sparse

Precision

Gaudi 2

A100

9.7 TFLOPS

45 TFLOPS 20 TFLOPS

362 TFLOPS 156 TFLOPS

312 TFLOPS sparse

432 TFLOPS 312 TFLOPS

624 TFLOPS sparse

432 TFLOPS 312 TFLOPS

624 TFLOPS sparse

865 TFLOPS

FP6

865 TOPS 624 TOPS

1,248 TOPS sparse

Figures are dense. Where a vendor publishes a figure it sits under the dense one and never decides a winner. Hover any precision for its definition, or read the GPU glossary.

FLOPS by Precision

What actually differs

The Gaudi 2 is the newer part: Gaudi 2 (2022) against the A100's Ampere (2020).

The Gaudi 2 carries 16 GB more memory (96 GB against 80 GB), so larger models fit on one GPU before you shard. Bandwidth is 2.5 TB/s against 2.0 TB/s, which sets the ceiling on decode speed.

Power is 600 W against 400 W, and at FP32 the Gaudi 2 does more work per watt. Weight the FP8 and FP16 rows for LLM work and the FP64 row for HPC.

New to the terms? The glossary explains FP64, FP32, TF32, BF16, FP16, FP8, INT8, Sparsity and TFLOPS. For how the generations relate, see NVIDIA architectures explained.

Frequently asked questions

Is the Gaudi 2 faster than the A100?

At FP16 precision the Gaudi 2 reaches 432 TFLOPS dense against 312 TFLOPS for the A100. The performance table on this page lists every published precision for both GPUs.

Which should you choose, the Gaudi 2 or the A100?

Choose the Gaudi 2 for training and high-throughput inference: it has 38% more FP16 throughput than the A100. It also carries 16 GB more memory, so larger models fit on one GPU before you shard. The Gaudi 2 draws 200 W more power, so budget for cooling and power delivery before you commit. Packaging differs too: the Gaudi 2 is an OAM module while the A100 is an SXM module, so check what your servers accept.

Which has more memory, the Gaudi 2 or the A100?

The Gaudi 2 carries 96 GB of memory versus 80 GB for the A100. Memory bandwidth is 2.5 TB/s for the Gaudi 2 and 2.0 TB/s for the A100.

How much power do the Gaudi 2 and the A100 draw?

The Gaudi 2 is rated at 600 W TDP and the A100 at 400 W. On FP32 throughput per watt, the Gaudi 2 is the more efficient part.

How much do the Gaudi 2 and the A100 cost to rent?

Live cloud listings tracked by Flopper price the Gaudi 2 from $1.10 per GPU-hour across 1 provider, and the A100 from $0.55 per GPU-hour across 25 providers. The rental section on this page links to every provider, rate and the price history for each GPU.

Where to Rent

Intel Gaudi 2 96GB

Rent from

$1.10 / GPU-hr

Lowest of 1 live listing across 1 provider

See all Gaudi 2 rental prices →

Price history and every provider on the pricing page. Some links are affiliate links.

NVIDIA A100 SXM4 80GB

Rent from

$0.55 / GPU-hr

Lowest of 56 live listings across 25 providers · median $1.79

See all A100 rental prices →

Price history and every provider on the pricing page. Some links are affiliate links.

Get Comparison Updates

New GPUs added weekly. Be the first to see how they compare.

Open in Full Compare Tool → Gaudi 2 full specs A100 full specs Estimate cluster cost in the calculator →

Intel Gaudi 2 96GB vs NVIDIA A100 SXM4 80GB

Intel Gaudi 2 96GB

What is the Intel Gaudi 2?

Can I rent the Gaudi 2 in the cloud?

How does the Gaudi 2 compare with other accelerators?

Where do the Gaudi 2 figures come from?

NVIDIA A100 SXM4 80GB

What is the NVIDIA A100?

Can I rent the A100 in the cloud?

How does the A100 compare with other accelerators?

Where do the A100 figures come from?

Specifications

Performance (TFLOPS)

FLOPS by Precision

What actually differs

Frequently asked questions

Is the Gaudi 2 faster than the A100?

Which should you choose, the Gaudi 2 or the A100?

Which has more memory, the Gaudi 2 or the A100?

How much power do the Gaudi 2 and the A100 draw?

How much do the Gaudi 2 and the A100 cost to rent?

Where to Rent

Intel Gaudi 2 96GB

NVIDIA A100 SXM4 80GB

Related comparisons

Get Comparison Updates

Stay Updated on GPU Specs