Existing Confirmed

Nemotron-3-8B training cluster

NVIDIA • April 4, 2024
Total GPUs
1.0K
H100 Equiv
322
Power
0.8 MW
FP16
319 PF

System Equivalents

This datacenter's 1.0K GPUs equal:

NVIDIA DGX A100
8 GPUs/system
128
NVIDIA HGX A100 (8-GPU)
8 GPUs/system
128

Specifications

Primary GPUNVIDIA A100
GPU Count1.0K
SectorPrivate

Additional Information

"1,024 A100s were used for 19 days to train the model"