Flopper.io Flopper.io
Existing Confirmed

Paper on BLOOM

• November 9, 2022
Total GPUs
416
H100 Equiv
131
Power
0.4 MW
FP16
130 PF

System Equivalents

This datacenter's 416 GPUs equal:

NVIDIA DGX A100
8 GPUs/system
52
NVIDIA HGX A100 (8-GPU)
8 GPUs/system
52

Specifications

Primary GPUNVIDIA A100
GPU Count416

Additional Information

Training was conducted on 48 nodes, each having 8 NVIDIA A100 80GB GPUs (a total of 384 GPUs); due to possible hardware failures during training, we also maintained a reserve of 4 spare nodes.

Related Datacenters

Oracle OCI Supercluster A100s
10.3K H100 equiv
Existing
Microsoft GPT-4 cluster
7.9K H100 equiv
Existing
Meta Research SuperCluster (RSC-1) Phase 2
5.0K H100 equiv
Existing
EuroHPC Leonardo
4.4K H100 equiv
Existing
Amazon Titan training cluster
4.3K H100 equiv
Existing

Data Source

Data from Epoch AI (CC-BY 4.0)

Sources:

Explore More

View All Datacenters