Flopper.io Flopper.io
Existing Confirmed

Paper on StarCoder

• May 9, 2023
Total GPUs
512
H100 Equiv
161
Power
0.4 MW
FP16
160 PF

System Equivalents

This datacenter's 512 GPUs equal:

NVIDIA DGX A100
8 GPUs/system
64
NVIDIA HGX A100 (8-GPU)
8 GPUs/system
64

Specifications

Primary GPUNVIDIA A100
GPU Count512

Additional Information

"We trained our model on a GPU cluster with 512 A100 80 GB GPUs distributed across 64 nodes."

Related Datacenters

Oracle OCI Supercluster A100s
10.3K H100 equiv
Existing
Microsoft GPT-4 cluster
7.9K H100 equiv
Existing
Meta Research SuperCluster (RSC-1) Phase 2
5.0K H100 equiv
Existing
EuroHPC Leonardo
4.4K H100 equiv
Existing
Amazon Titan training cluster
4.3K H100 equiv
Existing

Data Source

Data from Epoch AI (CC-BY 4.0)

Sources:

Explore More

View All Datacenters