Flopper.io Flopper.io
Existing Confirmed

Paper on XGLM

Meta AI • December 20, 2021
Total GPUs
256
H100 Equiv
80
Power
0.2 MW
FP16
80 PF

System Equivalents

This datacenter's 256 GPUs equal:

NVIDIA DGX A100
8 GPUs/system
32
NVIDIA HGX A100 (8-GPU)
8 GPUs/system
32

Specifications

Primary GPUNVIDIA A100
GPU Count256
SectorPrivate

Additional Information

"The XGLM 7.5B model was trained on 256 A100 GPUs for about 3 weeks, at a speed of 311.6k words per second"

Related Datacenters

Meta $200B Campus Rumor
20.3M H100 equiv
Planned
Meta Louisiana Datacenter
5.1M H100 equiv
Planned
Meta Prometheus New Albany
1.3M H100 equiv
Planned
Meta 100k
100.0K H100 equiv
Existing
Meta GenAI 2024a
24.6K H100 equiv
Existing

Data Source

Data from Epoch AI (CC-BY 4.0)

Sources:

Explore More

View All Datacenters