Flopper.io Flopper.io
Existing Confirmed

Paper on Falcon Mamba 7B

Technology Innovation Institute • July 24, 2024
Total GPUs
256
H100 Equiv
256
Power
0.4 MW
FP16
253 PF

System Equivalents

This datacenter's 256 GPUs equal:

NVIDIA DGX H100
8 GPUs/system
32
NVIDIA HGX H100 (8-GPU)
8 GPUs/system
32
NVIDIA HGX H100 (4-GPU)
4 GPUs/system
64

Specifications

Primary GPUNVIDIA H100 SXM5 80GB
GPU Count256
SectorPublic

Additional Information

"Falcon-Mamba-7B was trained on 256 H100 80GB GPUs"

Possibly used cloud

Related Datacenters

xAI Colossus Memphis Phase 3
275.8K H100 equiv
Existing
xAI Colossus Memphis Phase 2
200.0K H100 equiv
Existing
OpenAI/Microsoft Goodyear Arizona
100.0K H100 equiv
Existing
Meta 100k
100.0K H100 equiv
Existing
xAI Colossus Memphis Phase 1
100.0K H100 equiv
Existing

Data Source

Data from Epoch AI (CC-BY 4.0)

Sources:

Explore More

View All Datacenters