- Home
- Datacenters
- Paper on Mamba 2 Hybrid
Existing Confirmed
Paper on Mamba 2 Hybrid
NVIDIA • June 12, 2024
Total GPUs
1.0K
H100 Equiv
1.0K
Power
1.5 MW
FP16
1.0K PF
System Equivalents
This datacenter's 1.0K GPUs equal:
NVIDIA DGX H100
8 GPUs/system
128
NVIDIA HGX H100 (8-GPU)
8 GPUs/system
128
NVIDIA HGX H100 (4-GPU)
4 GPUs/system
256
Specifications
| Primary GPU | NVIDIA H100 SXM5 80GB |
| GPU Count | 1.0K |
| Sector | Private |
Additional Information
"When training on NVIDIA H100 GPUs (NVIDIA 2023), with a tensor-parallel size of four and data-parallel size of 256 (1024 total GPUs)"