Existing Confirmed

Paper on CodeFuse

Ant Group • October 10, 2023
Total GPUs
512
H100 Equiv
161
Power
0.4 MW
FP16
160 PF

System Equivalents

This datacenter's 512 GPUs equal:

NVIDIA DGX A100
8 GPUs/system
64
NVIDIA HGX A100 (8-GPU)
8 GPUs/system
64

Specifications

Primary GPUNVIDIA A100
GPU Count512
SectorPrivate

Additional Information

CodeFuse-13B was trained using 512 Nvidia A100 GPU cards, with a Hardware FLOPs Utilization (HFU) of approximately 60%. The training process took approximately 40 days to complete.