- Home
- Datacenters
- Paper on CogView
Existing Confirmed
Paper on CogView
Alibaba • May 26, 2021
Total GPUs
512
H100 Equiv
32
Power
0.3 MW
FP16
64 PF
Specifications
| Primary GPU | NVIDIA Tesla V100 SXM2 32 GB |
| GPU Count | 512 |
| Sector | Private |
Additional Information
We train the model with batch size of 6,144 sequences (6.7 million tokens per batch) for 144,000 steps on 512 V100 GPUs (32GB).