- Home
- Datacenters
- Paper on ViT
Existing Confirmed
Paper on ViT
Google • June 8, 2021
Total GPUs
2.0K
H100 Equiv
127
Power
1.0 MW
FP16
252 PF
Specifications
| Primary GPU | Google TPU v3 |
| GPU Count | 2.0K |
| Sector | Private |
Additional Information
"For models ViT-g and ViT-G, to speed up the training, we scale up the batch size at most to 32 768 and distribute the training to 2048 TPUv3 chips."