- Home
- Datacenters
- Paper on GLaM
Existing Confirmed
Paper on GLaM
Google • United States of America • December 13, 2021
Total GPUs
1.0K
H100 Equiv
142
Power
0.4 MW
FP16
282 PF
Specifications
| Primary GPU | Google TPU v4 |
| GPU Count | 1.0K |
| Sector | Private |
Additional Information
"Moreover, to reach similar (and slightly exceeded) scores as GPT-3, we train using 1,024 TPU-v4 chips for 574 hours (with 280B tokens)."