- Home
- Datacenters
- Paper on Switch transformer
Existing Likely
Paper on Switch transformer
Google • January 11, 2021
Total GPUs
1.0K
H100 Equiv
63
Power
0.5 MW
FP16
126 PF
Specifications
| Primary GPU | Google TPU v3 |
| GPU Count | 1.0K |
| Sector | Private |
Additional Information
The TPU count is from the expert parallelism (1 expert per core) with 2048 experts and 2 tensor cores per TPU v3.