Gemini 1.0 Ultra training cluster A

Additional Information

Gemini Ultra, the most compute-intensive model known to date, was trained across multiple data centers totaling 55,000 TPU v4s

Believed to be trained on two sets of 7 pods of 4096 TPUs