Google TPU v4 Pod
A full TPU v4 pod is 4,096 chips in a 3D mesh. Google publishes a single peak compute figure of 275 teraflops per chip covering both bf16 and int8 rather than separate rows, giving 1,126.4 PFLOPS across the pod; Google prints this rounded as "1.1 exaflops (bf16 or int8)". Also published per pod: 1.1 PB/s all-reduce bandwidth and 24 TB/s bisection bandwidth. Memory is 4,096 x 32 GiB HBM2 at 1,200 GBps per chip. Google measures per-chip power at 90 W minimum, 170 W mean and 192 W maximum, but publishes no pod-level power, so total system power is left blank here rather than extrapolated from silicon alone. Google publishes no sparse figures for any TPU, so all figures are dense.
We will point you at suppliers who have it. Free, and no signup.
System Details
GPU Configuration
System Specifications
Precision Performance Breakdown
| Precision | System Performance | Per GPU | Efficiency |
|---|---|---|---|
| 1.13 EFLOPS | 275 TFLOPS | — | |
| 1.13 EOPS | 275 TOPS | — |
Where a vendor states a basis, the figure shown is the dense one, and any figure it publishes is listed separately. Vendors commonly headline the sparse number instead: for NVIDIA's 2:4 structured sparsity that is exactly twice the dense figure, though other vendors' sparse modes do not all follow that ratio.
Powered by Google TPU v4
This system utilizes 4096 × Google TPU v4 32GB GPUs, each delivering exceptional performance for AI and HPC workloads.
Per GPU TDP
—
Per GPU Memory
32 GB
Process Node
—
Architecture
TPU
Documentation & Resources
Flopper Spec Sheet
Google TPU v4 Pod specifications, generated from our database. Printable.
Vendor product page
Google TPU v4 Pod documentation
Cloud TPU v5p
Google • 2023-12-07
Typical Use Cases
The Google TPU v4 Pod runs 4096× TPU v4 GPUs over 3D mesh ICI, 1.1 PB/s all-reduce bandwidth per pod, 24 TB/s bisection bandwidth, delivering 1.13 EFLOPS BF16 dense.