From systolic arrays to specialized AI systems.
Google’s TPU roadmap increasingly separates training, embedding, post-training and serving needs at the system level. This guide explains the silicon, memory hierarchy, topology and public cloud economics behind that shift.
Evolution of specialization
Current TPU family
The important 2026 distinction is specialization.
v5e and Trillium are cost/versatility-oriented. v5p and Ironwood emphasize large-scale performance. Google’s eighth generation explicitly splits large-scale pre-training (8t) from reasoning/post-training/serving (8i).
| TPU | Status | Primary role | HBM | HBM bandwidth | Scale/topology |
|---|---|---|---|---|---|
| TPU v5e | Available | Cost-efficient training & inference | 16 GB | 819 GB/s | Up to 256 chips · 2D torus |
| TPU v5p | Available | Large-scale training | 95 GiB | 2.765 TB/s | Up to 8,960 chips · 3D torus |
| Trillium / v6e | Available | Training, fine-tuning & serving | 32 GiB | 1.638 TB/s | 256 chips · 2D torus |
| Ironwood / TPU7x | Available | High-performance training & inference | 192 GiB | 7.38 TB/s | Up to 9,216 chips · 3D torus |
| TPU 8t | Coming soon | Massive pre-training & embeddings | 216 GB | 6.528 TB/s | 9,600-chip superpod · 3D torus + Virgo |
| TPU 8i | Coming soon | Post-training, reasoning & serving | 288 GB | 8.601 TB/s | Boardfly · up to 1,152 connected chips |
Silicon architecture
TPUs are systems for structured tensor dataflow.
The useful mental model is not “a GPU with fewer features.” TPU performance comes from co-design across matrix engines, vector units, on-chip scratchpad, HBM, inter-chip fabric and compiler scheduling.
Memory hierarchy
Specialized blocks
SparseCore offloads irregular embedding-related work in several TPU generations. TPU 8t continues that path for embedding-heavy training. TPU 8i replaces SparseCore with a Collectives Acceleration Engine designed to reduce synchronization latency for serving and reasoning.
This is why “TPU” is no longer one architecture profile: the eighth generation deliberately changes specialized blocks and topology by workload.
Conceptual systolic-array demo
This animation illustrates regular data reuse through neighboring processing elements. It is not cycle-accurate hardware behavior.
Cycle: 0
Why compilers matter
The XLA stack can reason globally about tensor placement, sharding and communication. JAX remains central, while PyTorch/XLA is mature and Google has continued expanding native PyTorch support. Portability is much better than early TPU generations, but performance still depends on mapping the model and serving stack to TPU topology.
Pod networking
Network topology follows communication pattern.
v5e / Trillium
Compact slices prioritize cost-efficient training, fine-tuning and serving. The topology fits the scale profile of these versatile systems.
v5p / Ironwood / 8t
Neighbor-oriented communication scales to very large pods and is well suited to dense training collectives and tensor/model parallel workloads.
TPU 8i
Serving and MoE workloads need lower all-to-all latency. Google says Boardfly reduces network diameter from 16 hops to seven in a representative 1,024-chip configuration.
LLM workloads
Training and reasoning now diverge enough to justify different TPU systems.
Throughput-oriented training
Google positions 8t for massive-scale pre-training and embedding-heavy workloads. It uses a 3D torus, SparseCore, 216 GB HBM, 128 MB SRAM, native FP4 and a 9,600-chip superpod; Virgo provides large scale-out bandwidth beyond a superpod.
Reasoning and high-concurrency serving
8i raises on-chip SRAM to 384 MB, uses 288 GB HBM, adds the Collectives Acceleration Engine and switches to Boardfly topology to reduce synchronization/all-to-all latency for MoE and reasoning workloads.
Cloud economics
Current public TPU prices are transparent enough for capacity planning.
Google publishes prices per chip-hour. Rates vary by region and consumption model; the table below uses representative U.S. regions from the current Google Cloud pricing page.
| TPU / region | On demand | DWS Flex-start | DWS Calendar | 1-year | 3-year |
|---|---|---|---|---|---|
| TPU v5e · us-central1 | $1.20 | $0.60 | $0.84 | $0.84 | $0.54 |
| Trillium (v6e) · us-east1 | $2.70 | $1.35 | $1.89 | $1.89 | $1.22 |
| TPU v5p · us-east1 | $4.20 | $2.10 | $2.94 | $2.94 | $1.89 |
| Ironwood · us-central1 | $12.00 | $6.00 | $8.40 | $8.40 | $5.40 |
Price is not cost per token. Equal chip counts do not imply equal throughput. Use these rates only after measuring the number of chip-hours needed to meet the same quality and SLO.
Interactive worksheet
Estimate TPU capacity spend—not workload performance.
Choose a TPU, consumption mode, chip count and hours. The result is a simple list-price capacity estimate using the representative U.S. rates above.
This worksheet intentionally does not compare TPU to GPU. A fair cross-platform comparison requires measured workload goodput, not equal accelerator counts.
Primary sources
First-party references used for the update.
FAQ
TPU architecture questions
Is Ironwood available now?
Yes. Google Cloud publishes current TPU7x/Ironwood documentation and public per-chip pricing. TPU 8t and 8i are still described by Google as coming soon for Cloud customers.
Is Trillium only for inference?
No. Google documents Trillium/v6e for transformer, text-to-image and CNN training, fine-tuning and serving.
Why did Google split TPU 8 into 8t and 8i?
Google says the operational requirements of large-scale pre-training and high-concurrency reasoning/serving have diverged. 8t emphasizes training throughput and embeddings; 8i emphasizes on-chip SRAM, collectives latency and a serving-oriented network topology.
Can PyTorch run on modern TPUs?
Yes. PyTorch/XLA is a mature TPU path, and Google has continued expanding PyTorch support. Performance still needs workload-specific validation.