Training Data · Oct 6, 2026 · 1h 4m · 45m saved
With Amin Vahdat, Chief Technologist for AI Infrastructure at Google
When 100,000 accelerators work on one job, something breaks multiple times a day and perhaps multiple times an hour, Amin Vahdat said. That is why Google judges its AI hardware by a different number than most of the industry does.
Chip specifications lead with FLOPS, the theoretical peak number of calculations a chip can perform each second. Vahdat said Google tracks goodput instead: the useful work a real workload delivers once failures, restarts and software bugs are counted.
"How we hold ourselves accountable is the goodput that we deliver for the workloads that actually matter in the data center." — Amin Vahdat
Amin Vahdat, Chief Technologist for AI Infrastructure at Google, on Sequoia Capital's Training Data podcast, was named head of Google's AI infrastructure at the end of last year, the host said. She put Google's expected capital spending this year at more than $200 billion, most of it going into data centers.
The full interview is covered here so you can skip it. 1 hour 4 minutes of audio, 19 minutes of reading.
Here are the 17 insights that matter.
Key Takeaways
Google's 7- and 8-year-old TPUs are still at 100% utilization, and are replaced once they are fully depreciated after about 6 years
At 100,000 accelerators, something fails multiple times an hour, so Google measures the work actually delivered, not the chip's peak speed
Google has to double its token-generation capability every 6 months, and most of the gain comes from models and software rather than new chips
Google split its 2026 TPU into an inference chip and a training chip because it expects serving to be 30–60% of the market over the chips' lifetime
Either chip can still do the other's job, so a wrong forecast does not strand capacity
DeepMind can change a chip's design shortly before it goes to the manufacturer, something Vahdat said would be close to impossible between two companies
AI agents are pushing demand for ordinary CPUs and storage "through the roof" alongside demand for accelerators
Power is the most fundamental constraint, and Google still prefers to buy it from the local utility rather than generate its own
Supplying a gigawatt at 99.99% reliability on its own would mean building 2 gigawatts
Solar panels in a sun-synchronous orbit get sunlight 98–100% of the time, against 28–35% on land, which is why Google is pursuing orbital data centers
1. Built for the Hardware
The host asked what makes an AI data center different from any other. Vahdat said the concrete, electrical and mechanical yards, cooling, networking and storage are much the same. The difference is specialization.
A conventional data center is planned to outlast several generations of hardware
"But it's a 25-, 30-year planning horizon, and the lifetime of the hardware might be six years." — Amin Vahdat
An AI building is designed together with the racks that go into it
A storage rack might draw 10 to 40 kilowatts. A CPU or GPU rack easily reaches hundreds of kilowatts today, he said, and people are talking about racks that draw megawatts within two years. That changes how power, cooling and networking are laid out across the floor.
"Designing a building that might take 30 storage racks in a row versus one or two AI racks in that same row is a very, very different design." — Amin Vahdat
A building designed to host any kind of hardware for its whole life ends up oversized.
"So if you're trying to make it fully fungible, you'll probably make it too big and too overbuilt—fungible over, certainly, a 30-year period." — Amin Vahdat
A photo of the cabling became one of Google's most popular posts
The host said Google delivered a large Nvidia Vera Rubin cluster to Ineffable Intelligence, one of her portfolio companies. Vahdat said people responded less to the customer than to the pictures of the fiber.
"One of the most popular posts that we've ever had, actually, was around that." — Amin Vahdat
