Google Exec Claims Company Needs to Double Its AI Serving Capacity ‘Every Six Months’: Report

Tech companies are racing to build out their infrastructure as their increasingly resource-intensive AI products gobble up capacity, clean out chipmakers’ supply, and require more power. Google, once dubbed the “King of the Web,†is one of those companies, and a high-level exec for The Big G is reported to have told staff that the company needs to scale up its serving capabilities exponentially if it wishes to keep up with the demand for its AI services.

CNBC got its hands on a recent presentation given by Amin Vahdat, VP of Machine Learning, Systems, and Cloud AI at Google. The presentation includes a slide on “AI compute demand†that asserts that Google “must double every 6 months…. the next 1000x in 4-5 years.â€

“The competition in AI infrastructure is the most critical and also the most expensive part of the AI race,†Vahdat reportedly said at the all-hands meeting where the presentation took place. Google’s “job is of course to build this infrastructure, but it’s not to outspend the competition, necessarily,†he added. “We’re going to spend a lot,†he said, in an effort to create AI infrastructure that is “more reliable, more performant and more scalable than what’s available anywhere else.â€

Since CNBC’s story was published, Google has quibbled with the reporting. While CNBC originally quoted Vahdat as saying that the company would need to “double†its compute capacity every six months, a Google spokesperson told Gizmodo that the executive’s words were taken out of context. The spokesperson further explained that Vahdat “was not talking about a capital buildout of anything approaching the magnitude suggested. In reality, he simply noted that demand for AI services means we are being asked to provide significantly more computing capacity, which we are driving through efficiency across hardware, software, and model optimizations, in addition to new investments.â€Â 

CNBC has since updated its reporting from “compute†to “serving†capacity. Serve capacity would refer to Google’s ability to handle a rising tide of user requests, while compute capacity woud refer to the company’s overall infrastructure dedicated to AI, including what is needed to train new models and other expenditures. When asked for further clarification about the difference between the two, the spokesperson said that the original headline “read as if he was implying that we are doubling the amount of compute we have — either measured by the # of chips we operate or the amount of MW of electricity.†Instead, “the capacity increases Amin described will be reached in a number of ways, including new more capable chips and model efficiency and optimization,†they added.

Whatever’s happening under the hood, it would appear that Google—like its competitors—needs to scale up its operations to support its nascent AI infrastructure business. Vahdat’s comments come not long after the tech giant reported some chunky profits from its Cloud business, with the company announcing it plans to ramp up spending in the coming year.

During his presentation, Vahdat also reportedly claimed that Google needs to “be able to deliver 1,000 times more capability, compute, storage networking [than its competitors] for essentially the same cost and increasingly, the same power, the same energy level.†He admitted that it “won’t be easy†but said that “through collaboration and co-design, we’re going to get there.â€

The race to build data centers—or “AI infrastructure†as the tech industry calls it—is getting crazy. Like Google, Microsoft, Amazon, and Meta all claim they are going to ramp up their capital expenditures in an effort to build out the future of computing (cumulatively, Big Tech is expected to spend at least $400 billion in the next twelve months). As these facilities go up, they are causing all sorts of drama in the communities where they reside. Environmental and economic concerns abound. Some communities have begun to protest data center projects—and, in some cases, they’re successfully repelling them. Still, given the sheer amount of money invested in this industry, it will be an ongoing fight for Americans who don’t want the AI colossus in their backyards.