The CEO of generative AI company Anthropic said that $100 billion data centers are on the horizon.

Dario Amodei, who co-founded the company in 2021 after quitting OpenAI, discussed the growth of data center cluster sizes during a five-hour podcast with Lex Fridman.

“So right now, I think most of the frontier model companies, I would guess, are operating in roughly $1 billion scale, plus or minus a factor of three,” Amodei said.

“Those are the models that exist now or are being trained now. I think next year we’re going to go to a few billion [dollars], and then 2026, we may go to above $10 billion. And probably by 2027, their ambitions to build $100 billion dollar clusters. And I think all of that actually will happen.

“There’s a lot of determination to build the compute, to do it within this country, and I would guess that it actually does happen.”

He added that, once we get to that size, if “that’s still not enough compute, that’s still not enough scale, then either we need even more scale, or we need to develop some way of doing it more efficiently of shifting the curve.”

OpenAI has been the most public about its desires for a $100bn data center, with DCD first to publish a document the company shared with the White House pitching the economic benefit of a 5GW data center in a number of different states.

That theoretical facility would span some 30 million square feet (2,787,100 sqm) and feature some two million GPUs. The single data center would be nearly as large as the combined footprint and capacity of AWS or Microsoft from just a year ago.

An OpenAI source told DCD in September that the 5GW figure was “illustrative of the level of scale we and many believe is needed,” but noted that it was not definitive.

OpenAI is now taking a more hands-on approach to finding data center capacity and will use a 1GW data center in Texas that is being built by Crusoe, used by Oracle, and then leased by Microsoft.

It is not clear if a 5GW data center is economically plausible, nor whether such a system could get power. It was originally reported that Microsoft was considering building the facility for OpenAI, something it has disputed. However, Microsoft is one of the few companies able to afford such a project.

Amodei and fellow Anthropic founders left OpenAI partly due to Microsoft’s increased involvement in the then-non-profit. Anthropic has since raised $2.3bn from Google and $4bn from Amazon Web Services, and primarily uses the latter company’s cloud.

AWS is considering investing another $1bn into the business, in a deal that would see Anthropic use Trainium chips.

The cloud giant is currently building a $10bn data center campus in Madison County, Mississippi. In July, it said that it would invest $100bn into data centers over the next decade.

However, OpenAI and Google have both found that there may be limits to scaling laws – that simply increasing compute along with the data and network leads to a better model. This may impact plans for ever larger data center clusters.

https://www.datacenterdynamics.com/en/news/anthropic-ceo-ai-training-data-centers-to-be-10bn-in-2026-100bn-from-2027/