Definition
A conventional data center is planned in racks and square feet. An AI data center is planned in megawatts — because power, not floor space and not even chips, is what it runs out of first.
The Uptime Institute's 2024 global survey put the average typical rack density among respondents at around 8 kW. A single NVIDIA GB200 NVL72 — 72 Blackwell GPUs and 36 Grace CPUs bolted into one NVLink domain and sold as a rack — draws roughly 120 kW. That is about fifteen conventional racks' worth of electricity in one cabinet, and it is the fact from which everything else on this page follows.
Notice what that does not buy you. Concentrating the same electrical load into fifteen times fewer cabinets saves floor space, and floor space was never the expensive part. You still buy the same megawatts. What has actually happened is that the facility stopped needing a warehouse and started needing a substation — and a warehouse is cheap and quick, while a substation is neither.
How It Works
Power became the unit because floor space stopped being scarce
Take a fixed IT load of 166 MW and lay it out two ways. At 8 kW per rack it needs about 20,800 racks — a genuinely enormous building. At 120 kW per rack it needs 1,389. Same electricity, same electricity bill, one-fifteenth of the hall.
So the resource that used to bind — square feet, and the rows of cabinets you could fit in them — stopped binding. The resource that now binds is the one that did not shrink: the amps arriving at the property line, the transformers that step them down, and the heat that has to leave afterwards. A planner who still thinks in racks is optimising the abundant input.
Air has a hard ceiling, and it is a physics ceiling
Heat removal is not an engineering preference; it is a mass-flow problem. The heat a fluid can carry
is Q = ṁ × c_p × ΔT — mass flow times specific heat times the temperature rise you allow it. Air
is a terrible working fluid on both counts. At room conditions its density is about 1.2 kg/m³ and
its specific heat about 1,005 J/(kg·K), so a cubic metre of air absorbs about 1,206 joules per
degree. A cubic metre of water absorbs 4,182,000. Per unit of volume you have to physically
move, water carries about 3,500 times more heat.
Price the 120 kW rack both ways. Allow air a generous 15 °C rise across the servers:
air: V = 120,000 W / (1.2 kg/m³ × 1005 J/(kg·K) × 15 K)
= 120,000 / 18,090
= 6.63 m³/s (about 14,000 CFM)
across a rack face of roughly 0.6 m × 2.0 m = 1.2 m²:
v = 6.63 / 1.2 = 5.5 m/s — a 20 km/h wind, through the cabinet, forever
Now water, allowing a 10 °C rise:
water: V = 120,000 W / (1000 kg/m³ × 4182 J/(kg·K) × 10 K)
= 120,000 / 41,820,000
= 0.0029 m³/s = 2.9 litres per second
A gale versus a garden hose. That is why liquid cooling stopped being exotic somewhere around 30 kW per rack — the point at which the fans required to shift the air draw serious power, make serious noise, and still lose. NVIDIA does not offer the GB200 NVL72 in an air-cooled variant. It ships with cold plates on the GPUs, the CPUs and the NVLink switches, and the heat leaves in water. The physics decided; the vendor merely complied.
The electricity has to get into the rack, too
The distribution beneath the floor breaks in the same way. Racks have historically been fed at around 54 VDC internally, and at that voltage a megawatt-class rack needs a current so large that NVIDIA estimates it would require up to 200 kg of copper busbar in a single rack — and up to 200,000 kg of rack busbar across a one-gigawatt site. Its answer, announced in 2025, is to move rack-scale distribution to 800 VDC, which pushes about 85% more power through the same conductor. This is the same story as the cooling: the old assumptions were not wrong, they were sized for a load an order of magnitude smaller.
PUE measures the building, not the computer
Power Usage Effectiveness is the industry's efficiency number, and it is one division:
PUE = total facility power / IT equipment power
A PUE of 1.5 means that for every watt reaching a server, half a watt is spent on cooling, power conversion, lighting and losses. The Uptime Institute's 2025 survey put the global average at 1.54, a figure that has barely moved in six years. Google reports a fleet-wide trailing-twelve- month average of 1.09 — genuinely excellent, and achieved partly by counting more overhead than most operators do, which is itself a warning about comparing the numerator across companies.
PUE is useful and it is worth tracking. But look at what is in the denominator. PUE says nothing whatsoever about whether the IT load did any useful work. A cluster grinding away at a training run that will be thrown away, a job stuck in a restart loop, a badly parallelised workload leaving two-thirds of its GPUs idle-but-powered — all of them post the same PUE as a facility doing something valuable. Worse, the incentive runs backwards: because a good deal of facility overhead is fixed, increasing the IT load dilutes it and improves the ratio. A data center can improve its PUE by wasting more compute. It is a metric about a building that gets quoted as if it were a metric about computing, and the work actually done per joule belongs to a different measurement entirely.
Worked example: sizing a 100,000-GPU facility
Start with the accelerator and end at the meter. Nothing here is estimated; every input is above.
1. The rack. A GB200 NVL72 holds 72 GPUs and draws about 120 kW.
2. The racks. A 100,000-GPU cluster therefore needs:
racks = 100,000 / 72 = 1,389 racks
3. The IT load.
IT = 1,389 racks × 120 kW = 166,680 kW = 166.7 MW
(Storage, the scale-out network spine and head nodes sit on top of that; the 120 kW already covers the NVLink switches inside the rack.)
4. The load at the meter. Apply a PUE. This single choice is worth tens of megawatts:
at Google's 1.09: 166.7 × 1.09 = 181.7 MW
at the global average 1.54: 166.7 × 1.54 = 256.7 MW
The 75 MW gap between a well-run facility and an average one is, by itself, larger than most data centers that existed a decade ago.
5. What that means over a year. Training runs flat out, so take the 181.7 MW figure and run it:
181.7 MW × 8,760 h = 1,591,692 MWh ≈ 1.59 TWh per year
The average US residential utility customer purchased 10,791 kWh in 2022 (EIA). So:
1.59e9 kWh / 10,791 kWh = ~147,000 homes
One cluster draws the household electricity of about 147,000 American homes — a mid-sized city's worth of residential demand, from one building. And its 1.59 TWh is roughly 0.9% of the 176 TWh that every data center in the United States consumed in 2023, according to Lawrence Berkeley National Laboratory. A single site, approaching one percent of the entire national fleet's draw. That is the unit change, in one number.
How much energy that translates into per query or per training run, and what it means environmentally, is the subject of AI energy consumption; this page is about the facility whose draw is the aggregate of all of it.
Real-World Applications
xAI's Colossus, and the 8 MW that were actually available. When Colossus began operating in Memphis in September 2024, its grid connection was 8 MW — against a site needing roughly 150 MW. xAI bridged the gap with mobile gas turbines; by April 2025 aerial imagery counted 35 of them totalling around 422 MW, and the resulting air-permit fight with local residents and the Southern Environmental Law Center became a national story. The grid connection was upgraded to 150 MW in November 2024. Colossus is the cleanest available demonstration that the chips arrive first and the electricity arrives later, and that what an operator does in between is a real, expensive, sometimes ugly decision.
Stargate, announced in the unit. In September 2025 OpenAI said that its Abilene, Texas site plus five new locations brought the Stargate program to "nearly 7 gigawatts of planned capacity." Not seven hundred thousand servers, not ten million square feet — gigawatts. When the press release itself reaches for the electrical unit, the argument of this page is no longer contestable.
Buying a reactor's output. Constellation announced on 20 September 2024 a 20-year power purchase agreement with Microsoft that underwrites restarting Three Mile Island Unit 1 as the Crane Clean Energy Center — approximately 835 MW, targeted for 2028, at a stated capital cost of about $1.6 billion. Read it precisely, because the nuclear story is routinely oversold: this is a long-term offtake contract against an existing reactor being brought back, not a new reactor built for a data center. It is a procurement decision, and what it is really buying is not electricity but certainty about electricity in 2028 — which, given the queue below, is the scarcer good.
Key Concepts
- Critical IT load, not floor area, is what you are buying. Colocation and cloud capacity for AI is priced and contracted in kW or MW of critical load. Ask how many megawatts a facility can deliver and cool, not how many racks it can hold — a hall rated for 8 kW cabinets cannot host 120 kW cabinets no matter how much empty floor it has.
- Utilisation and PUE are orthogonal. A facility can be superb at PUE and terrible at model FLOPs utilisation, and the two numbers will never notice each other. Ask for both, and if the second one is unavailable, assume it is the reason.
- Density concentrates every other problem. 120 kW in one cabinet is also a floor-loading problem, a fire-suppression problem, a water-quality problem, and a single-point-of-failure problem: a rack that is one NVLink domain is also one thing that can go down.
Challenges
The substation is the long pole, and almost nobody plans for it
This is the failure mode. A team builds a cluster plan around GPU delivery dates — allocation secured, racks ordered, shell under construction — and discovers eighteen months in that the utility cannot energise 180 MW at that address on that date, and that no amount of money makes it faster.
The scale of the queue is public. LBNL reported that at the end of 2024, roughly 2,290 GW of generation and storage capacity sat active in US interconnection queues. Of the capacity that requested interconnection between 2000 and 2019, only 13% had reached commercial operation by the end of 2024 — most of the rest withdrew. And the typical project that did reach commercial operation in 2024 had spent about 55 months, four and a half years, in the queue. (That queue is for generators; a large load has its own utility study process. But the supply that has to exist before your load can be served comes through this one.)
Set that against the schedule for everything else. Accelerators ship on a lead time measured in months. A building goes up in around two years. The power takes longer than the hardware and the building put together, and it is the only item on the list that cannot be accelerated by spending more. This is why operators sign multi-year PPAs before they have a design, site next to existing generation rather than next to customers, and — in the Colossus case — burn gas on site while they wait.
Heat is not a footnote
A facility that cannot reject the heat cannot run the chips it has bought. This is the second schedule trap, and it is less obvious than the first because heat rejection looks like a sub-contract rather than a constraint. It is not: 166 MW in means 166 MW of heat out, into chillers, into cooling towers, into a river, into the air. That capacity is a permit, a water allocation, and a piece of plant with its own lead time. Retrofitting an air-cooled hall for 120 kW racks means new pipework, CDUs, a floor rated for the load, and a wet loop in a building designed to keep water out — which is why so many operators conclude it is cheaper to build new.
The metrics do not measure the thing you care about
PUE measures the building. Rack count measures the furniture. Neither tells you the one thing that determines whether the capital was well spent: how much useful computation came out per joule that went in. A cluster running at 15% model FLOPs utilisation in a facility with a world-class PUE is a very efficient way to waste two-thirds of a nine-figure investment, and every published number about the site will look excellent.
Future Trends
The direction of travel in the hardware is straightforwardly up: NVIDIA has said it is preparing the industry for 1 MW IT racks starting in 2027, which is the entire reason for the 800 VDC push. Everything in the building — busbars, cold plates, coolant distribution units, floor ratings — is being re-specified around a load an order of magnitude above what the current generation of halls was designed for. Any facility being built today is a bet on where that curve stops.
The interesting question is where the megawatts come from, and the honest answer in 2026 is: mostly from the existing grid, plus gas, plus long-term contracts against nuclear plants that already exist. New nuclear — small modular reactors in particular — is real, funded and years away; treating it as a solution to a 2027 interconnection problem is a category error. The nearer-term levers are less romantic and more effective: siting where the power already is rather than where the customers are, running facilities as flexible load that can curtail when the grid is stressed, and reducing demand at the source through quantization and model compression, which are the only interventions here that make the megawatts smaller rather than merely finding them somewhere else.