GPU data center cooling: air, liquid, and the kW/rack threshold where the choice flips

There is one number that decides your cooling architecture: kilowatts per rack. Below ~30 kW/rack, well-designed air cooling works. Between 30 and 50 kW/rack, air still works but only with containment plus an in-row or rear-door assist. Above 50 kW/rack, liquid is not optional. The B200 NVL72 sits at 120 kW — there is no air-cooled answer to that rack.

This page walks through each cooling regime with the numbers, the failure modes, and the operator-visible signals. For the power side of the same equation see power; for layout choices see rack design.

The thermal floor: what you're actually removing

A GPU node converts essentially 100% of its electrical input into heat. There's no useful mechanical work, no chemistry, no light. Every watt that comes in the PSU has to leave the building as heat. So the cooling plant is sized to the IT load — which means a 1 MW IT hall removes 1 MW of heat continuously.

The cooling architecture answers two questions:

1. How does the heat get from the silicon junction to the rack exhaust?
2. How does the heat get from the rack exhaust to the outside world?

Air, liquid, and immersion are different answers to (1). Chillers, dry coolers, evaporative towers, and free cooling are different answers to (2). You can mix and match, but the pairings have to be consistent — a 120 kW liquid rack still needs heat rejection, and that rejection plant is what determines your PUE and WUE.

ASHRAE classes — the envelope you have to live in

ASHRAE TC 9.9 publishes the thermal envelopes IT manufacturers warrant their gear against. Your facility temperature has to stay inside the envelope of whatever class your hardware claims.

Two things to note for AI clusters:

• GPU servers are typically rated A2. Some hyperscale-tuned designs claim A3 or A4 to enable free cooling in hot climates, but stock DGX/HGX gear is A2.
• The "recommended" envelope is tighter than the "allowable." Running cold-aisle inlets at 18-22 °C is conservative; pushing to 27 °C is allowed and saves cooling energy but reduces your headroom for excursion events.

For altitudes above 900 m the maximum dry-bulb derates: 1 °C per 175 m for A3 and 1 °C per 125 m for A4. Most colos handle this transparently but verify if your site is at altitude (Mexico City, Denver, parts of APAC).

The ΔT for an air-cooled GPU node is large by enterprise standards — typically 15-20 °C between cold-aisle inlet and hot-aisle exhaust at full load. A 22 °C inlet becomes a 38-42 °C exhaust. That's air you can't recirculate without containment.

Air cooling: how high can you push it

Room-air cooling has a hard ceiling set by airflow physics. The amount of heat air can carry is:

Q_kW = mass_flow_kg_per_s * Cp * delta_T
     = density * volume_flow * Cp * delta_T

For air:  density ~ 1.2 kg/m^3, Cp ~ 1.0 kJ/kg-K
At delta_T = 15 K:
Q_kW = 1.2 * 1.0 * 15 * volume_flow_m3_per_s = 18 * volume_flow_m3_per_s

For a 30 kW rack, you need volume_flow ~ 1.7 m^3/s (~3600 CFM) of air across it.
For a 50 kW rack, you need volume_flow ~ 2.8 m^3/s (~5900 CFM).

Pushing 5900 CFM across a single rack means high static pressure, loud fans, and air velocities at the perforated tile that rip cables off cold-aisle bottoms if you're not careful. In practice, 30 kW/rack is the comfortable air ceiling and 50 kW/rack is heroic. Above that, liquid wins not because it's "better" but because air doesn't physically carry enough heat without unreasonable airflow.

Hot/cold aisle containment (mandatory above ~10 kW)

The default layout: rows arranged back-to-back, cold air delivered to the front (cold aisle), hot exhaust collected from the back (hot aisle). Containment is a roof and end-cap doors that physically separate the two airstreams. Without containment, hot exhaust recirculates into the cold inlet, supply temperature rises, and CRAH set-points have to drop — eating PUE.

Containment is the single biggest PUE lever for air-cooled facilities. Adding it to an existing room can cut cooling energy 20-30%.

CRAC vs CRAH

• CRAC (Computer Room Air Conditioner): self-contained DX (direct expansion) unit, has its own compressor, blows cold air. Older, less efficient, used in small facilities.
• CRAH (Computer Room Air Handler): chilled-water coil, no compressor on the unit itself. The chiller plant is centralized. More efficient at scale, dominant in modern halls.

For GPU rooms with 30+ kW racks you want CRAH plus chilled water at 12-16 °C supply, with in-row units placed every 2-4 racks instead of perimeter-only. In-row gets the cold air much closer to the inlet, reducing fan energy and bypass.

When air stops being enough

Hard signals you've outgrown air:

• Cold-aisle inlet temperature creeps up under load even with the CRAH at full output.
• Per-rack power is climbing past 30 kW, hot-aisle temps hitting 45-50 °C, and ASHRAE A2 max (35 °C inlet) is at risk during excursion events.
• Adding more CRAHs no longer helps because air velocity is the bottleneck, not chilled water capacity.
• GPUs start thermal-throttling under load. nvidia-smi -q -d TEMPERATURE shows GPU temps in the high 80s/90s and dmesg shows nvidia: Throttle reason: SW Thermal Slowdown.

When you see those, it's a liquid conversation.

Direct-to-chip liquid (DLC): the GPU default for 2025+

Direct-to-chip (DLC, also called direct liquid cooling or "cold plate") puts a liquid-filled cold plate physically on top of the GPU die package and the CPU. Coolant flows through the plate, picks up heat, and carries it to a coolant distribution unit (CDU) that exchanges to facility water.

GPU package
   |
   v
[ cold plate ] -- coolant out (warm) --
   |                                    \
[ cold plate ]                            [  CDU  ]  <-- facility water in
   |                                    /
[ cold plate ] -- coolant in (cool)  --

What's plumbed to the chip vs the CRAH:

• GPU dies, CPUs, NVSwitch ASICs: cold plate. ~85-90% of node heat goes to liquid.
• DRAM, NVMe, PSU, NICs, fans: still air-cooled inside the chassis. The remaining ~10-15% goes out the back of the rack as warm air.

That last point matters: a "fully liquid-cooled" rack still has an air component. You still need some CRAH capacity, just much less. Typical mix: 80-90% of the rack heat to liquid, 10-20% to residual air.

Coolant, flow, temperature

• Coolant: typically a propylene-glycol/water mix (PG25 or PG30) for freeze protection; some hyperscalers use treated water + biocide. Two-phase fluorocarbons exist for advanced systems.
• Inlet temperature to the cold plate: usually 25-35 °C ("warm water" cooling). NVIDIA's GB200 NVL72 spec is 25 °C inlet.
• Outlet temperature: 40-50 °C. NVIDIA's GB200 spec exits the rack about 20 K warmer than inlet, so ~45 °C.
• Flow rate: GB200 NVL72 publishes ~2 L/s at the rack manifold; per-tray flow is in the 2-3 L/min range per cold plate, with strict minimums (the cooling community quotes ≥2-3 L/min per module on GB200).
• Pressure drop across the rack: 1-2 bar typical, depending on plate count.

CDU placement and capacity

A coolant distribution unit (CDU) is the heat exchanger between the secondary loop (clean coolant inside the rack) and the primary loop (facility water). Two flavors:

• In-row CDU: 100-300 kW capacity, sits adjacent to 1-3 racks. Common for retrofits and small DLC deployments.
• In-rack CDU: smaller, 30-50 kW, lives inside the GPU rack itself. Lower latency to leak detection but eats U budget.
• Centralized CDU room: 1-5 MW capacity, one big plant feeds many rack manifolds via overhead piping. Hyperscale standard.

Capacity rule of thumb: size the CDU to handle the liquid-cooled portion of the rack at design draw, plus 25% margin. For a 120 kW GB200 rack with 85% to liquid, that's ~102 kW design heat load, so a 130 kW CDU minimum.

Facility-side: rejecting the heat

DLC moves the question one step out: the CDU dumps heat into facility water, and the facility water has to be cooled by something. Three common designs:

The pairing of DLC + dry cooler in cool climates or DLC + hybrid evaporative in moderate climates is the dominant 2025+ AI factory design. It's how you get both PUE 1.15 and WUE near zero — see the numbers in the power doc.

Rear-door heat exchangers (RDHX): the hybrid step

Rear-door heat exchangers are a chilled-water coil mounted on the back of a rack, in the path of the hot exhaust air. As exhaust passes through the coil it transfers heat into water, leaving the rear of the rack at near-room temperature.

[ cold aisle ]           [ rack ]            [ hot side ]
   18 C      -->  inlet  servers exhaust  -->  RDHX coil  -->  ~22 C
                                                  |
                                              chilled water

Where RDHX wins:

• Existing rooms that need to push past their air-cooling ceiling without a forklift upgrade.
• Racks 30-60 kW where DLC is overkill or the hardware doesn't support cold plates.
• Mixed environments where DLC racks coexist with traditional servers and you want one cooling loop to handle the high-density rows.

Capacity: passive RDHX (no fan) handles ~25-35 kW per rack. Active RDHX (with fan assist on the door) handles 60-80 kW per rack and in some published designs up to 150 kW with aggressive water flow. OCP has a published spec for active door HX targeting these higher densities.

Operator notes:

• The room-air load goes to near-zero on RDHX-equipped racks. The room CRAH is essentially handling only the residual leakage and the unrelated low-density gear.
• Water leak detection at the door is mandatory. A leak in front of an exhaust grille puts mist into the rack interior, which is bad for PSUs and worse for switching gear.
• RDHX adds 6-10 inches to the rack depth. Aisle clearance has to be checked.

Immersion cooling: niche, but growing

Immersion drops the entire server into a tank of dielectric coolant. Two flavors:

• Single-phase immersion: server lives in mineral oil or synthetic dielectric. Coolant doesn't change phase. Pumps move coolant through external heat exchangers. PUE 1.03-1.06. Mature for crypto, growing for AI but still a small slice.
• Two-phase immersion: coolant is a fluorocarbon that boils at ~50 °C at chip surface, vapor rises, condenses on a coil at the top of the tank, drips back. No pumps. PUE 1.02-1.05. Higher capex, regulatory headaches around fluorocarbons.

Where immersion fits:

• High density per square meter: a 200 kW immersion tank takes the floor space of one rack.
• Edge sites with no chilled water plant: a tank + dry cooler can run with minimal facility infrastructure.
• Specialized workloads where you control the hardware end-to-end (Bitcoin ASICs, custom AI accelerators).

Where it doesn't:

• Stock DGX/HGX gear isn't certified for immersion. Voiding the warranty on $400k chassis isn't a great trade.
• Servicing is messy: pulling a board out of oil is a real procedure with drip trays and absorbent mats. Field-replaceable parts are slow.
• Cabling is awkward — fiber and DAC cables route through tank seals, optics have to be immersion-rated.

For mainstream AI build-outs in 2025, DLC is the answer, RDHX is the upgrade path, immersion is a special case.

Real numbers: DGX H100 and GB200 NVL72

DGX H100 (air or RDHX)

• 8U chassis, ~6.5 kW sustained, 10.2 kW peak.
• Rack of 4-5 DGX H100 = 30-50 kW. Air with containment + in-row CRAH works; RDHX assists at the high end.
• Cold-aisle inlet: 18-22 °C target (well inside ASHRAE A2).
• Hot-aisle exhaust: 38-42 °C at full load.
• Per-rack airflow at 50 kW: ~6000 CFM, distributed across multiple perforated tiles.

A typical H100 SuperPOD scalable unit uses 32 DGX nodes spread across 8 racks (4 nodes/rack at ~30 kW), with cold aisle containment and in-row CRAH. NVIDIA's published SuperPOD design guide cites "rack power exceeds 40 kW" — that's the design ceiling, not a comfortable steady-state.

GB200 NVL72 (full liquid)

• 1 rack = 72 Blackwell GPUs + 36 Grace CPUs + 9 NVSwitch trays.
• 18 compute trays (1U each, 2 Grace + 4 Blackwell each), 9 NVSwitch trays (1U).
• ~120 kW continuous, four 30 kW power shelves at the bottom feeding a 48 V busbar.
• Coolant: 25 °C inlet, ~45 °C outlet, ~2 L/s rack flow.
• Cooling split: ~85% to cold plates (compute, switch), ~15% residual air for memory, NVMe, PSU.
• Cold plate per Blackwell GPU: minimum 2-3 L/min coolant, max inlet 45 °C.

You don't fit a GB200 NVL72 into a 25 kW/rack room. You design a hall around 120 kW racks: 480 V three-phase distribution, in-rack busbar, primary chilled-water loop sized for tens of kW per rack at warm-water temperatures, and a heat-rejection plant matched to the climate.

Failure modes the spec sheets don't show you

Air-side

• Bypass airflow: cold air finding a path to hot aisle without going through a server. Holes in the floor under unused racks, gaps in containment, missing blanking panels. Fix: blanking panels mandatory in every empty U; under-floor seals at all rack positions.
• Recirculation: hot exhaust finding a path back to cold inlet. Almost always: containment ends not sealed at the row ends, or doors propped open. Operator wisdom: walk the hot aisle every shift, feel the cold-aisle inlet temperatures with the back of your hand at top, middle, bottom of each rack.
• Tile starvation: too many perforated tiles relative to underfloor static pressure, so the rack at the end of the row gets no air. Detected by inlet thermometers.
• CRAH fight: two CRAH units on the same row with different set-points, one cooling and the other dehumidifying, both wasting energy. Fix: single set-point logic, or BMS coordination.

Liquid-side

• CDU loss: the unit fails or trips. The rack heats up in seconds. The right response is GPU power-cap or shutdown, not "wait for the loop to recover." A real CDU failure on a 120 kW rack with no failover is a 90 °C-junction emergency in under a minute.
• Leaks: cold plate gasket fails, quick-disconnect mis-seats, manifold cracks. Leak detection is mandatory — drip trays under every rack, conductive cable along manifolds, leak relays into BMS. Every CDU should also pressure-monitor the secondary loop and alert on fast pressure drops.
• Bubbles in the loop: air ingress reduces flow at random plates, causing localized overheating. Fix: degassing valve at the highest point of the loop, monthly bleed.
• Biofouling: in poorly treated water loops, biofilm accumulates and reduces heat transfer. Fix: scheduled biocide dosing, water analysis quarterly.
• Condensation: if facility water is colder than dew point at the cold plate connection, condensate forms on cold piping. Solves the heat problem but creates a water-on-PCB problem inches away. Fix: keep secondary loop above local dew point — typical rule, ≥ ambient dew point + 2 K.

Pressure and temperature alarms that matter

Coolant chemistry and water quality

The fluid in your secondary loop is one of three things, and the choice has lifetime cost implications nobody flags upfront.

Treated water

Deionized or low-conductivity water with corrosion inhibitors and biocide. Highest specific heat (4.18 kJ/kg·K), best heat transfer per liter. Used by Meta, Google, and most hyperscale DLC deployments where they own the water-treatment plant.

Operator notes:

• Conductivity must stay below ~5 μS/cm for typical cold plates. Above that, galvanic corrosion eats the copper plate over months.
• Biocide is mandatory. A recent OCP DLC discussion thread documents loops that fouled in 90 days from a single skipped dosing.
• pH window: 7.5-9.0 typically. Outside that, corrosion accelerates.
• Quarterly water analysis (conductivity, pH, dissolved metals, biocide concentration) is the minimum cadence. Annual deep analysis catches ion migration.

Propylene glycol (PG25 / PG30) mix

25% or 30% propylene glycol in water with corrosion inhibitors. Lower specific heat (~3.8 kJ/kg·K at PG25, ~3.65 at PG30), so 5-10% less heat carried per liter at the same flow rate.

Why operators pick it:

• Freeze protection. PG25 protects to ~-9 °C, PG30 to ~-13 °C. If your facility piping touches outdoor air or unheated space, this is non-negotiable.
• Corrosion behavior is more forgiving — minor system contamination doesn't immediately attack the plumbing.
• Pre-mixed shipments take the on-site mixing error out of bring-up.

Costs:

• ~10% reduction in heat-carrying capacity vs treated water — the manifold has to flow proportionally faster.
• Higher viscosity means higher pump head loss; pump power is 5-10% higher.
• PG can degrade over time; 5-7 year service life under normal conditions, less if the loop runs hot.

Two-phase fluorocarbon

Used in two-phase immersion and some advanced direct-on-chip systems. The fluid is a 3M / Chemours engineered dielectric with a boiling point near operating chip temperature (~50-60 °C). Heat absorption is by latent heat of vaporization, which is enormous per gram — a few mL boiling carries hundreds of W.

Why it's niche for mainstream AI:

• Fluorinated chemistries are coming under regulatory pressure (PFAS rules vary by jurisdiction).
• Cost per liter is 100-1000× water-glycol.
• Specialized mechanical design — vapor-tight seals, condenser placement — that mainstream OEMs don't ship.

For most AI build-outs the answer is PG25 in colder climates, treated water in indoor-only or warm-climate facilities, fluorocarbon only when you've built a specialized lab.

What goes wrong if water quality drifts

A cold plate is a copper micro-channel array with channels ~200-400 μm wide. Three things foul it:

• Particulate contamination: rust flakes from a poorly passivated steel pipe upstream. Filters (10-50 μm in-line) catch most, but a single missed flush can deposit enough particulate to halve flow through one plate. Detection: rising plate-to-plate temperature variance — one GPU runs 5-8 K hotter than its peers.
• Biofilm: bacteria colonize stagnant points, secrete polysaccharide goo. Reduces heat transfer coefficient by 30-50% before flow drops noticeably. Detection: rising approach temp (coolant out - air temp) at constant load.
• Mineral scale: from hard water tap-fill in an open-loop system. Calcium carbonate precipitates on the warmest surfaces (the chip-facing side of the plate). Detection: gradually rising chip temp at the same coolant inlet temp.

The cheapest insurance is a 5 μm bag filter on the loop fill line, a 10-25 μm cartridge filter in-line, and a documented water-treatment SOP. The most expensive uninsured failure is replacing 72 cold plates because you let the loop foul.

Heat rejection plant: where the kilowatts actually go

The cold plates and CDUs move heat from chip to facility water. The facility plant moves it from facility water to outside air (or sky). The choices here determine PUE and WUE more than anything else.

Mechanical chiller plant

Compressor-driven chillers cool a closed water loop to 7-15 °C supply. The compressor's electrical load is roughly 20-30% of the heat it removes. Best-in-class centrifugal chillers hit 0.5-0.6 kW/ton (0.14-0.17 kW per kW of cooling) at full load; absorption chillers are 0.7-0.9.

When mechanical chillers are required:

• Climate is warm year-round (>30 °C wet-bulb regularly).
• Facility water needs to be cold (e.g., feeding a CRAH at 7 °C).
• A precision-control loop where ambient drift would propagate to chip temperatures.

Why DLC + chiller is wasteful: you spent capex on direct-to-chip plumbing so you could run warm water (25-35 °C inlet). If your chiller produces 12 °C water you've thrown away the efficiency. DLC plants are best paired with non-chiller heat rejection.

Cooling tower (open / closed evaporative)

Wet cooling tower — water sprays over fill, fan pulls air through, evaporation drops the water temperature toward wet-bulb. Capable of producing condenser water at ambient wet-bulb + ~3-5 K approach.

In a 25 °C ambient at 50% RH (wet-bulb ~18 °C), a tower produces ~21-23 °C water. That's perfectly compatible with DLC (which wants 25-35 °C inlet, so you'd actually warm it up at the rack).

PUE impact: cooling tower fans + condenser pumps are ~1-3% of the cooling load. Total facility PUE in the 1.05-1.15 range is achievable.

WUE cost: evaporation. A cooling tower removing 1 MW continuously evaporates ~1500 L/hour, plus 200-400 L/hour of "blowdown" (purged water to control mineral concentration). At 1.5-3.0 L/kWh of IT load, the water bill is real and the regulatory exposure (in drought regions) is non-trivial.

Dry cooler (radiator + fans)

Closed-loop air-to-water radiator. No evaporation, no water consumption. Approach temperature is much worse: ambient dry-bulb + 5-12 K. In a 35 °C dry-bulb day a dry cooler produces 40-47 °C water — too warm for traditional CRAH but fine for DLC at warm-water temperatures.

Where dry coolers fit:

• Cool to moderate climates (Northern Europe, Pacific Northwest, parts of Canada).
• Combined with DLC where 35-45 °C facility supply is acceptable.
• Water-stressed regions where evaporative is regulatorily blocked.

PUE: 1.10-1.20 typical, slightly worse than evaporative because fans run more. WUE: ~0 (initial fill only, plus minor makeup for evaporation/leakage).

Hybrid (dry + adiabatic boost)

Dry cooler with a misting / pad system that activates only above a threshold temperature. Most of the year it runs dry (no water); on hot days adiabatic pre-cooling drops the inlet air toward wet-bulb, recovering most of the evaporative tower's performance.

Compromise position: PUE 1.10-1.18, WUE 0.3-0.7 L/kWh. Good fit for moderate climates with summer extremes (Texas, Spain, Northern California).

Free cooling / sea-water / deep-lake / sewage

Specialized: pumping sea water (Stockholm, Singapore proposals), deep lake water (Toronto Lake Ontario), or treated sewage (Microsoft pilot) through heat exchangers. Capex high, opex low, geographically constrained. Where you can do it, PUE drops below 1.10.

Heat reuse

The kilowatts you reject have to go somewhere. Increasingly, "somewhere" is district heating loops (Stockholm, Helsinki, parts of Germany) where the facility's warm water heats nearby buildings instead of being dumped. Doesn't change PUE in the conventional sense but eliminates one waste stream.

For warm-water DLC at 45 °C return temperature, the water is hot enough to feed many low-temperature district heating systems directly. This is becoming a planning requirement in some EU jurisdictions.

Setting up to fail-safe: what happens during a cooling event

Different cooling architectures have very different "what if" stories. Knowing them in advance is the difference between graceful degradation and a smoking rack.

Air cooling: minutes of margin

If a CRAH fails (or a chilled-water valve closes), room air slowly heats up. A 30 kW rack in a 200 m³ room with no fresh cool air rises ~1 °C per minute initially. Operators have 10-20 minutes to investigate before GPU thermal throttling.

Standard response:

1. Identify failed unit (BMS alert, walking the row).
2. If recoverable in <5 min, recover; if not, balance load by powering down lower-priority gear.
3. If room temp keeps climbing, drain training jobs and shut down GPUs gracefully before throttling and shutdowns happen at random.

DLC: seconds of margin

If a CDU pump fails or a manifold valve closes, flow stops. Coolant in the cold plate heats from 25 °C to >80 °C in 30-90 seconds depending on load. GPUs at full load throttle at ~85 °C junction and shut down at ~95 °C.

Standard response:

1. CDU alert fires (flow drop, pressure drop, temperature rise).
2. Automatic GPU power-cap or shutdown should trigger. This has to be wired in advance — manual response isn't fast enough.
3. Operator investigates after the racks are safe.

The wiring in advance is critical. Best practice:

• BMS detects CDU fault.
• BMS sends Redfish power-cap command to all nodes in affected racks within 5 seconds.
• BMS escalates to graceful-shutdown if cooling not restored in 60 seconds.
• BMS escalates to power-off if temperatures still rising at 120 seconds.

A cluster running production training jobs without this wiring will lose hardware in a CDU event. The CDU isn't your single point of failure — the absence of automated protection is.

Redundancy: N+1 CDUs

For 120 kW racks the standard is N+1 CDUs feeding a shared manifold ring, with valves that allow isolation of any single unit. Pump and heat-exchanger redundancy inside each CDU. The investment is meaningful (each unit is $50-150k) but it converts CDU failure from "rack outage" to "scheduled service."

Operating envelopes: cold start, ramp, steady state

A GPU rack doesn't operate at constant load. The cooling plant has to handle three distinct regimes.

Cold start

Room and loops are at ambient (say 22 °C). Nodes power on, GPUs idle. Heat load is minimal (~10% of design). Watch for:

• Condensation if outside is humid and loops are still cool. Run dehumidification if RH > 60%.
• CRAH set-point overshoot — controllers tuned for steady state may swing wildly during ramp.

Ramp

Workload starts; GPU power climbs from 100 W idle to 700+ W under training in seconds. Coolant temperature rise lags the heat input by 15-60 seconds depending on loop volume and flow. CRAH and CDU controllers have to keep up.

Where this fails:

• A CDU sized exactly to design heat load, with no margin. When all 18 trays simultaneously hit peak it briefly exceeds capacity, secondary outlet temp spikes 5-8 K, and one or two GPUs get a thermal warning.
• A CRAH on aggressive economy mode that isn't ready to deliver design airflow within 30 seconds.

Mitigation: oversize the cooling plant by 15-25% over design heat. The extra capacity costs less than the alternative.

Steady state

The cooling plant is in its happy place. Coolant inlet at 25 °C (DLC) or air inlet at 22 °C (air); deltas constant; pumps and fans modulating to hold setpoint. Operator should see:

• GPU junction temps in a tight band (within 5 K across all GPUs in a rack).
• Coolant ΔT stable at design value.
• Hot-aisle / return-air temps stable.

Drift in steady state is the early warning. A 1 K rise per week in coolant inlet temp at constant load is a heat-rejection plant slowly degrading (fouled fill, scaled tubes, dropping fan output). Catch it at week 2, fix it at week 3, before it becomes "the cluster started throttling on Tuesday."

Maintenance windows that actually work for liquid

You can't take a 120 kW rack offline casually. Plan windows in advance.

Quarterly — non-disruptive

• Water sample for chemistry analysis (drop into a bottle from a sample port; takes 2 min).
• Visual inspection of all manifolds, quick-disconnects, drip trays.
• Filter pressure-drop reading; replace if >0.3 bar over baseline.
• Pump amperage reading; investigate if >10% above baseline.

Annually — minor disruptive

• CDU heat-exchanger cleaning (chemical, in-place).
• Filter replacement (regardless of pressure drop).
• Biocide / corrosion-inhibitor refresh.
• Calibration of flow and temperature sensors.

Every 3-5 years — full drain

• Drain and refill the secondary loop. Inspect cold plates for fouling at random sample.
• Replace pump seals.
• Replace any aging quick-disconnects.

These are not optional. A loop that's never drained eventually fails — and the replacement cost dwarfs the maintenance cost.

Cooling decision matrix

Quick decision rule when designing a new room:

The honest answer for a new GPU build today is: plan for DLC. Even if your initial deployment is H100-class and air works, your second wave of hardware is going to be Blackwell-class and air won't. Building a hall that can accept DLC retrofit later — facility chilled water at warm-water temperatures, structural floor for CDU weight, manifold pathways — costs ~10% more than a pure-air design but saves you a building rebuild in three years.

Retrofit: bringing liquid into an air-cooled hall

Most operators will face a retrofit at some point — taking a hall that was built for 15-20 kW air-cooled racks and adding a row of 80-120 kW liquid racks. The transition is doable but has constraints.

What you need to add

• Primary chilled-water plant sized for the new heat load, plus piping into the room. New piping pathway: structural ceiling penetrations, drip pans under runs over IT space.
• CDUs in the new row(s) — in-row CDUs are most common for retrofit because they don't need a centralized plant rebuild.
• Power upgrade for the higher-density racks: typically 480 V three-phase service brought to the new row from the main switchgear.
• Slab reinforcement or raised-floor removal under the heavy NVL72-class racks (1500-1800 kg loaded).
• Leak detection integrated with BMS, including conductive cables along all new manifolds.
• Drain provisions — somewhere safe to dump the secondary loop during maintenance.

What you can keep

• The room's existing CRAH for residual air load (servers' DRAM, NVMe, fans still need air).
• The hot/cold-aisle containment for the unaffected rows.
• The existing UPS and generator if sized for the additional load.

The retrofit failure mode

Underestimating residual air load. Operators install DLC and assume "100% of heat goes to liquid." The reality is 80-85%. A 100 kW liquid rack still puts 15-20 kW into the room as warm air. If your existing CRAH can handle it, fine; if it was already at capacity, the room temperature climbs and adjacent racks suffer.

Always size the residual air capacity at 20-25% of the new liquid rack power. Verify after install that hot-aisle / row-end temperatures stay in spec.

Cooling instrumentation that detects problems early

A liquid plant is more instrumented than air, by necessity. Here's the minimum operator-visible telemetry per CDU + per rack:

# CDU primary side (facility water)
cdu_primary_inlet_temp_c{cdu=N}
cdu_primary_outlet_temp_c{cdu=N}
cdu_primary_flow_lpm{cdu=N}
cdu_primary_pressure_bar{cdu=N}

# CDU secondary side (rack-internal coolant)
cdu_secondary_inlet_temp_c{cdu=N}
cdu_secondary_outlet_temp_c{cdu=N}
cdu_secondary_flow_lpm{cdu=N}
cdu_secondary_pressure_bar{cdu=N}
cdu_secondary_pressure_drop_bar{cdu=N}
cdu_pump_power_w{cdu=N, pump=A|B}
cdu_pump_state{cdu=N, pump=A|B}  # ok / fail / standby
cdu_filter_dp_bar{cdu=N}

# Per-rack manifold
rack_coolant_inlet_temp_c{rack=R}
rack_coolant_outlet_temp_c{rack=R}
rack_coolant_flow_lpm{rack=R}
rack_leak_state{rack=R, sensor=N}  # ok / wet
rack_drip_pan_state{rack=R}  # ok / water

# Heat-rejection plant
chiller_kw_per_kw_cooling{plant=N}  # COP equivalent
tower_water_makeup_lpm{plant=N}
tower_blowdown_lpm{plant=N}
ambient_drybulb_c
ambient_wetbulb_c
ambient_rh_pct

Alert rules that catch real problems:

• cdu_secondary_flow_lpm < threshold * 0.85: pump or valve issue
• cdu_secondary_pressure_bar drop > 0.5 bar in 60 s: probable leak — page on-call, prepare drain
• rack_leak_state == wet OR rack_drip_pan_state == water: emergency, immediate response
• cdu_filter_dp_bar > baseline + 0.3: schedule filter swap
• chiller_kw_per_kw_cooling > 1.2 * baseline: degrading performance, schedule maintenance
• rack_coolant_outlet_temp - rack_coolant_inlet_temp != design_dT +/- 5K at constant load: heat-balance drift

The leak alerts get the most attention because they're rarest and most damaging. Test them quarterly by deliberately wetting a sensor with a few mL of water — if the alert doesn't fire, the wiring is broken.

Acoustic considerations: GPU rooms are loud

Modern AI halls run loud. Air-cooled racks at 50 kW push fans hard; even DLC racks have residual-air fans that contribute. Typical noise levels:

• Single DGX H100 at idle: ~75-80 dBA at 1 m.
• Single DGX H100 at full load: ~85-95 dBA at 1 m.
• Hot-aisle of an air-cooled GPU row: 95-105 dBA at 1 m.
• DLC rack hot-aisle: 80-90 dBA (lower because residual air is much less).

OSHA permissible exposure limit is 90 dBA for an 8-hour shift. AI halls regularly exceed this. Hearing protection in hot aisles is mandatory. Some operators issue earmuffs at the door.

For dense halls, noise can drive layout decisions: workspaces where operators sit for hours need to be physically isolated from the IT space, often with a glass-walled NOC adjacent to the hall.

Climate matching: where to put the building

The site's climate determines which cooling architecture delivers the best PUE/WUE.

The cold-continental regions are getting saturated because every hyperscaler wants them. The next wave is hot-dry where water rights are negotiated upfront, and the wave after that is hot-humid with novel cooling (dry coolers paired with industrial heat pumps for waste heat reuse).

For site selection, the cooling math is rarely the only factor — power price, latency, regulatory, and fiber connectivity all matter. But cooling is what determines whether your PUE 1.10 plan works.

See also

• Power — the heat input side; PUE and WUE numbers live there
• Rack design — how cooling choice constrains rack layout
• Physical network — cable plant routing past liquid manifolds
• InfiniBand implementation — bring-up sequence, including thermal soak tests
• Health check runbook — daily checks for thermal throttling, GPU temps

Sources

• ASHRAE TC 9.9 Thermal Guidelines for Data Processing Environments (5th ed.): A1-A4 envelopes, recommended 18-27 °C, allowable up to 45 °C (A4), altitude derating.
• NVIDIA DGX SuperPOD H100 Reference Architecture: 32-node SU layout, "rack power exceeds 40 kW" air-cooled.
• NVIDIA GB200 NVL72: 72 GPUs/rack, ~120 kW continuous, 25 °C inlet / ~20 K rise / ~2 L/s flow, liquid at compute and NVSwitch trays.
• NVIDIA Cooling Requirements (public): cold-plate inlet ≤45 °C, ≥2-3 L/min per module on GB200-class.
• Open Compute Project Door HX Whitepaper: passive RDHX 25-35 kW/rack, active 60-80 kW typical, up to 150 kW in advanced designs.
• Microsoft Sustainable Datacenter blog (2024): zero-water designs, fleet WUE 0.30 L/kWh.
• AWS public sustainability disclosure: 1.15 global PUE (2024).