AI Data Center Cooling: Planning Around the NVIDIA GPU Roadmap

As AI accelerator rack power rises from today's roughly 100 kW per rack toward 600 kW and beyond, cooling architecture is fast becoming the defining constraint in data center design. Air cooling remains effective for high-density AI data centers in the low-to-mid density range, but as rack power continues to climb, direct-to-chip liquid cooling is emerging as the critical pathway for absorbing significantly higher power densities. Two-phase direct-to-chip liquid cooling, which exploits phase change within the cold plate, extends the density curve even further, delivering the scalability needed for rack power levels between 200 kW and 1 MW+. In this landscape, a forward-looking data centre liquid cooling system is no longer a future upgrade, but a strategic prerequisite for deploying next-generation AI infrastructure.

The NVIDIA GPU roadmap is driving rack-level power density upward at an unprecedented pace. From today's Blackwell-class racks in the 100 to 130 kW range, to a future Rubin Ultra platform approaching 600 kW per rack and subsequent platforms climbing still higher, the thermal design envelope of the AI data center is being compressed on every front. Cooling architecture must therefore evolve from single-generation platform fit to multi-generation roadmap planning. AI data center liquid cooling decisions made today will determine whether thermal capacity keeps pace with the next several generations of GPU deployment, or becomes the bottleneck that stalls them.

This shift is best understood as a power density curve, in which each cooling technology occupies a distinct zone of applicability. Air cooling and enhanced air cooling remain effective at low to mid densities. Single-phase direct-to-chip liquid cooling covers the mid-to-high range and serves as the essential bridge from air cooling to higher-density liquid cooling. Two-phase direct-to-chip liquid cooling absorbs very high heat flux through phase-change heat transfer, extending the scalable reach of direct-to-chip cooling into still higher density territory. Immersion cooling represents a system-level architecture in its own right, not a linear extension of air or direct-to-chip cooling but a complete transformation of the heat transfer interface and operating model by submerging IT hardware in dielectric fluid. For high density data center cooling, the most critical zone for near-term deployments is neither the extreme low nor the extreme high end, but the transition band in between, where air cooling approaches its limits, single-phase direct-to-chip liquid cooling moves from optional to baseline, and two-phase solutions add further headroom.

Within this transition band, air cooling remains a dependable choice at low-to-mid rack densities, provided that airflow organization, hot/cold aisle containment, room layout, and air volume management are handled properly. Rear-door heat exchangers (RDHx) extend the conventional air-cooling envelope, typically pushing it into the 40 to 80 kW per rack range depending on facility conditions, and can serve as an intermediate step on the path from air cooling to liquid architectures. Beyond that, however, air cooling becomes progressively constrained by growing airflow demand, rising fan energy, space limitations, noise restrictions, and integration complexity. Conventional air-cooled layouts grow inefficient past roughly 30 to 40 kW, and even with RDHx the practical boundary usually stops at 40 to 80 kW. This is exactly why rack cooling suppliers are putting liquid-based solutions at the center of their high-density roadmaps, as these outperform air cooling in efficiency, predictability, and facility adaptability at higher densities.

Immersion liquid cooling comes in two principal forms. Single-phase immersion uses liquid to absorb heat and circulate it to a heat exchanger without changing phase, while two-phase immersion relies on the fluid boiling at the device surface and re-condensing on a condenser surface, using phase change to lift heat transfer efficiency. Immersion supports mid-to-high rack-equivalent power densities and suits newly built or highly customized facilities, but it brings hardware compatibility, tank maintenance, fluid management, structural loading, and retrofit challenges. Direct-to-chip cooling (DTC), by contrast, attacks heat at its source. Cold plates extract heat near the chip or package, reducing thermal resistance and establishing a controlled thermal path from the chip, through the rack liquid loop, to facility water. A typical DTC deployment comprises cold plates, tubing, quick disconnects, rack manifolds, the data center air liquid CDU (coolant distribution unit), and the heat exchange interface to the building's chilled-water system.

Within DTC, both single-phase and two-phase cold plates can cool AI accelerators; the right choice depends on target rack density, chip heat flux, facility water temperature, pressure-drop limits, maintenance requirements, and future expansion needs. The reason DTC is becoming a foundational capability is clear. Local heat flux on modern accelerator chips frequently exceeds 100 W/cm², shifting the dominant engineering problem from room-level heat rejection to chip-level heat extraction. If the cold plate cannot remove heat effectively at the chip, no amount of airflow organization, rear-door cooling, or room air-conditioning capacity can hold junction temperature in check. This is where the cold plate CDU manufacturer's engineering becomes decisive. An effective DTC package must address chip heat flux, coolant flow rate and distribution uniformity, the pressure drop imposed by microchannel structures, supply water temperature and approach-temperature limits, and junction-temperature margin under dynamic loads.

Single-phase cold plates offer maturity, predictable performance, and straightforward integration with existing water or glycol systems, making them well suited to many current Blackwell-class deployments and a portion of near-term Rubin deployments. Two-phase cold plates harness the phase change of the working fluid inside the plate to absorb very high heat flux at lower flow rates, extending the reach of direct-to-chip cooling into higher-density racks. This capability becomes increasingly relevant as the GPU roadmap pushes toward the 200 kW to 1 MW+ rack range. The choice between single-phase and two-phase should therefore never rest on point heat-transfer performance alone; it must weigh rack power, facility conditions, operational capability, and the future platform roadmap. As the industry designs the next generation of GPU cabinet air liquid CDU solutions, the goal is not complexity for its own sake, but a cooling architecture that keeps absorbing the rising power density of current and successive GPU platforms.

Data center thermal infrastructure typically carries a long lifecycle, while GPU platforms iterate rapidly. If the cooling architecture is designed around a single GPU generation, bottlenecks can surface within one or two platform cycles. Aligning the thermal architecture with a multi-generation GPU roadmap lowers the risk of retrofits, idle infrastructure, over-design, and unnecessary capital expenditure, while reducing deployment delays caused by thermal limits. That means treating thermal planning as a forward-looking design discipline. Early-stage design should factor in expected chip TDP, rack power density, supply water temperature, CDU capacity, flow distribution, and cooling-technology switch points. The facility side must keep pace as well, covering supply water temperature, CDU capacity, rack-level flow distribution, chilled-water system capability, and room for expansion. Otherwise, even the most capable server-side cold plates will find that the facility side, from the server rack liquid cooling CDU to the water system behind it, becomes the new constraint. With NVIDIA's sustained power-density growth making cooling architecture a core variable in AI data center deployment, the sensible path is to plan cooling not for a single generation, but for the multi-generation density trend ahead.

Against this backdrop of fast-evolving AI hardware, scalable, holistic liquid-cooling hardware is vital to turn forward-looking thermal planning into reality. As AI expands, so does the challenge of keeping it cool. That's where EXTRCOOL comes in—offering a turnkey liquid-cooling setup that brings together CDUs, cold plates, manifolds, and pipework under one roof. You can select ready-made units or collaborate with us on a tailor-made configuration. Our designs maximise heat removal, minimise energy draw, and reduce environmental impact—backed by flawless operation at gigawatt-scale sites around the world. For critical applications where every minute counts, EXTRCOOL is the reliable choice. Discover the full product line or request a custom quote here: https://www.extrcool.com