Meeting the Cooling Demands of Modern Data Centers: From Air to Liquid with AxSTREAM

Introduction

To match the computing demand of advanced AI and cloud computing tasks, data centers are deploying the latest chips whose thermal design power continues to rise. This in turn increases the required cooling load of the servers. If the data center cooling load increases too much, traditional air-cooling methods may be insufficient, requiring liquid cooling approaches to be used. In this blog, we explore how AxSTREAM System Simulation can model several data center cooling schemes, from traditional air-cooling systems to those employing liquid cooling.

Existing Cooling Methods

As the data center operates, all the energy consumed by the server chips is ultimately converted into heat. This heat must be dissipated via a cooling system, meaning that the data center cooling systems require approximately the same amount of power consumption as the energy consumption of the servers (Fig. 1). Given that the cooling systems use a large portion of the total power consumption of the data center, improvements to the cooling system architecture can lead to significant energetic savings.

Traditional air-cooling methods use a hot and cold aisle approach (Fig. 2). Typically, a computer room air conditioner (CRAC) provides cold air to the data center. The conditioned air is sent below the floor and enters the data center via perforated floor tiles adjacent to the server racks. Server fans then move the cold air across the server equipment for cooling. In a hot and cold aisle arrangement, the cold intake and hot exhaust of adjacent server racks face each other, which helps prevent mixing of the conditioned air with the hot exhaust.

There are several approaches to liquid cooling for server equipment such as rear-door heat exchangers and immersion cooling. The method being analyzed in this blog is direct-to-chip liquid cooling (Fig. 3) where cold plates sit atop the server’s heat generating units, such as the CPUs, GPUs, and memory modules. Cooling fluid then flows through the cold plate to absorb heat generated by the server components.

Comparison of Cooling Approaches

A study was performed using AxSTREAM System Simulation to understand how the cooling system may behave as liquid-cooled server equipment is added to a data center with existing air-cooled equipment. The first system configuration to be modeled was a baseline air-cooled system (with no additional liquid cooling) scaled for the cooling loads of a 100 MW hyperscale data center (Fig. 4). In this model, a vapor compression system with R1233zd(E) is used in the chiller plant. The chiller plant chills cooling water which is sent to the CRAC heat exchangers. The cooling water is used to condition the air in the data center, which is ultimately used to dissipate the heat loads provided by the server equipment. Fig. 4 shows the data center having 3 CRAC heat exchanger elements and 3 lumped server heat load elements.

Next, it is assumed that additional liquid-cooled server hardware is installed which increases the power consumption and cooling load of the data center. For simplicity’s sake, it was assumed the amount of required cooling increased by 10% due to the added hardware. When the cooling water is provided directly from the evaporator (Fig. 5), the compressor work also increases by 10%. However, when the cooling water is provided from the exhaust of the CRAC heat exchangers (Fig. 6), there is an increase in the water temperature supplied to the evaporator, resulting in improved performance of the chiller plant refrigeration sub-loop and the compressor work only increasing by 9.3%. Thus, there is a slight improvement in the cooling system performance compared to the baseline case.

 

Conclusion

AxSTREAM System Simulation provides an efficient method for performing both design and advanced simulation studies. Utilizing a rapid system modelling approach, design engineers can quickly analyze and compare the performance of several data center cooling system configurations. Identifying the most effective cooling approach can lead to large benefits for the data center operator, including reduced system capital and O&M costs, as well as higher system energetic efficiency.

To learn more about using AxSTREAM System Simulation for cooling system modeling, schedule a meeting with our team at Info@SoftInWay.com.

References

[1] E. Masanet and N. Lei, “How Much Energy Do Data Centers Really Use?,” Energy Innovation, 2020.

[2] The Severn Group, “Data Center Hot Aisle/Cold Aisle Layout Design,” 2016.

[3] HDR, Inc., “Direct-To-Chip Liquid Cooling,” 2023.