1. Energy Savings in CERN’s Main Data Centre
Vincent Doré & Wayne Salter
IT Computing Facilities Group
Mauro Nonis & Sabri Masrie
EN Cooling and Ventilation Group

2. Overview of CERN’s Data Centre Cooling system for main room
Summary
Overview of CERN’s Data Centre
Cooling system for main room
Situation before 2008
Improvements since 2008
Foreseen further improvements
Energy savings
Conclusions
Energy Savings in CERN’s Main Data Centre - 2

3. CERN’s Main Data Centre
Data centre built in early 1970s
Has been refurbished/upgraded several times
Has three main computer rooms:
1440m2 main room with air cooling
1240m2 basement room mixture of air/water cooling
225m2 room for critical servers only with water cooling
Data B/U in a separate building
3.5 MW available for computing equipment
Energy Savings in CERN’s Main Data Centre - 3

4. Main Room Cooling System
3+1 chillers (each 1.45 MW)
9+1 AHUs just under roof
Cold air passes down through large ducts and under the false floor
Enters cold aisle through perforated tiles
Server fans draw the air though the servers
Exiting hot air sucked out via the false ceiling
Energy Savings in CERN’s Main Data Centre - 4

5. Situation before 2008 From 70s to 90s mainframes Early 00s tower PCs
In 2007 started with rack mounted PCs
No hot/cold aisle separation
Inefficient cooling
High energy consumption
Not possible to properly cool in the summer
Energy Savings in CERN’s Main Data Centre - 5

6. Installation of cold corridors (2008)
Enclosed cold aisles
Doors from rack supplier
Roof made at CERN
Strict hot/cold air separation
All servers comfortably cooled even during very hot weather
Unfortunately energy savings from this are not known
Energy Savings in CERN’s Main Data Centre - 6

7. Optimisation of air distribution (2010/2011)
Analysing and modifying the operation of AHUs to increase the use of free cooling by
improving measurement and implementing full automation of air intake and mixing
reducing the intake of hot exhaust air
modifying the anti-freeze protection
Maximum server inlet temperature increased from 25 ºC to 27 ºC (ASHRAE)
Set points increased
temperature of air produced by AHUs increased in steps from 14 ºC to 22 ºC
temperature of chilled water circuit increased from 5 ºC to 8 ºC
Thermal analysis
Energy Savings in CERN’s Main Data Centre - 7

8. Other measures since 2011 Improved monitoring
Identified and fixed leaks between the hot and cold plenums
Now able to run with 4 from 10 AHUs switched off
Investigate the possibility to change the fixed speed motors of the AHUs to variable speed motors
Allow better control of the flow rate to meet actual needs
Introduction of water cooled racks; first active and now passive
Increasing power density per rack
Energy Savings in CERN’s Main Data Centre - 8

9. What are the energy savings?
Not known exactly!
Missing important monitoring data before 2010
Chillers are used not only for main room but also for other rooms
Nevertheless we estimate for the changes made in 2010/2011 for the main room
With DC load of 2.6 MW and 1.9 MW in main room
Annual power consumption of chillers decreased from 6.5 GWh to 1.4 GWh
PUE for the entire DC reduced from 1.7 to 1.5
Switching off 3 AHUs (40 kW each) saves an additional 1.1 GWh
Chiller Energy Consumption
For Main Computer Room
July
Nov
Feb
Jan
Energy Savings in CERN’s Main Data Centre - 9

10. Thanks to the CERN EN/CV Group.
Conclusion
Data centre built during mainframe era
Has needed to be adapted to changing computing technologies
Major improvements have been made:
Installation of hot/cold aisles
More complete use of free cooling
More aggressive temperature environment
Impressive energy savings achieved through fairly simple measures
Further measurements are continually being sought
Thanks to the CERN EN/CV Group.
Energy Savings in CERN’s Main Data Centre - 10