1. Dialing it in: A Year of Tuning the NCAR-Wyoming Supercomputing Center Aaron Andersen, Deputy Director of Operations, National Center for Atmospheric Research(NCAR)

2. Aaron Andersen Nov., 2013 “… to provide, or arrange for provision of facilities for the scientific community as a whole whose initial cost and upkeep lie beyond the capability of individual universities or research groups.” Preliminary Plans for a National Institute for Atmospheric Research, 1959, NCAR Blue Book NCAR’s Founding Mission

3. Aaron Andersen Nov., 2013 History of Computing at NCAR Exotic Cooling Freon R-22 Fluorinert Water Return of Liquid Cooling Water Warm Water FR3

4. Aaron Andersen Nov., 2013 Key to Optimum Performance (Three M’s) Monitoring Measurement Maintenance Infrastructure Support Group – Cheyenne Electricians Mechanics Building Automation Controls HVAC Technician Commissioning Agent Measurement & Verification 12 months post completion

5. Aaron Andersen Nov., 2013 Tuning the Facility – Student Interns Clair Christofersen - 2012 University of Miami Ohio BS Mechanical Engineering – Applied Math Air Flow Modeling (CFD Analysis) Jason Jones - 2012 Clemson University -> Georgia Tech BS Mechanical Engineering Air Flow, Water Flow and Total Energy Ademola Olarinde - 2013 Texas A&M Kingsville MS Mechanical Engineering

6. Aaron Andersen Nov., 2013 65F (18.3 C) Degree Chilled Water Evaporative Solution

7. Aaron Andersen Nov., 2013 Liquid Cooling – ASHRAE 2011 Guidelines 121 hrs 1.38% 8639 hrs 98.62%

8. Aaron Andersen Nov., 2013 Characterize Building Response

9. Aaron Andersen Nov., 2013 Saw an increase in mechanical consumption in winter months Energy Recovery Heat Pumps consuming way more electricity than expected Why do we care? $5000 per month What is the root cause? Mechanical Load in Winter

10. Aaron Andersen Nov., 2013 Heat Rejection from Yellowstone is too low 1.2MW vs. 4MW Design Significantly greater heating demand from the building during winter months Commissioning Agent identified the same problem and possible culprit, large Air Handling Units Root Cause

11. Aaron Andersen Nov., 2013 Why This Matters

12. Aaron Andersen Nov., 2013 UPS Efficiency Only Critical Systems are Covered on UPS Disk Network AuthN/AuthZ Login Management Low load causes low efficiency Firmware upgrade moves UPS efficiency from 70% ~ 95% Maintenance really matters Cleaning strainers improve pump energy by 20 KW More frequent maintenance than recommended Water Usage Zero Blow down for Cooling Tower saving 40% Negotiated reduction in Sewer cost ~$2500/month Other Issues

13. Aaron Andersen Nov., 2013 PUE Trend Since Startup

14. Aaron Andersen Nov., 2013 PUE – One Year Aug 2012 – July 2013