Data Centers - They’re Back… E SOURCE Forum September, 2007 William Tschudi

Acknowledgement to LBNL sponsors California Energy Commission – PIER program California Energy Commission – PIER program Pacific Gas and Electric Company Pacific Gas and Electric Company New York State Energy and Development Agency (NYSERDA) New York State Energy and Development Agency (NYSERDA) US - Environmental Protection Agency US - Environmental Protection Agency US – Department of Energy US – Department of Energy

Data Center research activities Research Roadmap Research Roadmap Benchmarking and 23 data center case studies Benchmarking and 23 data center case studies Self-benchmarking protocol Self-benchmarking protocol Power supply efficiency study Power supply efficiency study UPS systems efficiency study UPS systems efficiency study Standby generation losses Standby generation losses Performance metrics – Computation/watt Performance metrics – Computation/watt EPA report to Congress EPA report to Congress DOE Save Energy Now program DOE Save Energy Now program

LBNL data center demonstrations “Air management” demonstration Outside air economizer demonstration DC powering demonstrations

Data center energy roadmap July, 2003 – Jay Stein a major contributor July, 2003 – Jay Stein a major contributor The problem receded for several years as excess data center space was absorbed The problem receded for several years as excess data center space was absorbed Now the concerns are back – and they are badder than ever! Now the concerns are back – and they are badder than ever!

Performance varies The relative percentages of the energy actually doing computing varies considerably.

Percentage of electricity delivered to IT equipment Average.57 Higher is better Source: LBNL Benchmarking

Benchmark results helped to find best practices The ratio of IT equipment power to the total is an indicator of relative overall efficiency. Examination of individual systems and components in the centers that performed well helped to identify best practices. Lets talk about a few….

Server Load/ Computing Operations Cooling Equipment Power Conversion & Distribution Alternative Power Generation High voltage distribution Use of DC power Highly efficient UPS systems Efficient redundancy strategies Load management Server innovation Energy efficiency opportunities are everywhere Better air management Move to liquid cooling Optimized chilled-water plants Use of free cooling On-site generation Waste heat for cooling Use of renewable energy/fuel cells

Best practices topics identified through benchmarking

A word about appropriate environmental conditions… ASHRAE published thermal guidelines ASHRAE published thermal guidelines –All IT suppliers participated –Guidelines allow most centers to relax setpoints Recommended and allowable ranges of temperature and humidity are provided – at the inlet to the IT equipment Recommended and allowable ranges of temperature and humidity are provided – at the inlet to the IT equipment High temperatures in the “hot aisles” and return to air conditioners is desirable. High temperatures in the “hot aisles” and return to air conditioners is desirable.

Humidity guidelines – at the inlet to IT equipment ASHRAE Allowable Maximum ASHRAE Allowable Minimum ASHRAE Recommended Maximum ASHRAE Recommended Minimum

Temperature guidelines – at the inlet to IT equipment ASHRAE Allowable Maximum ASHRAE Allowable Minimum ASHRAE Recommended Maximum ASHRAE Recommended Minimum

Air management best scenario – isolate cold and hot 70-75º º

Another isolation scheme

Measured fan energy savings – 75% If mixing of cold supply air with hot return air can be eliminated- fan speed can be reduced

Better temperature control can allow raising the temperature in the entire data center! ASHRAE Recommended Range Ranges during demonstration

Best practices – Free cooling with air economizers

Encouraging outside air economizers Issue: Issue: –Many are reluctant to use air economizers –Outdoor pollutants and humidity control considered equipment risk Goal: Goal: –Encourage use of outside air economizers where climate is appropriate Strategy: Strategy: –Address concerns: contamination/humidity control –Quantify energy savings benefits

Outdoor measurements IBM Standard EPA Annual Health Standard EPA 24-Hour Health Standard and ASHRAE Standard

Measurements inside the centers IBM Standard EPA Annual Health Standard EPA 24-Hour Health Standard and ASHRAE Standard

Indoor measurements

Data center w/economizer

Humidity measurements ASHRAE Recommended Upper Limit ASHRAE Recommended Lower Limit ASHRAE Allowable Lower Limit ASHRAE Allowable Upper Limit

Findings Water soluble salts in combination with high humidity can cause failures Water soluble salts in combination with high humidity can cause failures Static electricity can occur with very low humidity Static electricity can occur with very low humidity New ASHRAE particle limits drastically lower than manufacturer standard New ASHRAE particle limits drastically lower than manufacturer standard Particle concentration typically (no economizer) an order of magnitude lower than new ASHRAE limits Particle concentration typically (no economizer) an order of magnitude lower than new ASHRAE limits Economizers, without other mitigation, can cause particle concentration to approach new ASHRAE limits but filtration can mitigate this Economizers, without other mitigation, can cause particle concentration to approach new ASHRAE limits but filtration can mitigate this

Best practices – power conversion

Inverter InOut Bypass Battery/Charger Rectifier Internal Drive External Drive I/O Memory Controller  Processor SDRAM Graphics Controller DC/DC AC/DC DC/DC AC/DC Multi output PS Voltage Regulator Modules 5V 12V 3.3V 12V 1.5/2. 5V 1.1V- 1.85V 3.3V 12V PWM/PFC Switcher Unregulated DC To Multi Output Regulated DC Voltages Data center power conversions AC voltage conversions

Prior research illustrated large losses in power conversion Uninterruptible Power Supplies (UPS) Power Supplies in IT equipment

With over 25 industry partners direct DC powering of servers was demonstrated

Typical AC distribution today 480 Volt AC

Facility-level DC distribution 380V.DC 480 Volt AC

AC system loss compared to DC AC system loss compared to DC 7-7.3% measured improvement 2-5% measured improvement Rotary UPS

What’s new Infrared thermography can be used as a visualization tool

websites:

Discussion/Questions??