Data Center Research Roadmap A “research roadmap” developed for the California Energy Commission outlines key areas for energy efficiency research, development, and demonstration – This includes strategies that can be implemented today.

Data Center research activities Benchmarking and 22 data center case studies Best practices identified Self-benchmarking protocol Power supply efficiency study UPS systems efficiency study Standby generation losses Performance metrics – Computation/watt

LBNL data center demonstration projects “Air management” Outside air economizer Contamination concerns Humidity control concerns DC powering Facility level Rack level

LBNL data center Federal projects Case studies Technical assistance Emerging technology Investigating use of infrared thermography as a visualization tool

Benchmarking energy end use

IT equipment load density

Overall power use in Data Centers Courtesy of Michael Patterson, Intel Corporation

Performance varies The relative percentages of the energy actually doing computing varies considerably.

Percentage of power delivered to IT equipment Average 0.49

Benchmark results helped to find best practices The ratio of IT equipment power to the total is an indicator of relative overall efficiency. Examination of individual systems and components in the centers that performed well helped to identify best practices. Lets talk about a few….

Best practices topics identified through benchmarking

A word about appropriate environmental conditions… ASHRAE published thermal guidelines Majority of IT suppliers participated Guidelines allow most centers to relax setpoints over standard practice Recommended and allowable ranges of temperature and humidity are provided – at the inlet to the IT equipment High temperatures in the “hot aisles” and return to air conditioners is desirable.

Temperature guidelines – at the inlet to IT equipment ASHRAE Allowable Maximum ASHRAE Recommended Maximum ASHRAE Recommended Minimum ASHRAE Allowable Minimum

Humidity guidelines – at the inlet to IT equipment ASHRAE Allowable Maximum ASHRAE Recommended Maximum ASHRAE Recommended Minimum ASHRAE Allowable Minimum

Best scenario – isolate cold and hot 95-100ºF 70-75ºF

Another isolation scheme

Fan energy savings – 75% If mixing of cold supply air with hot return air can be eliminated- fan speed can be reduced

Better temperature control can allow raising the temperature in the entire data center ASHRAE Recommended Range Ranges during demonstration

Best practices – free cooling with air economizers

Encouraging outside air economizers Issue: Many are reluctant to use air economizers Outdoor pollutants and humidity control considered equipment risk Goal: Encourage use of outside air economizers where climate is appropriate Strategy: Address concerns: contamination/humidity control Quantify energy savings benefits

Project objectives Identify potential failure mechanisms Measure contamination levels inside and outside of data centers Observe humidity control Evaluate economizer effect on cumulative particulate exposure Compare particle concentrations to guidelines

Measurements inside the centers IBM Standard EPA 24-Hour Health Standard EPA Annual Health Standard and ASHRAE Standard

Outdoor measurements IBM Standard EPA 24-Hour Health Standard EPA Annual Health Standard and ASHRAE Standard

Indoor measurements (note scale)

Data center w/economizer

Humidity measurements ASHRAE Allowable Upper Limit ASHRAE Recommended Upper Limit ASHRAE Recommended Lower Limit ASHRAE Allowable Lower Limit

Findings Water soluble salts in combination with high humidity can cause current leakage Static electricity (caused by humans) can occur with very low humidity Particle concentration typically is an order of magnitude lower than new ASHRAE limits (without economizer) Filtration and humidity control on make-up air can provide environments similar to those in closed data centers

Best practices – Power conversion

Data Center power conversions AC voltage conversions Internal Drive External Drive I/O Memory Controller m Processor SDRAM Graphics Controller DC/DC AC/DC AC/DC Multi output PS Voltage Regulator Modules 5V 12V 3.3V 1.5/2.5V 1.1V-1.85V PWM/PFC Switcher Unregulated DC To Multi Output Regulated DC Voltages Inverter In Out Bypass Battery/Charger Rectifier

Power Supplies in IT equipment Research illustrated large losses in power conversion Power Supplies in IT equipment Uninterruptible Power Supplies (UPS)

DC powering data centers Goal: Show that a DC system could be assembled with commercially available components and measure actual energy savings – a proof of concept demonstration.

Included in the demonstration Side-by-side comparison of traditional AC system with new DC system Facility level distribution Rack level distribution Power measurements at conversion points Servers modified to accept 380 V. DC Artificial loads to more fully simulate data center

Typical AC distribution today 380 V DC after first stage conversion 480 Volt AC

Facility-level DC distribution 480 Volt AC 380V.DC 380 V DC is delivered directly into the server to the same point as in an AC powered server. This eliminates the DC-AC conversion at the UPS and the AC-DC conversion in the server. Also, less equipment is needed.

Rack-level DC distribution 480 Volt AC

AC system loss compared to DC 7-7.3% measured improvement 2-5% measured improvement Rotary UPS

Energy savings for a typical data center 20% or more facility level energy savings because: Redundant UPS and server power supplies operate at reduced efficiency Cooling loads would be reduced. The demonstration comparisons were against “best in class” systems which performed better than typical systems we benchmarked. Further optimization of conversion devices/voltages is possible

Demonstration set-up – see website for more detail

DC power – next steps DC power pilot installation(s) Standardize distribution voltage Standardize DC connector and power strips Server manufacturers develop power supply specification Power supply manufacturers develop prototype UL and communications certification

Design and Training Resources A web-based training resource available on LBL’s website Design guidelines available through PG&E’s Energy Design Resources website

website: http://hightech.lbl.gov/datacenters/

Discussion/Questions??