Reliability and Energy Efficiency – Not Mutually Exclusive William Tschudi Principal Investigator Lawrence Berkeley National Laboratory

Acknowledgements California Energy Commission Pacific Gas and Electric Company Uptime Institute Critical Facilities Roundtable Rumsey Engineers RMI E Source Power Supply Manufacturers Association Industry Partners (Too many to name all) Subcontractors: EYP Mission Critical Facilities, Ecos Consulting, EPRI-PEAC

Energy Intensive High-tech Buildings

We Also Operate Data Centers

Why Research Data Centers? Overview Why Research Data Centers? Utilities were receiving requests for unrealistic power densities A lot of misinformation was (is?) circulating Large continually operating base loads We saw large energy efficiency opportunities in other High-Tech buildings Technology improvements are transferable to other building types

Prior Data Center Research Overview Prior Data Center Research Research “roadmap” for the CA Energy Commission Participation in design charrette organized by Rocky Mountain Institute Energy benchmarking and case studies for 14 data centers Market assessment in CA

Current Data Center Research Activities Benchmarking and Best Practices Investigate UPS Efficiency Improvement Investigate Power Supply Improvement Develop performance metrics – computing horsepower and energy use Demonstration Projects Technology Transfer

Benchmarking Energy Benchmarking Benchmark (measure) energy use in 6-10 additional data centers Energy intensity and end-use System and component efficiency Efficiency improvement suggestions Solicit additional benchmarks from other sources Identify and analyze better performing systems and document in “Best Practices” summary Develop self-benchmarking protocol

Electricity Flows In Data Centers HVAC system local distribution lines lights, office space, etc. uninterruptible load UPS to the building, 480 V computer equipment PDU computer racks backup generators

IT Equipment Loading Benchmarking Both LBNL and Uptime Institute found average IT equipment loading at ~25 W/ft2

Projecting IT Load When Fully Loaded Benchmarking Projecting IT Load When Fully Loaded (W/Sq.Ft. of electrically active floor space)

Data Center Benchmarking

Benchmarking Chiller Comparison Average 0.75

Total Chilled Water System Efficiency Benchmarking Total Chilled Water System Efficiency Average 1.69

UPS System Benchmarking

Standby Generation Loss Benchmarking Standby Generation Loss Several load sources Heaters Battery chargers Transfer switches Fuel management systems Heaters alone (many operating hours) use more electricity than ever produced by the generator (few operating hours) Opportunity may be to reduce or eliminate heating, batteries, and chargers

Standby Generator Heater

No Impact on Reliability Benchmarking No Impact on Reliability Chiller and pumping efficiency Use of lighting controls Variable speed drives – pumps, chillers, fans Free cooling Improved UPS efficiency

Improved Reliability and Energy Performance Benchmarking Improved Reliability and Energy Performance Better match systems to their loads Improve humidity control and eliminate CRAC unit fighting Water side economizers Better control strategies – setpoints, cooling tower staging Better thermal stratification – high ceilings and properly sized underfloor Air management – delivering air where it’s needed

General Recommendations Benchmarking General Recommendations Benchmark to know where you stand Life-cycle cost analysis Facilities partnership with IT professionals Evaluate load spreading vs. compaction

Available Benchmark Data? Benchmarking Available Benchmark Data? Sources of other benchmark data? Please contact LBNL: Steve Greenberg: segreenberg@lbl.gov Bill Tschudi: wftschudi@lbl.gov

Benchmarking Future Direction Develop consensus on performance benchmarks, collect data, quantify energy savings potential Incorporate other industry benchmark data

Data Center Power Delivery UPS 88 - 92% Power Dist 98 - 99% Power Supply 68 - 72% dc/dc 78 - 85% HVAC 1,200W / 1 Ton (76%) US Annual Energy Consumption of 30TW-h flows through this inefficient delivery path

Total efficiency ≈ 40% Cost of power delivery = $8,200 / 100 Power Path Efficiency Power (kW) MW-h Cost Load Cooling 3.1 113 $8,500 Loads Systems (x100) (not including dc/dc & ac/dc) 9.8 x 85% Dc/dc 2.1 109 $8,200 x 70% Power supply 5.1 x 90% UPS 1.9 x 98% Distribution 0.4 x 76% Delivery Cooling 3.0 Total = 40% Total 222 $16,700 Source: EPRI PEAC Total efficiency ≈ 40% Cost of power delivery = $8,200 / 100

Improvements in Cost of Power Delivery Power Path Efficiency Power (kW) MW-h Cost ($k) Load cooling 3.1 113 8.5 Loads Systems (x100) (not including dc/dc & ac/dc) 9.8 x 85% Dc/dc 2.1 109  62 8.2  4.7 x 70%  85% Power supply 5.1  2.1 x 90%  94% UPS 1.9  0.9 x 98% Distribution 0.4  0.3 x 76% Delivery Cooling 3.0  1.7 Total = 40%  51% Total 222  175 16.7  13.2 Source: EPRI PEAC Annual cost reduced by $3,500 / 100

UPS Systems UPS Activities Determine the range of current UPS efficiencies, highlight more efficient designs, and provide a means for comparing their total cost of ownership (TCO). Propose a new UPS labeling scheme that could be considered by Energy Star and other third -party efficiency labeling organizations. Conduct a scoping study to analyze the energy efficiency savings potential and performance of a complete DC power architecture for data centers.

UPS Efficiency and Loading UPS Systems UPS Efficiency and Loading Manufacturers data for efficiency versus load for current generation static and inertial UPS. Based on review of more than 100 static UPS models

UPS Measured Performance UPS Systems UPS Measured Performance Sample of 12 field measurements.

Measuring UPS Efficiency - “High Efficiency” Option UPS Systems Measuring UPS Efficiency - “High Efficiency” Option Measured Result Manufacturer Spec Source: EPRI PEAC On average, existing high efficiency modes can make a 4 to 5 % difference in UPS efficiency.

Analyzing UPS Performance in “High Efficiency” Option UPS Systems Analyzing UPS Performance in “High Efficiency” Option Source: EPRI PEAC In “high efficiency” mode, there can be one cycle (16.6 msec for 60 Hz) of voltage deviation on UPS output. Power supplies downstream of UPS can ride through this.

Research of Inertial Units UPS Systems Research of Inertial Units Caterpillar Pentadyne Piller Quoted efficiencies are in the 96% range and higher; we will be measuring actual performance in field/lab tests

Possible UPS efficiency labeling criteria UPS Systems Labeling Efficiency and Reliability Coordinating with International labeling effort addressing quality & efficiency. Possible UPS efficiency labeling criteria

Labeling will consider non-energy, reliability issues UPS Systems Labeling will consider non-energy, reliability issues

Power Supply Activities Measure efficiencies of current server power supplies Field testing to document achievable energy savings Recommend new efficiency levels to Server System Infrastructure Initiative (SSI) for consideration Assess other PS saving opportunities in DC applications

Power Supplies Can Be Much More Efficient – Without Affecting Reliability Performance The nation’s 3.1 billion power supplies waste about 3 to 4% of the entire U.S. electricity bill Worst designs are 30 to 50% efficient Designs already exist in the market that can bring power supply efficiencies to the range of 75 to 93% with big non-energy benefits More efficient designs will be developed if there is incentive.

A consistent test protocol using a standard loading guideline & test report format will allow more visibility on power supply efficiency Sample of our testing report – similar to the desktop power supply study

Server Power Supply Efficiency Test Data: Current Market 95th Percentile efficiency Median % power supply loading Source: EPRI PEAC

Importance of a Flat Power Supply Efficiency Curve Since PS loading are low it is important to have a flat efficiency curve as much as possible More design focus on efficiency when lightly loaded is needed

Goals of the Power Supply Task Long term: Move the market towards widespread adoption of energy efficient PS in data centers Create an energy efficiency labeling program such as ENERGY-STAR® for server power supplies

Power Supplies in IT Equipment

Power Supplies in IT Equipment

Server Power Supply Efficiency Lab Test Setup Electronic Load Banks Fluke 41 Power Harmonic Analyzer Yokogawa Digital Power Meter Computer Interface Power Supply Load Test Fixtures Server Power Supply

Overall Computing Efficiency Performance Metrics Overall Computing Efficiency …does not relate to very low power consumption Very Low Processor Activity… Most of the time the GHz processor is doing activities that can be done by a MHz processor but the input power consumption is not changing much

Performance Metrics - limited activity includes: An assessment of: Server activity profile based on application/server type Correlation of server activity and power consumption Work with Intel, AMD, IBM, HP, and potentially others to reach consensus on metrics for some applications. Develop efficiency guidelines based on performance metrics

LBNL’s Role in Demonstrations Scoping demonstrations of technologies or strategies to improve energy efficiency in high-tech buildings Showcase new/emerging or under-utilized technologies or approaches

Currently Planned Demonstrations Heat removal from servers without fans DC powering rack of servers Air management improvements (PG&E)

Conclusion Many strategies to improve energy performance in a data center can be implemented with no impact on reliability – in fact, many will actually improve reliability and lower capital cost. Major industry firms are interested in supporting efficiency (and reliability) improvement. If you want more efficient data centers – ask for them.

Thank You Questions? Bill Tschudi, Principal Investigator Lawrence Berkeley National Lab 510.495.2417 wftschudi@lbl.gov http://hightech.lbl.gov