Pursuing Operational Excellence Better Data Center Management Through the Use of Metrics

Objective Use financial modeling of your data center costs to optimize their utilization.

Data Centers are Expensive to Build A 1MW 7,500 ft2 “white space” commercial data center will cost about $16.7M to build. The building to house that data center will be about 23K ft2. Total cost > $700/ft2. Compare that to $100 - $150 / ft2 for a custom home in middle America.

Data Centers Are Expensive to Operate Monthly electric bills for a 1MW data center average about $82.6k ($991K/yr.) (At 85% occupancy, with a 1.5 PUE) Network charges are very high as well. The data center infrastructure is one of the fastest growing cost of deploying IT. At 85% occupancy and a PUE of 1.5 the power draw would be 1000 X 0.85 X 1.5 = 1,275kW 1,275 X 24hr/d X 30d/mo = 918,000 kWhr At $0.09/kW; 918,000 X 0.09 = $82.6K/mo

Data Center Cost Accounting Do you know how much your data center is costing you?

Data Center OpEx The best financial modeling strategy for data center OpEx is to boil down all operational costs into a monthly recurring cost (MRC) expressed in $/kW for every data center or colocation facility you operate in. This is your “cost of compute”.

Components of Data Center OpEx Depreciation Maintenance of infrastructure components (Including preventative maintenance) Utilities (power, water, …) Fuel (e.g. diesel for generator tests) Security Salaries (for data center infrastructure personnel, dedicated security personnel)

Know your “cost of compute” Question you are answering: How much is it costing me to deploy a rack full of IT gear at a given location? Boil everything down to MRC in $/kW

Cost of Compute Know this number for every data center you own and for every colocation vendor you use. Easier to compute for dedicated data centers. More difficult but possible to compute for shared use buildings.

OpEx Financial Accounting Issues MRC in $/kW is not constant over time. There are seasonal variations for cooling loads. Design targeted PUEs are attained at full loads For a given location, MRC in $/kW will decrease as IT HW occupancy increases.

Typical Cost of Compute For a large Enterprise (economies of scale apply) in a concurrently maintainable data center $235 - $300 / kW MRC For small / medium Enterprise (no economies of scale) in a non-concurrently maintainable DC $300 - $400 /kW MRC; much higher for a concurrently maintainable DC (which a small enterprise is unlikely to have)

Ways to Manage Cost of Compute in Your Data Center Capacity Planning Energy Management Cooling Management Heat Management

Capacity Planning One of best ways to control Costs of Compute is to control capacity. Benefits Improved quality of forecast over time Reduced CapEx if you can delay build out of new data center Better decision support

Financial Control Over Space & Power Measure and publish utilization over time. Know your metrics: (avg. kW/rack, avg. ft2/rack) Set-up a space & power governance program Must pre-authorize HW intake to data center Must follow-through with decommissioning equipment.

Financial Control Over Energy Management You are probably spending too much! You probably don’t know your PUE (Power Utilization Efficiency) You are probably running too cold!

Energy Management Strategies Measure, track, and publish your PUE Use ASHRAE’s new recommended operating temperatures. See Green Grid paper on the ROI of cooling system improvements

Why PUE is Important 1 rack = 5kw 5kw rack x 8,760 hrs = 43,800kwh/ year 43,800 kwh x $0.07/ = $3,066 per rack annual 5,000 sqft data center with 140 racks 140 racks x $3,066 = $429,240 annual IT energy costs At PUE of 2 = $858,000 Total DC energy At PUE of 1.5 = $643,500 Total DC energy Difference: $214,500

Energy Management ROI Generally speaking, for every 1.8°F that you raise the temperature in your data center, you save 2-4% of your total energy bill.

Seemingly Small Changes Can be Big 91 watts vs 70 watts processors = 21 watts / processor 21 watts x 2 processors = 42 watts savings 42 watts x 8760hrs / 1000 = 368 kwh 368kwh x $0.07/kwh = $25.75 per server 140 racks / 5000sqft DC 13 servers / rack 140 racks x 13 servers / rack = 1680 servers 1680 x $25.75 x 2 $86,520

Cooling Management Understanding how much cooling your data center needs. Convert Formula Example Tons of air to BTU 12,000 BTUs = 1 ton of air   To find BTUs Watts x 3.41 = BTU 2500w x 3.41 = 8525 (BTU) 1 kW = .2843 tons 1 ton = 3.517kW Tons to kW Tons x 3.52 = kW 20 tons x 3.52 = 70kW kW to tons kW x .28 = tons 70kW x .28 = 19.6 tons

Cooling – effectiveness How does your cooling capacity stack up against my IT load? Load on the UPS in kW x .28 = tons of cooling. Compare this against tons of cooling in your data center.

Heat Management DC looking to add another CRAC unit. Consider heat containment first. Lets look at some simple napkin math, pursuant to a containment solution. CRAC Containment CapEx $80k $45k OpEx (5 yrs.) $50k $ 0k Total $130k $45k

Other Green Grid Recommendations Install OEM variable speed drives (VSDs) in all computer room air handlers (CRAHs). Upgrade older CRAH units with newer more efficient models. Improve rack airflow management by adding baffles and blanking panels, which improve isolation of hot and cold air aisles. Reposition the CRAH temperature/humidity sensors from the inlets of the CRAHs to the front of the IT equipment racks. Adjust the temperature set-points of the CRAH sensors and the chiller units

What About the Cost of Outsourcing Is you organization benefiting from selectively outsourcing some of your infrastructure?

There May be Financial Advantages to Outsourcing Consider colocation You can probably not build and operate anything smaller than a 2MW data center for less than what you would pay for colocation. (For the same level of redundancy & service levels) Consider other outsourcing options

The Outsourcing Spectrum Application Middleware Operating System IT Physical Infrastructure Virtual Infrastructure Physical Hardware Data Center Environment BYODC Colocation Managed Hosting SaaS PaaS IaaS

Cost of Colocation Location $/kW MRC Boston $300 - 400 Chicago $225 - 425 Dallas $200 - 500 NY/NJ $275 - 510 No VA $250 - 400 No CA $275 - 450 So CA $300 - 650

Funding IT in Your Data Center IT Funding Models The IT funding model is IT's primary mechanism for gaining approval for spending on behalf of the business, accounting for that spending, and communicating costs back to business stakeholders. IT funding models vary substantially across organizations, but they all seek to strike three critical balances: Balancing cost with service levels Balancing maintenance with new investments Balancing effective IT coordination with responsiveness to diverse business partner needs

Why Provide Financial Transparency Transparency improves IT accountability and focuses the mission on what is important. Transparency supports maintaining proper balance. Transparency helps the business determine better buy versus build decisions, including brokering external services.

Deliver Cost Transparency via Showbacks or Chargebacks ?

How to Get Started Recommendations you can implement now to improve or initiate your showback or chargeback program. Create a service catalog. Identify and align business services to technology capabilities currently being delivered to business units. Gather service costing information. Start with gathering data and information from the finance department about the cost and expense of each    technology-based product and service. Relate services and costs. Create a financial cost model of how technology services and costs are related and allocated to business units. Make it plain. Implement greater IT service costing transparency that plainly details the relationships to budgets and/or P&L statements.

David Sipes, Global Data Center Manager, SOCAL AFCOM Vice President of Operations David.Sipes@Verizon.net