1. Accelerating business innovation; a Technology Strategy Board programme www.gridcomputingnow.org Energy efficiency and research computing RUGIT Away Day, 24th Jan 2008 Dave Berry Deputy Director, NeSC & Technology Lead, Grid Computing Now! daveb@nesc.ac.uk

2. Accelerating business innovation; a Technology Strategy Board programme www.gridcomputingnow.org Contents Background and goals Desktop Grids for High Throughput Computing Data centres and HPC centres Ways forward

3. Accelerating business innovation; a Technology Strategy Board programme www.gridcomputingnow.org GCN! Aims Accelerate the benefits to the UK economy of adopting modern computing technologies, i.e.: The creation of scalable, secure, efficient ICT infrastructures, For delivering IT services, linked to business processes through a service oriented architecture, While achieving greater utilisation with reduced energy consumption and reduced costs

4. Accelerating business innovation; a Technology Strategy Board programme www.gridcomputingnow.org Knowledge Transfer Network Part of the Technology Strategy Boards Innovation Programme Run by Intellect, NeSC and CNR Activities Web platform; user case studies; events; webinars; active sector and regional programme Communities of Practice Green IT (including MBE KTN, BCS, …) Transport Modelling, Grids in Health, Public sector IT, …

5. Accelerating business innovation; a Technology Strategy Board programme www.gridcomputingnow.org Aims for this session To expand our community of practice Stimulate discussion Share knowledge Form relationships Suggest a best practices document Input to policy (Unis, RCs, government) Want to learn as much as inform

6. Accelerating business innovation; a Technology Strategy Board programme www.gridcomputingnow.org Not a new topic… UCISA Annual Conference 2007 Panel session: Green IT/buildings Gary Bark, Converged Building Solutions Ltd Gill Ferrell, JISC infoNet Chris Cartledge, University of Sheffield Simon Tindall, Sun Microsystems Ltd

7. Accelerating business innovation; a Technology Strategy Board programme www.gridcomputingnow.org Desktop Grids High Throughput Computing Many experimental scientists are more interested in number of jobs/month rather than instantaneous computing power One approach: use Spare Cycles No need for air conditioning, etc. But dont we want to switch off machines at night? Claim: desktop grids can give computing power for less cost and less electricity

8. Central Manager 1600 Workstations 30 Workstations Submit Nodes Execute Nodes master, startd, starter master, schedd, shadow master, collector, negotiator Cardiff slides from James Osborne, High-Throughput Computing Week NeSC, 27-30 November 2008

9. Power Consumption Based on a P4 3GHz PC with 512MB RAM

10. Economic Viability Makes sound financial sense Hibernate saves £60 per year Condor = £30 per year (max) Dedicated = £150 per year Condor is 5 times cheaper Based on a P4 3GHz PC with 512MB RAM Saving of Hibernate = Cost of 100W Electricity (Idle State) for 16 Hours out of 24 Cost of Condor = Cost of 150W Electricity (Condor State) – Cost of 100W Electricity (Idle State) Cost of Dedicated = Cost of 150W Electricity (Condor State) + Cost of 100W Electricity (Air Con)

11. Environmental Impact Makes sound environmental sense Hibernate saves 650Kg CO 2 per year Condor = 325Kg CO 2 per year (max) Dedicated = 1,625Kg CO 2 per year Condor is 5 times greener Based on a P4 3GHz PC with 512MB RAM Saving of Hibernate = Cost of 100W Electricity (Idle State) for 16 Hours out of 24 Cost of Condor = Cost of 150W Electricity (Condor State) – Cost of 100W Electricity (Idle State) Cost of Dedicated = Cost of 150W Electricity (Condor State) + Cost of 100W Electricity (Air Con)

12. Across Campus Makes sound financial sense –Hibernate would save £600,000 per year Hibernate 16 out of 24 hours Makes sound environmental sense –Hibernate would save 6,500T CO 2 per year –Rainforest required = 52Km 2 –Rainforest required = 40% area of Cardiff Based on 10,000 P4 3GHz PCs with 512MB RAM Saving of Hibernate = Cost of 100W Electricity (Idle State) for 16 Hours out of 24 Cost of Condor = Cost of 150W Electricity (Condor State) – Cost of 100W Electricity (Idle State) Cost of Dedicated = Cost of 150W Electricity (Condor State) + Cost of 100W Electricity (Air Con)

13. Accelerating business innovation; a Technology Strategy Board programme www.gridcomputingnow.org The best of both worlds? What if we could power-off idle machines and wake them up when Condor has jobs to run? JISC Low Carbon ICT project http://projects.oucs.ox.ac.uk/lowcarbonict/ Develop an institution-wide wake-on-LAN service Monitor energy consumption across the University Write and implement a communications strategy Towards Low Carbon ICT conference Oxford, 19 th March 2008

14. Accelerating business innovation; a Technology Strategy Board programme www.gridcomputingnow.org Need: 2xProcessing Capacity per annum; Target: 60% Energy Reduction from 1990 levels by 2050

15. Accelerating business innovation; a Technology Strategy Board programme www.gridcomputingnow.org The scale of the problem Power consumption by data centres: Estimated 1.5% of UK national electricity generation Peak consumption of 8GW Estimated to rise from 46 TWH in 2006 to 93 TWH in 2020 Comparable with consumption by refrigeration… … or greenhouse gases emitted by aviation

16. Technology Power Loss Chain: Fossil Fuel – CPU Used Slide from BCS DCSG

17. Accelerating business innovation; a Technology Strategy Board programme www.gridcomputingnow.org Policies and measurement BCS DCSG model Open source model of energy-efficiency for data centres and servers The Green Grid Vendor consortium EU code of conduct on data centres Development of baseline measures Voluntary contribution of data from subscribing organisations

18. Accelerating business innovation; a Technology Strategy Board programme www.gridcomputingnow.org Press Examples Ultraspeed DC-based system in East London Claims 30% power saving from use of DC Extra 10% saving from diskless servers http://pcworld.about.com/od/recyclin1/Data-center-claims- power-cuts.htm Plan for green data farm in Lockerbie Using renewable energy sources Waste heat used to heat new eco village http://www.redwasp.co.uk/newsitem.asp?id=280

19. Accelerating business innovation; a Technology Strategy Board programme www.gridcomputingnow.org GCN! Webinar The Business Case and Methods for the Green Data Centre Recording available on the web Zahl Limbuwala Chair, BCS Data Centre Specialist Group Motivation and an introduction to the BCS model Kate Craig Wood Managing Director, Memset Ltd. Practical steps to running a carbon-neutral data centre

20. Accelerating business innovation; a Technology Strategy Board programme www.gridcomputingnow.org Finance and administration Who sees the power bills? Who calculates the total cost of ownership? Does this affect purchasing decisions? Trade-off: cost of reliability vs. cost of downtime Dont overspecify Staff requirements Specialised HPC kit may need specialised staff Other kit may not

21. Accelerating business innovation; a Technology Strategy Board programme www.gridcomputingnow.org Power engineering Location Near power stations Or local generation (CHP?) AC or DC? Provisioning levels Nameplate provisioning is inefficient Cables Oversize for lower resistance (can halve losses) Route through cool underfloor area

22. Example benchmarks Slide from Kate Craig-Wood

23. Graded UPS usage Latest switch-mode based generation vastly more efficient 96-96% vs. ~90% Most efficient when fully loaded (98%) Don't run at half-capacity 40KVA steps, rather than typical 500KVA steps Slide from Kate Craig-Wood

24. Accelerating business innovation; a Technology Strategy Board programme www.gridcomputingnow.org Cooling What target temperature? Water or air? Store heated water externally? External air or recycled air? Savings from fresh air cooling of 72% (Source: BCS) Modular cooling Cabinets or rooms? Modelling and controlling heat flow Waste heat How to sell or reuse?

25. Accelerating business innovation; a Technology Strategy Board programme www.gridcomputingnow.org Fresh Air Cooling

26. Accelerating business innovation; a Technology Strategy Board programme www.gridcomputingnow.org State of the art? By careful design, matching the specification of the plant to the needs of the machine(s) we have brought the summer time cooling overhead down from 60 - 65% (very typical of most computer rooms) to 25 - 30%. Further, by using freecool this drops to 7% in the winter.

27. Accelerating business innovation; a Technology Strategy Board programme www.gridcomputingnow.org System utilisation Server virtualisation Load balancing Note – this is typically done already for compute clusters May still apply to other university systems Choice of equipment Multi-core, power management, etc. Diskless servers

28. Accelerating business innovation; a Technology Strategy Board programme www.gridcomputingnow.org Ways forward Best practice document(s) Perhaps proven + experimental? Shared facilities Shared data centres? Shared compute clusters (a la NGS)? Training for IT staff? Case studies? Input to policy (Unis, RCs, government)

29. Benchmarks and Labelling Slide from BCS DCSG

30. Accelerating business innovation; a Technology Strategy Board programme www.gridcomputingnow.org More references GCN! Webinar http://tinyurl.com/2gtslj BCS Data Centre Specialist Group http://tinyurl.com/2dyy5t EU Code of Conduct http://tinyurl.com/2drxoh HTC week http://www.nesc.ac.uk/action/esi/contribution.cfm?Title=831 Technology Strategy Board http://www.berr.gov.uk/innovation/technologystrategyboard