Energy Efficient Ethernet An Overview Mike Bennett Lawrence Berkeley National Lab

Agenda What is Energy Efficient Ethernet (EEE)? How we got this far Progress What’s next? Ethernet Alliance as an incubator for EEE

What is EEE? An idea –Reduce energy consumption during periods of low link utilization Low hanging fruit Minimizes impact on industry Minimizes impact on standard An IEEE Study Group –Formed in November 2006 to study the idea Technical and economic feasibility Compatibility and distinct identity Broad market potential

Things to be studied Rapid PHY Selection Switch to a lower speed (power) PHY during low link utilization –How to minimize transition time? –How to avoid thrashing between speeds? –Interaction with higher-layer protocols –Link utilization on servers –Interaction with control policy But, how did we get to this point?

The road to EEE Tutorial given in July 2005 to IEEE 802 –Bruce Nordman and Ken Christensen discuss the problem and possible ways to solve it They propose the idea of Adaptive Link Rate (ALR) –This includes a control policy This raises awareness within IEEE 802 –Gauge interest between July ’05 and July ’06 ALR whitepaper posted on EA site August ’06 – CFI team forms in August ’06

The CFI – identify the problem “ Big IT” – all electronics –PCs/etc., consumer electronics, telephony Residential, commercial, industrial –200 TWh/year –$16 billion/year –Nearly 150 million tons of CO 2 per year Roughly equivalent to 30 million cars! One central baseload power plant (about 7 TWh/yr) PCs etc. are digitally networked now — Consumer Electronics (CE) will be soon Numbers represent U.S. only

The CFI – more about the problem “ Little IT” — office equipment, network equipment, servers 97 TWh/year 3% of national electricity 9% of commercial building electricity Almost $8 billion/year TWh/year Network Equipment13 Servers12 PCs / Workstations20 Imaging (Printers, etc.)15 Monitors / Displays22 UPS / Other16 … However Old data (energy use has risen) Doesn’t include residential IT or networked CE products Note: Year 2000 data taken from Energy Consumption by Office and Telecommunications Equipment in Commercial Buildings--Volume I: Energy Consumption Baseline Roth et al., 2002 Available at:

The CFI – even more about the problem Source: Luiz André Barroso, (Google) “The Price of Performance,” ACM Queue, Vol. 2, No. 7, pp , September (Modified with permission.) Unrestrained IT power consumption could eclipse hardware costs and put great pressure on affordability, data center infrastructure, and the environment.

The CFI - motivation to start EEE Electric Utility Rebates Appliances … HVAC systems … Lighting … … PC power supplies (2005) … Server computers (2006) … PG&E (California) provides rebates for more energy-efficient servers Reference:

EEE – motivation April 19, 2006 “Green Grid” formed “A group of technology industry leaders form The Green Grid to help reduce growing power and cooling demands in enterprise datacenters Energy Efficiency gets U.S. congressional recognition –July 12, 2006 House Resolution 5646 passes “To study and promote the use of energy efficient computer servers in the United States” Green500 Supercomputer list created Nov. ‘06 –To “take energy efficiency into account when ranking supercomputers” Using Flops/watt (total system power) as the metric

EEE – motivation Energy Star –Requirements coming in 2009 “All computers shall reduce their network link speeds during times of low data traffic levels in accordance with any industry standards that provide for quick transitions among link rates” The market for Energy Efficient Ethernet –Driven by customer’s desire to save energy costs –Ethernet is used in markets where saving energy is crucial –Accelerate deployment for new applications –Enables use of incentives by energy industry Ultimately these translate to increased demand

The CFI – possible energy savings Typical switch with 24 ports 10/100/1000 Mb/s Various computer NICs averaged Results from (rough) measurements — all incremental AC power — measuring 1 st order

The CFI – possible solution Desktop-to-switch links –Are mostly idle Lots of very low bandwidth “chatter” –High bandwidth needed for bursts Bursts are often seconds to hours apart Server links are also often not fully utilized –Higher speed links offer more opportunity to save energy –This is an area where more data is needed Evidence of low utilization (desktop users) –LAN link utilization is generally in range 1 to 5% [1, 2] –Utilization for “busiest” user in USF was 4% of 100 Mb/s [1] A. Odlyzko, “Data Networks are Lightly Utilized, and Will Stay That Way”, Review of Network Economics, Vol. 2, No. 3, pp , September [2]R. Pang, M. Allman, M. Bennett, J. Lee, V. Paxson, and B. Tierney, “A First Look at Modern Enterprise Traffic,” Proceedings of IMC 2005, October 2005

The CFI – possible solution Snapshot of a typical 100 Mb FE link –Shows time versus utilization (trace from Portland State Univ.) Typical bursty usage (utilization = 1.0 %)

The CFI – possible solution Control policy determines when to transition data rate –RPS can support many different control policies –Need to consider but not define Trade-off of energy saved versus packet delay –Energy savings achieved by operating at low data rate –Delay occurs during transition from low to high data rate switch desktop PC link Control policy runs on both ends of the link Must prevent link from thrashing Control policy can be based on queue length thresholds

The CFI – possible solution Device A Request Acknowledge Device B Time establish new data rate inhibit frame transmission change link speed inhibit frame transmission allow frame transmission change link speed allow frame transmission ~1 ms Sequence

The CFI – possible solution Snapshot of a typical Ethernet link with simulated RPS Mean packet delay with RPS is 0.67 ms Mean packet delay without RPS is 0.12 ms Utilization-threshold policy is used Switching time: 1ms Data rates: 100 and 10 Mb/s Low/High thresh: 0KB and 32KB

The CFI – Benefits of EEE 1 Gb/s –Most NICs and most energy to be saved –Substantial benefits for homes and offices –Battery life benefit for notebooks 10 Gb/s (copper) –Reduces power burden in data centers –Reduces cooling burden in data centers –May increase switch/router port capacity Generally… –Provides real economic benefit through energy savings

The CFI – potential savings identified Assume 100% adoption (U.S. Only) Residential –PCs, network equipment, other –1.73 to 2.60 TWh/year –$139 to $208 million/year Commercial (Office) –PCs, switches, printers, etc. –1.47 to 2.21 TWh/year –$118 to $177 million/year Data Centers –Servers, storage, switches, routers, etc. –0.53 to 1.05 TWh/year –$42 to $84 million/year These figures do not include savings from cooling/power infrastructure Total: $298 to $469 million/year

EEE - progress November Plenary Meeting (CFI) –33 supporters 15 from EA member companies –73 attendees Passed 65 – 2 – 6 –Many EA members in the room January Interim Meeting –Met for almost 2 days people attended per day –most were EA members Heard several presentations –Update on control policies and impact to higher layer protocols –Open questions to the study group regarding scope

EEE - progress January Interim Meeting –Presentations (continued) “Strawman” proposal for 5 criteria and objectives Update on candidate protocols Impact on clause 28 (auto-negotiation) –Defined some terms –Took several straw polls –People took action items What’s next?

EEE - what’s next? Preparing for March –Several people have offered to present More simulations on control policy Technical feasibility of Rapid PHY selection Alternatives to Rapid PHY selection Work related to Clause 28 (auto-negotiation) A look at RPS for backplanes

EEE - what’s next? Preparing for March –What we still need Traces from networks in different markets –Server network utilization –Home network utilization –Consumer electronics network utilization How to communicate with upper layers? –Don’t want a network melt-down because of RPS Economic feasibility of building EEE PHYs Further study on broad market potential Ultimately preparing for PAR Where does the Ethernet Alliance fit in this?

EEE/EA – Technology Incubation How did EEE get this far? –ALR Whitepaper posted on EA site –CFI Team and supporters Mostly EA members –First study group meeting contributors and participants Mostly EA members EA Mission –The mission is to promote industry awareness, acceptance, and advancement of technology and products based on both existing and emerging IEEE 802 Ethernet standards and their management.

EEE/EA – Technology Incubation Marketing Activities –Proactive cohesive messaging to external target audiences Press –Most recently interviewed with CMP/Information Week to promote EEE »Arranged by the EA Planning to form an EEE technical subcommittee –When the time is right Bottom line –The EA has been there to help us be successful Membership has its benefits

EEE/EA – summary EEE is a classic case of technology incubation by the EA –EA support helped to keep EEE moving in the right direction CFI preparation –EA marketing support is raising industry awareness Press The Ethernet Alliance will be there for us –When we need a place for consensus building among members –When the time comes for testing and demonstrating interoperability

Thanks! Questions or comments?