1. Guangdeng Liao, Xia Zhu, Steen Larsen, Laxmi Bhuyan, Ram Huggahalli University of California, Riverside Intel Labs

2.  Motivation  Network Processing  Experiment Setup  Power Studies ◦ Intel Nehalem Server ◦ Niagara 2 Server  System Architecture Implications

3.  Network speed increases at a rate of 10X and rapidly transits to 10Gbps and beyond.  Existing studies on network processing focused on performance.  Power becomes increasingly important due to environmental and economic concerns.  Detailed power studies of network processing over high speed networks guide us to design a more power-efficient platform.

4.  Unlike traditional CPU/Memory-intensive apps, network processing involves various platform components.

5.  Focused on mainstream servers: Intel Nehalem servers interconnected with 10GbE.  Used Iperf to generate network traffic.  Used Data Acquisition System (DAQ) to measure power consumption on individual hardware components.  Studied power benefits of integrating 10GbE NICs into CPUs by using a Niagara 2 server.

6.  Sensing resistors are added to +12V, +5V, +3.3V power supply, as well as each DIMM and the 10GbE PCI-E NIC card. AC/DC power supply 120V AC CPU power and regulation Memory DIMM power PCIe NIC power Current + Voltage sensing data acquisition

7.  NIC Idling Power: ~9 Watts.  ~25 Watts and ~17 Watts are dissipated for small and large I/O sizes, respectively. CPU is the major power consumer, followed by memory.

8.  Small packets have very low power-efficiency due to inefficiency of processing small packets on current systems. CPU utilization breakdown shows the overhead is mainly from OS kernel.

9.  ~22 Watts and ~18 Watts are dissipated by small and large I/O sizes. CPU is the major consumer, followed by memory.

10.  Small packets have very low power-efficiency. Unlike the receive side, CPU utilization breakdown shows the overhead is mainly from SoftIRQ.

11.  Sun Niagara 2 integrates two 10GbE NICs into CPU die.  Conducted experiments on an integrated NIC (INIC) and a discrete NIC (DNIC), and compared power efficiency of network processing.

12.  DNIC has ~20 watts idling power with dual port, INIC has ~17 watts with dual port. Saving is from PCI-E interface.  INIC has better power efficiency than DNIC, mainly due to less CPU cycles consumed for processing received packets.

13.  INIC has similar power efficiency to DNIC.

14.  High speed NICs have high idling power and are not energy proportional at all.  Small packets have low power-efficiency. Optimizations lie in OS kernel such as buffer management, context switches etc.  CPU is the major contributor of power consumption of network processing, followed by memory.  Integrating NICs has a little bit better power efficiency.

15.  Motivation  Network Processing  Experiment Setup  Power Studies ◦ Intel Nehalem Server ◦ Niagara 2 Server  System Architecture Implications

16.  Reducing CPU power cost ◦ Small in-order cores have better power efficiency for network processing. ◦ Heterogeneous CPUs incorporating small cores for network processing. core Core Cache/Interconnect core…. atom NHM

17.  Reducing memory power cost ◦ In the receive side, there are two memory access for each packet. ◦ Solution: packets are delivered to caches and a new instruction is added to invalidate packets in caches.

18.  Reducing memory power cost ◦ In the transmit side, there are packet write-backs. ◦ Solution: transmitted packets are fed from caches and invalidated after data transfer.

19.  Reducing NIC power cost (Idling power) ◦ Integrate NICs into CPU to avoid PCI-E power consumption. ◦ Apply rate-adaption scheme into NICs to save power with low traffic rates.

20. Q & A