1. 1 of 20 Low Power and Dynamic Optimization Techniques for Power-Constrained Domains Ann Gordon-Ross Department of Electrical and Computer Engineering University of Florida, Gainesville

2. 2 of 23 2 Power-Constrained Domains Embedded Systems Domain –Increased battery life –Decreased cooling requirements

3. 3 of 23 3 Dynamic Optimizations ?

4. 4 of 23 4 Dynamic Optimizations Benefits No user effort –User doesn’t know what dynamic optimizations are No application-designer effort –Reduced time to market –Reduced NRE costs System reacts to changing environment –Changes in input stimulus –Changes in software/firmware Reduced power consumption –Longer battery life –Reduced cooling requirements

5. 5 of 23 5 Cache Power Consumption Memory access: 50% of embedded processor’s system power –Caches are power hungry –ARM920T (Segars 01) –M*CORE (Lee/Moyer/Arends 99) Thus, caches are a good candidate for optimizations Main Mem L1 Cache Processor L2 Cache 53%

6. 6 of 23 6 Reducing Cache Energy Consumption Different applications have vastly different cache requirements –Total size, line size, and associativity Cache parameters that don’t match an application’s behavior can waste over 60% of energy (Gordon-Ross 05) 4KB 16 byte 2-way 2KB 32 byte direct-mapped 8KB 64 byte 4-way

7. 7 of 23 7 Dynamic Cache Tuning Cache tuning is the process of determining the appropriate cache parameters for an application –Requires a tunable cache Cache parameter values can be varied during runtime –Requires tuning hardware Orchestrates cache tuning Energy Executing in base configuration Tunable cache Tuning hw TC Cache Tuning TC Download application Microprocessor Cache energy savings of 62% on average!

8. 8 of 23 8 Dynamic Cache Tuning Reconfigure the cache dynamically to adapt to different phases of program execution or different applications in a multi- application environment Base cache energy Time Energy Consumption Phase-tuned Change cache

9. 9 of 23 9 Dynamic Cache Tuning Challenges Base cache energy Time Energy Consumption Phase Interval Base cache energy Time Energy Consumption Runtime energy Tuning interval Excess tuning energy Tuning interval too short Tuning interval too long Base cache energy Time Energy Consumption Runtime energy Tuning interval Wasted energy in suboptimal configuration Need a good tuning interval –Tuning interval is the time between invocations of the tuning hardware –Should closely match phase interval - length of time the system executes between phase changes Problem: How does the tuning hardware determine when to invoke cache tuning - must have knowledge of the future to obtain optimal results

10. 10 of 23 10 Periodic System - Fixed Phase Interval Phase interval fixed at 10 million cycles Tuning interval too short Tuning interval too long Energy savings = 32% (includes 7% overhead due to tuning) Base Line Negative savings if tuning interval is greater than phase interval!

11. 11 of 23 11 Dynamic Cache Tuner Energy Savings Base line Observed similar results for less periodic systems, but still much work to be done. 29% energy savings - within 8% of optimal Normalized Energy

12. 12 of 23 12 Future Directions Dynamic optimizations in a multi-core environment –Cache hierarchy – some levels may be shared –Dynamic load distribution –Dynamic per-core shutdown or voltage reduction for reduced power consumption –Etc – Many single-core optimizations can be non- trivially applied to a multi-core environment –Dynamic tuning enables energy savings with no extra designer effort – suitable for standard binary situations, changing environment situations, etc.

13. 13 of 23 13 Power-Constrained Domains INTERNET

14. 14 of 23 14 Internet Power Consumption Connected edge devices account for 2% of the total power consumed in the US [EPA-06] –130 TWh/Year This is $1.3 billion @ $.10 per kWh 1 single-unit nuclear power plant outputs 8 TWh/Year Translates to 16 single-unit nuclear power plants! Why so much power? –PCs can consume up to 200 W –1 billion PCs worldwide by 2010 [Kanellos-04] What can we do? –PCs are idle 75% of the time [Purushothaman-06] –But only 10% of PCs are allowed to sleep during that time [EPA-06] –Sleeping reduces power consumption by 80% or more –If PCs were allowed to sleep, only 3 single-unit nuclear power plants would be required Question: Why aren’t these PCs asleep?!?!

15. 15 of 23 15 Maintaining Network Connectivity INTERNET IDLE GNUTELLA FILE SHARING APPLICATION FILE QUERY PACKET FILE RESPONSE PACKET Bob Alice Alice checks to see if Bob has a file needed for p2p file sharing Z Z z z FILE QUERY PACKET Problem: PC must be awake to maintain network connectivity

16. 16 of 23 16 A Solution – Power Proxying Primary challenge is to maintain network connectivity while the PC is power down to standby mode - sleeping Some packets do not require a complex response –Automated responses are sufficient –Network Interface Card (NIC) can act as proxy for the PC –Allow the PC to sleep while NIC services packets with automated responses –A technique known as power proxying –We call such a NIC a “Smart”-NIC - SNIC

17. 17 of 23 17 Power Proxying INTERNET IDLE GNUTELLA FILE SHARING APPLICATION Alice Bob Z Z z z PC delegates power to the SNIC to handle to network traffic FILE QUERY PACKET FILE RESPONSE PACKET

18. 18 of 23 18 Power Proxying INTERNET IDLE Proxiable Packet Response Z Z z z Chatter Packet Non-Proxiable/Wake up Packet SNIC Response Bob

19. 19 of 23 19 What to Proxy? - Proxiable Protocols Proxiable protocols - Network protocols amenable to proxying –Responses may be automated –Keep alive packets, IP conflict avoidance, etc. Z Z z z IDLE FOUR Categories of Proxiable Packets ARP QUERY ARP RESPONSE PING PING RESPONSE P2P FILE QUERY P2P RESPONSE Mail Notification ARP (Address Resolution Protocol) ICMP (Internet Control Message Protocol) TCP (Transmission Control Protocol) UDP (User Datagram Protocol) What application support is needed to increase sleep time?

20. 20 of 23 20 Network Slowdown Link rates are increasing to meet network traffic demands –10 Gbps soon to be common place Power of these links increase exponentially However, research shows that links are largely underutilized –1-5% for 1 Gbps –Need high speed for traffic bursts During times of light utilization, don’t need full link speed –i.e. Adaptive Link Rate –Change link speed to meet traffic demands –i.e. Switch between 100 Mbps and 1 Gbps –IEEE 802.3az task force established in Nov 2006

21. 21 of 23 21 Network Slowdown Which components within the NIC (or edge device i.e. first level switches and routers) can exploit times of low utilization –Reconfigurable switch fabrics –Clock down processor –Different processor sleep levels –Disable links –Reconfigurable buffers –Etc

22. 22 of 23 22 Network Slowdown Challenges –Network devices do not contain hardware primitives for low power operation –Designed to operate for peak traffic load Devices are too power hungry during average traffic load –Propose to design network devices optimized for average traffic load, but can handle peak traffic load –What hardware primitives are necessary? Low hardware sleep levels take longer to wake up than link speeds Bursts are unpredictable How can we architect support for these bursts to allow network devices time to wake up?

23. 23 of 23 23 Conclusions Dynamic optimizations of embedded systems –Self-tuning system for reduced energy and/or power consumption –Determining when to reconfigure is challenging –Move dynamic optimization research into multi-core Internet power consumption –Power proxying Allows host to sleep and maintain network connectivity –Network slowdown Designing network devices optimized for average traffic load