1. Evaluation and Validation Peter Marwedel TU Dortmund, Informatik 12 Germany Graphics: © Alexandra Nolte, Gesine Marwedel, 2003 These slides use Microsoft clip arts. Microsoft copyright restrictions apply. 2012 年 06 月 25 日

2. - 2 -  p. marwedel, informatik 12, 2012 Structure of this course 2: Specification 3: ES-hardware 4: system software (RTOS, middleware, …) 8: Test 5: Evaluation & Validation (energy, cost, performance, …) 7: Optimization 6: Application mapping Application Knowledge Design repository Design Numbers denote sequence of chapters

3. - 3 -  p. marwedel, informatik 12, 2012 Real-time calculus (RTC)/ Modular performance analysis (MPA) 1 2 3  p 2p2p 3p3p 1 2 3  p 2p2p 3p3p p-J p+J periodic event streamperiodic event stream with jitter Thiele et al. (ETHZ): Extended network calculus: Arrival curves describe the maximum and minimum number of events arriving in some time interval . Examples: p p p-J P+J

4. - 4 -  p. marwedel, informatik 12, 2012 RTC/MPA: Service curves Service curves  u resp.  ℓ describe the maximum and minimum service capacity available in some time interval  s p bandwidth b TDMA bus p s p-sp+s b sb s 2p Example:

5. - 5 -  p. marwedel, informatik 12, 2012 RTC/MPA: Workload characterization  u resp.  ℓ describe the maximum and minimum service capacity required as a function of the number e of events. Example: 1 2 3 4 8 12 16 e WCET=4 BCET=3  u u  l l (Defined only for an integer number of events)

6. - 6 -  p. marwedel, informatik 12, 2012 RTC/MPA: Workload required for incoming stream Incoming workload Upper and lower bounds on the number of events

7. - 7 -  p. marwedel, informatik 12, 2012 RTC/MPA: System of real time components RTC RTC’ RTC” … Incoming event streams and available capacity are transformed by real-time components: Theoretical results allow the computation of properties of outgoing streams 

8. - 8 -  p. marwedel, informatik 12, 2012 RTC/MPA: Transformation of arrival and service curves Resulting arrival curves: Remaining service curves: Where:

9. - 9 -  p. marwedel, informatik 12, 2012 RTC/MPA: Remarks  Details of the proofs can be found in relevant references  Results also include bounds on buffer sizes and on maximum latency.  Theory has been extended into various directions, e.g. for computing remaining battery capacities

10. - 10 -  p. marwedel, informatik 12, 2012 Application: In-Car Navigation System Car radio with navigation system User interface needs to be responsive Traffic messages (TMC) must be processed in a timely way Several applications may execute concurrently © Thiele, ETHZ

11. - 11 -  p. marwedel, informatik 12, 2012 System Overview NAVRAD MMI DB Communication © Thiele, ETHZ

12. - 12 -  p. marwedel, informatik 12, 2012 NAVRAD MMI DB Communication Use case 1: Change Audio Volume < 200 ms < 50 ms © Thiele, ETHZ

13. - 13 -  p. marwedel, informatik 12, 2012 Use case 1: Change Audio Volume © Thiele, ETHZ Communication Resource Demand

14. - 14 -  p. marwedel, informatik 12, 2012 NAVRAD MMI DB Communication < 200 ms Use case 2: Lookup Destination Address © Thiele, ETHZ

15. - 15 -  p. marwedel, informatik 12, 2012 Use case 2: Lookup Destination Address © Thiele, ETHZ

16. - 16 -  p. marwedel, informatik 12, 2012 NAVRAD MMI DB Communication Use case 3: Receive TMC Messages < 1000 ms © Thiele, ETHZ

17. - 17 -  p. marwedel, informatik 12, 2012 Use case 3: Receive TMC Messages © Thiele, ETHZ

18. - 18 -  p. marwedel, informatik 12, 2012 Proposed Architecture Alternatives NAV RAD MMI 22 MIPS 11 MIPS 113 MIPS NAVRAD MMI 22 MIPS 11 MIPS113 MIPS RAD 260 MIPS NAV MMI 22 MIPS RAD 130 MIPS MMI NAV 113 MIPS MMI 260 MIPS RAD NAV 72 kbps 57 kbps 72 kbps (A) (E)(D)(C) (B) © Thiele, ETHZ

19. - 19 -  p. marwedel, informatik 12, 2012 Step 1: Environment (Event Steams) Event Stream Model e.g. Address Lookup (1 event / sec) uu ℓℓ [s] [events] 1 1 © Thiele, ETHZ

20. - 20 -  p. marwedel, informatik 12, 2012 Step 2: Architectural Elements Event Stream Model e.g. Address Lookup (1 event / sec) Resource Model e.g. unloaded RISC CPU (113 MIPS) l=ul=u uu ll [s] [MIPS] [events] 1 1 1 113 © Thiele, ETHZ

21. - 21 -  p. marwedel, informatik 12, 2012 Step 3: Mapping / Scheduling Rate Monotonic Scheduling (Pre-emptive fixed priority scheduling):  Priority 1:Change Volume (p=1/32 s)  Priority 2:Address Lookup (p=1 s)  Priority 3:Receive TMC (p=6 s) © Thiele, ETHZ

22. - 22 -  p. marwedel, informatik 12, 2012 Step 4: Performance Model CPU1BUSCPU3 CPU2 Change Volume Receive TMC     NAVRAD MMI Address Lookup   MMI NAVRAD © Thiele, ETHZ

23. - 23 -  p. marwedel, informatik 12, 2012 Step 5: Analysis CPU1BUSCPU3 CPU2 Change Volume Receive TMC     NAVRAD MMI Address Lookup   MMI NAVRAD © Thiele, ETHZ

24. - 24 -  p. marwedel, informatik 12, 2012 Analysis – Design Question 1 How do the proposed system architectures compare in respect to end-to-end delays? © Thiele, ETHZ

25. - 25 -  p. marwedel, informatik 12, 2012 End-to-end delays: (A)(E)(D)(C)(B) Analysis – Design Question 1 © Thiele, ETHZ

26. - 26 -  p. marwedel, informatik 12, 2012 Analysis – Design Question 2 How robust is architecture A? Where is the bottleneck of this architecture? NAVRAD MMI 22 MIPS 11 MIPS113 MIPS 72 kbps (A) © Thiele, ETHZ

27. - 27 -  p. marwedel, informatik 12, 2012 TMC delay vs. MMI processor speed: Analysis – Design Question 2 NAVRAD MMI 22 MIPS 11 MIPS113 MIPS 72 kbps (A) 26.4 MIPS © Thiele, ETHZ

28. - 28 -  p. marwedel, informatik 12, 2012 Conclusions  Easy to construct models (~ half day)  Evaluation speed is fast and linear to model complexity (~ 1s per evaluation)  Needs little information to construct early models (Fits early design cycle very well)  Even though involved mathematics is very complex, the method is easy to use (Language of engineers) © Thiele, ETHZ

29. - 29 -  p. marwedel, informatik 12, 2012 Evaluation of designs according to multiple objectives Different design objectives/criteria are relevant:  Average performance  Worst case performance  Energy/power consumption  Thermal behavior  Reliability  Electromagnetic compatibility  Numeric precision  Testability  Cost  Weight, robustness, usability, extendibility, security, safety, environmental friendliness

30. - 30 -  p. marwedel, informatik 12, 2012 Energy- and power models Power/energy models becoming increasingly important  due to mobile computing,  since energy availability becomes more relevant due to increased performance, and  due to environmental issues.

31. - 31 -  p. marwedel, informatik 12, 2012 Energy models  Tiwari (1994): Energy consumption within processors  Simunic (1999): Using values from data sheets. Allows modeling of all components, but not very precise.  Russell, Jacome (1998): Measurements for 2 fixed configurations  Steinke et al., UniDo (2001): mixed model using measurements and prediction  CACTI [Jouppi, 1996]: Predicted energy consumption of caches  Wattch [Brooks, 2000]: Power estimation at the architectural level, without circuit or layout

32. - 32 -  p. marwedel, informatik 12, 2012 Steinke’s & Knauer’s model E.g.: ATMEL board with ARM7TDMI and ext. SRAM Data Memory Instruction Memory IAddr V DD ALU Multi- plier Barrel Shifter Register File Instr. Decoder & Control Logic Instr Imm Reg Value Reg# Opcode ARM7 DAddr mA Data Instr E total = E cpu_instr + E cpu_data + E mem_instr + E mem_data

33. - 33 -  p. marwedel, informatik 12, 2012 Example: Instruction dependent costs in the CPU E cpu_instr =  MinCostCPU(Opcode i ) + FUCost(Instr i-1,Instr i )  1 *  w(Imm i,j ) + ß 1 *  h(Imm i-1,j, Imm i,j ) +  2 *  w(Reg i,k ) + ß 2 *  h(Reg i-1,k, Reg i,k ) +  3 *  w(RegVal i,k ) + ß 3 *  h(RegVal i-1,k, RegVal i,k ) +  4 *  w(IAddr i ) + ß 4 *  h(IAddr i-1, IAddr i ) Cost for a sequence of m instructions w : number of ones; h : Hamming distance; FUCost: cost of switching functional units , ß: determined through experiments

34. - 34 -  p. marwedel, informatik 12, 2012 Hamming Distance between adjacent addresses is playing major role

35. - 35 -  p. marwedel, informatik 12, 2012 Results  It is not important, which address bit is set to ‘1‘  The number of ‘1‘s in the address bus is irrelevant  The cost of flipping a bit on the address bus is independent of the bit position.  It is not important, which data bit is set to ‘1‘  The number of ‘1‘s on the data bus has a minor effect (3%)  The cost of flipping a bit on the data bus is independent of the bit position.

36. - 36 -  p. marwedel, informatik 12, 2012 CACTI model Comparison with SPICE Cache model used http://research.compaq.com/wrl/people/jouppi/CACTI.html

37. - 37 -  p. marwedel, informatik 12, 2012 Access times and energy consumption increase with the size of the memory Example (CACTI Model): "Currently, the size of some applications is doubling every 10 months" [STMicroelectronics, Medea+ Workshop, Stuttgart, Nov. 2003]

38. - 38 -  p. marwedel, informatik 12, 2012 Influence of the associativity Parameters different from previous slides [P. Marwedel et al., ASPDAC, 2004]

39. - 39 -  p. marwedel, informatik 12, 2012 Evaluation of designs according to multiple objectives Different design objectives/criteria are relevant:  Average performance  Worst case performance  Energy/power consumption  Thermal behavior  Reliability  Electromagnetic compatibility  Numeric precision  Testability  Cost  Weight, robustness, usability, extendibility, security, safety, environmental friendliness

40. - 40 -  p. marwedel, informatik 12, 2012 Energy models for communication: An Energy Model for Mobile WiMAX Devices  How does the application data rate influence the energy efficiency?  What is the impact of very small amounts of data on the efficiency?  Relationship between submitted power and consumed energy? WiMAX BS WiMAX Device Measurement Probe iPerf Client Evaluation PC iPerf Server Applicati on Testing Server Power Measurement Source: Wietfeld et al., 2012 Data Transfer

41. - 41 -  p. marwedel, informatik 12, 2012 Traffic Dependent Energy Consumption Idle State Data Transmission Channel Quality Reports  Average Power in Idle State: 880 mW  Channel Quality Reports every 300 ms lead to increased average power of 930 mW  Transmission is costing significantly increased energy consumption  Reception is not increasing the power consumption compared with idle state Mobile WiMAX Frame, P = 1.32 W Downlink Frame, P = 0.9 W Uplink Frame, P = 2.52 W Source: Wietfeld et al., 2012

42. - 42 -  p. marwedel, informatik 12, 2012 Modeling the Impact of T x -Power Variation 1.5 1.4 1.3 1.2 0.5 1.1 1.0 0.9 0.7 0.8 0.6 x10 -6 Energy Consumption per Bit [J/Bit] -10 -15-20-25-30-35-40 T x Power of the mobile Station [dBm] Ψ: 35/12 Ψ: 32/15 Ψ: 29/18 Ψ: 26/21 Different DL/UL Ratios of the TDD Frame αβγ C(Ψ=35/12)C(Ψ=32/15)C(Ψ=29/18)C(Ψ=26/21) P Tx,min 1.0325e-64.6e-55.225e-41.1084e-38.3530e-46.916e-45.994e-4-25 dBm  Energy per Bit is constant for low T x power (below -25 dBm)  For higher T x power, the consumed energy can be approximated be 2 nd degree polynomial  Significant energy savings can be achieved by using as low power as possible P Tx,min Energy Model Source: Wietfeld et al., 2012

43. - 43 -  p. marwedel, informatik 12, 2012 Modeling the Impact of Different File Sizes Energy Consumption per Bit [J/Bit] 10 -1 10 -2 10 -3 10 -4 10 -5 10 -6 10 -7 10 1 10 2 10 3 10 4 10 5 10 6 10 7 10 8 Size of Submitted Data (PS) [Byte] Log. linear area with 1/x characteristic Transition Area Saturation Area with constant Energy per Bit The Energy Model for different file sizes can be divided into three parts Log Linear Area: Rapidly decreasing energy consumption per Bit for packet sizes below 20 kByte Transition Area: Transition to constant energy consumption Saturation Area For packet sizes above 900 kByte, collecting more data does not make sense from an energy efficiency perspective Source: Wietfeld et al., 2012

44. - 44 -  p. marwedel, informatik 12, 2012 Validation for Different Devices Outdoor Unit Antenna GPS Energy Consumption per Bit [J/Bit] 10 -2 10 -3 10 -4 10 -5 10 -6 10 1 10 2 10 3 10 4 10 5 10 6 10 7 10 8 Size of Submitted Data (PS) [Byte] Log. linear area with 1/x characteristic Transition Area Saturation Area with constant Energy per Bit Real World Measurement  The overall model is valid for different devices, and different frequency bands  Chipset specific offsets have to be applied for the log linear area  Lab equipment consumes some energy after bits have been sent.  Real (more modern) BS instead of lab equipment is covered by the model (offset has to be applied for low PS) Source: Wietfeld et al., 2012

45. - 45 -  p. marwedel, informatik 12, 2012 Thermal models Thermal models becoming increasingly important  since temperatures become more relevant due to increased performance, and  since temperatures affect usability and dependability.

46. - 46 -  p. marwedel, informatik 12, 2012 Model components  Thermal conductance reflects the amount of heat transferred through a plate (made from that material) of area A and thickness L when the temperatures at the opposite sides differ by one Kelvin.  The reciprocal of thermal conductance is called thermal resistance.  Thermal resistances add up like electrical resistances  Masses storing heat correspond to capacitors  Thermal modeling typically uses equivalent electrical models and employs well-known techniques for solving electrical network equations

47. - 47 -  p. marwedel, informatik 12, 2012 Results of simulations based on thermal models (1) Source: http://www.coolingzone.com/Guest/News/ NL_JUN_2001/Campi/Jun_Campi_2001.html Encapsulated cryptographic coprocessor:

48. - 48 -  p. marwedel, informatik 12, 2012 Results of simulations based on thermal models (2) Source: http://www.flotherm.com/ applications/app141/hot_chip.pdf Microprocessor

49. - 49 -  p. marwedel, informatik 12, 2012 Summary  Thiele’s real-time calculus (RTC)/MPA Using bounds on the number of events in input streams Using bounds on available processing capability Derives bounds on the number of events in output streams Derives bound on remaining processing capability, buffer sizes, … Examples demonstrate design procedure based on RTC  Energy and power consumption  Thermal behavior

50. - 50 -  p. marwedel, informatik 12, 2012 Some more details

51. - 51 -  p. marwedel, informatik 12, 2012 Analysis – Design Question 3 Architecture D is chosen for further investigation. How should the processors be dimensioned? RAD 130 MIPS MMI NAV 113 MIPS 72 kbps (D) © Thiele, ETHZ

52. - 52 -  p. marwedel, informatik 12, 2012 RAD 130 MIPS MMI NAV 113 MIPS 72 kbps 33 MIPS 29 MIPS Analysis – Design Question 3 d max = 200 d max = 1000 d max = 50 d max = 200 © Thiele, ETHZ

53. - 53 -  p. marwedel, informatik 12, 2012 Hamming Distance between adjacent values on the data bus is important

54. - 54 -  p. marwedel, informatik 12, 2012 Other costs E cpu_data =   5 * w (DAddr i ) + ß 5 * h (DAddr i-1, DAddr i ) +  6 * w (Data i ) + ß 6 * h (Data i-1, Data i ) E mem_instr =  MinCostMem(InstrMem,Word_width i ) +  7 * w (IAddr i ) + ß 7 * h (IAddr i-1, IAddr i ) +  8 * w (IData i ) +ß 8 * h (IData i-1, IData i ) E mem_data =  MinCostMem (DataMem, Direction, Word_width i ) +  9 * w (DAddr i ) + ß 9 * h (DAddr i-1, DAddr i ) +  10 * w (Data i ) + ß 10 * h (Data i-1, Data i )