1. Philips Research ICS 252 class, February 3, 2000 1 The Trimedia CPU64 VLIW Media Processor Kees Vissers Philips Research Visiting Industrial Fellow vissers@eecs.berkeley.edu

2. Philips Research ICS 252 class, February 3, 2000 2 Outline Introduction Application Domain Processor Architecture Retargetable compiler Design Space Exploration Results

3. Philips Research ICS 252 class, February 3, 2000 3 Design Problem VLIW cpu I$ video-in video-out audio-in SDRAM audio-out PCI bridge Serial I/O timersI 2 C I/O D$

4. Philips Research ICS 252 class, February 3, 2000 4 Application Domain High volume consumer electronics products future TV, home theatre, set-top box, etc. Embedded core Media processing: audio, video, graphics, communication

5. Philips Research ICS 252 class, February 3, 2000 5 Media Processing CPU Design considerations: Cost (embedded in high volume products) Performance for the application domain Ease of use (programming model)  Benchmark suite needed

6. Philips Research ICS 252 class, February 3, 2000 6 Application: Natural Motion n n-1 n-1/2 picture number -D/2 D/2

7. Philips Research ICS 252 class, February 3, 2000 7 Project Approach: Y-chart Machine Description Benchmark Applications Compiler Simulator Performance Numbers

8. Philips Research ICS 252 class, February 3, 2000 8 Benchmark Suite Characteristics Each application: typical for a class of applications within application domain The set covers a significant area of the application domain Each benchmark is sufficiently well tuned to the architecture to measure its performance

9. Philips Research ICS 252 class, February 3, 2000 9 The Benchmark Suite

10. Philips Research ICS 252 class, February 3, 2000 10 Signal rates audio, video (at block and pixel rate) Basic data types byte (8), half-words (16), words (32), float (32) Data access patterns sample stream, bitstream, random access Data independent and dependent load Control processing and signal processing Processing Characteristics

11. Philips Research ICS 252 class, February 3, 2000 11 Application Development Algorithm Reference C TM1000 optimized CPU64 optimized simulation exploration IC video knowledge

12. Philips Research ICS 252 class, February 3, 2000 12 Initial Design Considerations Goal: 6-8 times TM1000 performance Standard ANSI-C, reuse of code Utilize instruction and data parallelism Limited complexity (embedded core) Compatibility through recompilation VLIW architecture

13. Philips Research ICS 252 class, February 3, 2000 13 Instruction Set Considerations A machine operation must be sufficiently generic within the application domain Sufficiently powerful operations Limited number of operations Consistency and orthogonality

14. Philips Research ICS 252 class, February 3, 2000 14 Results: Natural Motion nops cache stalls instructions Average gain: 2.7x (cycles) 10 15 5 10 15 5 CPU64 cycles x 10 7 TM1000 cycles x 10 7 SAD MED SUB UPC SEG SAD MED SUB UPC SEG 020406080100%020406080100% 00

15. Philips Research ICS 252 class, February 3, 2000 15 Natural Motion Dynamic Load 102030405060708090100110 0 50 100 150 Field number cpu load (%) 90% 16% tm1000 @ 125MHz cpu64 @ 300MHz

16. Philips Research ICS 252 class, February 3, 2000 16 Summary Application domain: –Media processing for CE industry Benchmark suite: –Targeted to application domain –Optimized for class of processors Future products: –Gradual shift to software implementations of signal processing functions

17. Philips Research ICS 252 class, February 3, 2000 17 Outline Introduction Application Domain Processor Architecture Retargetable compiler Design Space Exploration Results

18. Philips Research ICS 252 class, February 3, 2000 18 Application Target Processor core, to be embedded in different ICs and products Real time processing of media streams Cost-sensitive consumer electronics market

19. Philips Research ICS 252 class, February 3, 2000 19 Embedded Application VLIW cpu I$ video-in video-out audio-in SDRAM audio-out PCI bridge Serial I/O timersI 2 C I/O D$

20. Philips Research ICS 252 class, February 3, 2000 20 Performance Target 6x to 8x performance increase, to process more or higher-resolution video streams Not more than double transistor count Relative to TriMedia TM1000 (5-slot VLIW, 100MHz, 32-bit datapath, media operations):

21. Philips Research ICS 252 class, February 3, 2000 21 Efficiency Good performance/silicon cost ratio by: Optimize CPU architecture and media benchmark source code towards each other Solve resource conflicts at compile time

22. Philips Research ICS 252 class, February 3, 2000 22 Outline Introduction VLIW architecture Instruction set IDCT example Results

23. Philips Research ICS 252 class, February 3, 2000 23 Architectural Speedup Not simply increasing the VLIW issue width: Diminishing gain of compiler-generated ILP Increasing implementation complexity (area, timing)

24. Philips Research ICS 252 class, February 3, 2000 24 Architectural Speedup Extended SIMD: uniform 64-bit design, vectors of 1-, 2-, and 4-byte elements (data throughput per cycle) New, extensive, media instruction set (functionality per cycle) Improved cache control (prefetch, alloc)

25. Philips Research ICS 252 class, February 3, 2000 25 CPU64 Architecture data cache 16KB mmu 64-bit memory bus multi-port 128 words x 64 bits register file FU instruction cache 32 KB instruction cache 32 KB mmu bypass network PC exceptions 32-bit peripheral bus VLIW instruction decode and launch

26. Philips Research ICS 252 class, February 3, 2000 26 Code Example int a[n], b[n], c[n]; int i; for (i=0; i<n; i++) c[i]=a[i]+b[i]; RISC instructions:VLIW instructions (5 issueslots): 1x = *a1x = *ay = *bi + = 1a += 1b += 1 2y = *b2g = i<nnopnop nopnop 3i += 13jmpf g 7jmpt g 1nop nopnop 4g = i<n4z = x+ynopnop nopnop 5z = x+y5*c = zc+=1nopnop nop 6*c = z6nopnopnop nopnop 7jmpf g 12 8jmpt g 1 9a += 1 10b += 1 11c += 1

27. Philips Research ICS 252 class, February 3, 2000 27 VLIW challenges Achieve high Instruction Level Parallelism Compiler needs to know the pipeline for instructions Code size Design of a multi-ported register file. Design of the forwarding network

28. Philips Research ICS 252 class, February 3, 2000 28 Instruction Level Parallelism Loop unrolling, software pipelining Guarded execution Avoid branches –if conversion, –hand rewrite source!, valid in Embedded applications, done for TI, Philips and Apple software (Adobe fotoshop). Speculate branches

29. Philips Research ICS 252 class, February 3, 2000 29 Compiler Detect data dependency at compile time: –examples: c[i]=a[i]+b[i]; potential dependency d[i]=a[i]+c[j]; c[i] might be c[j] c[1]=a[i]+b[i]; no dependency d[i]=a[i]+c[2]; c[1] is never c[2]

30. Philips Research ICS 252 class, February 3, 2000 30 Compiler Schedule the instructions in the proper slot at the proper time, e.g.: –load and stores only in slot 1,2 –adds in all slots –floating point multiply only in slot 3 –add take 1 cycle, mult takes 2 cycles –3 branch delay slots

31. Philips Research ICS 252 class, February 3, 2000 31 Code size Large number of registers cost code size –add r1 r3 r127 Large number of media instructions: code size Use hardware compression techniques to remove nops Variable length instruction format! Vector instructions: SIMD, great for data processing performance

32. Philips Research ICS 252 class, February 3, 2000 32 Code Example char a[n], b[n], c[n]; int i; for (i=0; i<n; i++) c[i]=a[i]+b[i]; :VLIW instructions (5 issueslots): 1x = *ay = *bi + = 1a += 1b += 1 2g = i<nnopnopnopnop 3jmpf g 7jmpt g 1nopnopnop 4z = x+ynopnopnopnop 5*c = zc+=1nopnopnop 6nopnopnopnopnop

33. Philips Research ICS 252 class, February 3, 2000 33 Code Example Vector vec64sb a[n/8], b[n/8], c[n/8]; int i; for (i=0; i</8n; i++) c[i]=sb_add(a[i],b[i]); :VLIW instructions and SIMD (5 issueslots): 1x = ld64b ay =ld64b bi + = 1a += 8b += 8 2g = i<fraction nnopnopnopnop 3jmpf g 7jmpt g 1nopnopnop 4z = add64sb x ynopnopnopnop 5st64b c zc+=1nopnopnop 6nopnopnopnopnop

34. Philips Research ICS 252 class, February 3, 2000 34 Vector Instruction Example Sum of absolute differences (“ub_me”)

35. Philips Research ICS 252 class, February 3, 2000 35 Application Code Example int calc_sad(vec64ub *prv, vec64ub *cur, int s) { vec64ub left, right; int i; int sad = 0; for (i=0; i<8; i++) { left = prv[i*s]; right = cur[i*s]; sad += ub_me(left, right); } return(sad); }

36. Philips Research ICS 252 class, February 3, 2000 36 Architecture VLIW+SIMD Issues a 5-slot instruction every cycle Each slot supports a selection of FUs All FUs support vectorized (SIMD) data Double-slot FU allows powerful multi- argument, multi-result operations All FUs are pipelined, latency 1 to 4 (except floating point divide and sqrt)

37. Philips Research ICS 252 class, February 3, 2000 37 Instruction Format Per operation, per slot: Up to 2 arguments, up to 1 result Optionally an extra register for guarding (conditional execution) Immediate argument size can be 32 bit Instructions have compressed variable length format, decompressed during decode

38. Philips Research ICS 252 class, February 3, 2000 38 Operations Intends to cover combinations of: 1-, 2-, 4-byte elements in an 8-byte vector Signed or unsigned type Clipped versus wrap-around arithmetic Set of basic functions (ld, st, add, mul, compare, shift,...)

39. Philips Research ICS 252 class, February 3, 2000 39 Instruction Set

40. Philips Research ICS 252 class, February 3, 2000 40 Example: msp-multiply Multiply to internal double precision integer Each argument: 4-way 16-bit int Round lower half into upper half Return upper half ++++ 1111

41. Philips Research ICS 252 class, February 3, 2000 41 Example: Transpose m m n n o o p p i i j j k k l l e e f f g g h h a a b b c c d d b b f f j j n n a a e e i i m m 2-slot ‘super-op’: takes 4 arguments, shuffles bytes, produces 2 results (2-dimensional filtering)

42. Philips Research ICS 252 class, February 3, 2000 42 Example: 4-way average (MPEG 2-dimensional half-pixel average) ++++++++++++++++ 1 add with extended precision round 1

43. Philips Research ICS 252 class, February 3, 2000 43 Branches Branch operations have 3 branch delay slots Compiler & scheduler try to fill these (profiling, loop unrolling, inlining, guarding) Branch units are properly pipelined Up to 3 branch ops in 1 instruction No branch prediction hardware Branches are preferred moments for interrupt servicing

44. Philips Research ICS 252 class, February 3, 2000 44 2D-IDCT Example IDCT code was tested early in the project, also for studying the programming model Mapped through our experimental C compiler and instruction scheduler Operates entirely on (vectors of) 16-bit data elements Simulation showed IEEE 1180 accuracy compliancy

45. Philips Research ICS 252 class, February 3, 2000 45 2D-IDCT Instructions OK Execution time in cycles, excluding data cache misses VLIW: CPU64, TM1000, TI 320C60 others superscalar OK = accuracy compliant

46. Philips Research ICS 252 class, February 3, 2000 46 Status Now in construction at Philips Semiconductors, Sunnyvale 7M transistors (target) 300Mhz clock (target) 0.18  technology 1.8 volt

47. Philips Research ICS 252 class, February 3, 2000 47 Conclusion Processor Architecture The 5-slot 64-bit VLIW provides a powerful architecture for media processing Performance gain of 6x to 8x on TM1000 Rich and regular media instruction set Powerful multi-argument, multi-result ‘super- op’ naturally fits VLIW architecture

48. Philips Research ICS 252 class, February 3, 2000 48 Outline Introduction Application Domain Processor Architecture Retargetable compiler Design Space Exploration Results

49. Philips Research ICS 252 class, February 3, 2000 49 Traditional Compilation App 1 App 2 App 3 Compiler A Exe 1A Exe 2A Exe 3A Exe 1B Exe 2B Exe 3B Compiler B Machine A Machine B

50. Philips Research ICS 252 class, February 3, 2000 50 Retargetable Compilation App 1 App 2 App 3 Compiler Exe 1A Exe 2A Exe 3A Exe 1B Exe 2B Exe 3B Machine A Machine B Machine A Description Machine B Description

51. Philips Research ICS 252 class, February 3, 2000 51 MD file contents MD describes instance from class of machines Contains information such as: –number & width of registers –number of issue slots –number & placement of function units –instruction latencies

52. Philips Research ICS 252 class, February 3, 2000 52 Y-Chart Machine Description Benchmark Applications Compiler Simulator Performance Numbers

53. Philips Research ICS 252 class, February 3, 2000 53 Software and Hardware Operations Hardware view of instruction set –Implement minimal, changeable set Software view of instruction set –provide regular, orthogonal, stable set Two instruction sets were defined for these purposes

54. Philips Research ICS 252 class, February 3, 2000 54 Instruction set libraries Hardware operations library –untyped, minimal, changing Software operations library –vector-typed, orthogonal, stable Mapping in MD file and library

55. Philips Research ICS 252 class, February 3, 2000 55 Library structure simulatorhw_opssw_opsapplication simulatorapplication source code workstation executable CPU64 executable

56. Philips Research ICS 252 class, February 3, 2000 56 Functional Development hw_opssw_opsapplication source code workstation executable gcc

57. Philips Research ICS 252 class, February 3, 2000 57 Code Tuning simulatorhw_opsapplication simulatorapplication gcc interpretation tmcc source code workstation executable CPU64 executable

58. Philips Research ICS 252 class, February 3, 2000 58 Conclusions Compiler Applications are written in C code only Machine Description supports changes in ISA Vector types keep code legible Fast functional development track Accurate application tuning track

59. Philips Research ICS 252 class, February 3, 2000 59 Outline Introduction Application Domain Processor Architecture Retargetable compiler Design Space Exploration Results

60. Philips Research ICS 252 class, February 3, 2000 60 Methodology: the Y-chart Machine Description Benchmark Applications Compiler Simulator Performance Numbers

61. Philips Research ICS 252 class, February 3, 2000 61 64-bit VLIW CPU Many parameters, large design space(s) data cache 16KB mmu multi-port 128 words x 64 bits register file FU 64-bit memory bus instruction cache 32 KB instruction cache 32 KB mmu bypass network VLIW instruction decode and launch

62. Philips Research ICS 252 class, February 3, 2000 62 Multimedia benchmark Nine multimedia applications from: –Data communication –Audio coding –Video coding –Video processing –Graphics Representative –Applications –Code –Datasets

63. Philips Research ICS 252 class, February 3, 2000 63 Challenge for design space exploration Simulation time for the benchmark for a single machine is 18 hours The number of design points for functional unit configuration alone: Results in 2000,000,000,000 years of computation time

64. Philips Research ICS 252 class, February 3, 2000 64 Essential tooling Retargetable toolchain –Core compiler, scheduler, simulator Design and experiment management DSE support tools –Machine generation, pseudosim, analysis Glue software

65. Philips Research ICS 252 class, February 3, 2000 65 Pseudo-simulation Pseudosim: a retargetable pseudo-simulator Performs a cycle-accurate calculation of the amount of instruction cycles, without doing the actual machine simulation Gathers other machine statistics, such as utilisation of slots, functional units, operations Time for the whole benchmark is reduced from 18 hours to 4 minutes

66. Philips Research ICS 252 class, February 3, 2000 66 Pseudo-simulation once, 18 hours many times, 4 minutes Simulator Compiler Pseudosim Machine description Benchmark applications Machine description’ code statistics results

67. Philips Research ICS 252 class, February 3, 2000 67 Functional unit configuration Problem: How many of each type of FU do I need? Where do I place them? How do I prevent simulating too many machine variations? slot FU type

68. Philips Research ICS 252 class, February 3, 2000 68 Naïve calculation of the design space size 24 types of FU’s 31 configurations per type 6 types of super-op FU’s 7 configurations per type Size of design space:

69. Philips Research ICS 252 class, February 3, 2000 69 Not so naïve calculation of the design space size Assume relationship between FU types Best-guess lower and upper bounds on number of FU’s per type Reduce permutations Size of design space:

70. Philips Research ICS 252 class, February 3, 2000 70 Systematic exploration It is not feasible to exhaustively compute all design points in the design space Therefore we probe, partition, and then explore the design space Careful set-up of experiments

71. Philips Research ICS 252 class, February 3, 2000 71 Reduction of the design space Observation: over 93% of operations are done in 30% of FU types Action: partition space –Exhaustive exploration of important FU’s –Greedy exploration of the remainder 70%30% 7% 93% operations functional unit types

72. Philips Research ICS 252 class, February 3, 2000 72 Exhaustive experiment Close to 3000 machine variations Only 4 machines survive Large performance variation due to placement Time taken = 200 hours 7 area cycles Exhaustive experiment

73. Philips Research ICS 252 class, February 3, 2000 73 Greedy experiments 7 area First greedy experiment cycles Less machine variations per experiment More machines survive Less performance variation due to placement Close to another 3000 machine variations over all greedy experiments

74. Philips Research ICS 252 class, February 3, 2000 74 Outline Introduction Tools Exploration Real numbers Conclusions

75. Philips Research ICS 252 class, February 3, 2000 75 Simulated machines 180 Gbyte data transferred 800 Mbyte compressed data stored 2T cycles simulated 10T operations issued

76. Philips Research ICS 252 class, February 3, 2000 76 Outline Introduction Tools Exploration Real numbers Conclusions

77. Philips Research ICS 252 class, February 3, 2000 77 Summary and conclusions A full-fledged design space exploration was done for the 64-bit TriMedia VLIW CPU core The use of both pseudo-simulation and a systematic exploration has made this feasible The outcome is: –A range of machines in A-T space –Rules for functional unit placement –A flexible, integrated environment for continuation of DSE

78. Philips Research ICS 252 class, February 3, 2000 78 Summary and conclusions Design Space Exploration A large amount of time is spent in developing tools to enable the DSE The design of experiments and the analysis of results takes up more time than the execution Performing a DSE stretches the capabilities of the tools to the maximum The pseudo-simulator is a powerful tool that supports the full design space offered by the machine description

79. Philips Research ICS 252 class, February 3, 2000 79 Conclusions VLIW Difficult to exploit ILP in general purpose, great for embedded applications, streaming applications and DSP. Compiler is essential Good in combination with vector or SIMD Quantify, do experiments, analyse results and keep doing this: Y-chart. Promising for binary to binary (Transmeta: code morphing), Intel-HP: IA-64 Revival: Philips, Sun MAJC, TI 320C60, Crusoe 3120.