What Choices Make A Killer Video Processor Architecture? Jonah Probell Ultra Data Corp

© Copyright 2004 Jonah Probellslide 2 Outline Overview of Ultra Data UD3000 Software programmability Parallelism –VLIW –SIMD –Multiprocessing Appropriate use of on- and off-chip memory –Optimal organization of data structures in DRAM Deterministic performance –5-port regfile –2-port on-chip memory –DMA controller instead of caches

© Copyright 2004 Jonah Probellslide 3 Nobodys Video Decoder Chip SDRAM high-speed interconnect Video Decode Processor Peripheral bus bridge Host / audio processor SDRAM controller Video post- processing peripheral bus Video output S-video / raw 24-bit RGB or 8/16-bit YCrCb Audio output I 2 S / SPDIF / raw I 2 C, SATA, timers DVD optical interface SATA & I 2 C bussesOptics sled Audio / Video DACs

© Copyright 2004 Jonah Probellslide 4 The Ultra Data UD3000 Outer Loop Processor 0 Crossbar Switch Fabric System Bus Bridge Inner Loop Processor 1 instruction extensions Inner Loop Processor 0 Inner Loop Processor 2 Smart 2-D DMA Controller 2-port DMEM 2-port DMEM … FIFO … Test & Set Outer Loop Processor 1 instruction extensions

© Copyright 2004 Jonah Probellslide 5 H.264 Main Profile Decode ILP 0 DMA ctrl ILP 1 OLP 1 ILP 2 OLP 0 CABAC CA VLC interpolation inverse transform apply deltas Deblocking thresholds Deblocking Filter load prediction source store block

© Copyright 2004 Jonah Probellslide 6 The Inner Loop Processor Data Aligner IMEM Control Unit 32-bit RISC Program Counter Loads & Stores Vector Unit 64-bit SIMD data Multiply Acc Data packing 3-port Regfile 5-port Regfile Switch Fabric 32 64

© Copyright 2004 Jonah Probellslide 7 Video Codec Standards ITU-T standards ITU-T / MPEG joint standards MPEG standards H.261H.263 H.262 / MPEG-2 H.264 / MPEG-4 Part 10 AVC MPEG-1MPEG-4 VP3 On2 Technologies standards DivX Networks standard DivX VP4VP5VP6 Microsoft standard Windows Media Video

© Copyright 2004 Jonah Probellslide 8 VLIW Parallelism load multiply load multiply load multiply shift store add branch sequential DSP program program sequencer regfile data memory ALU + - x & | ! >> << load multiply store shift branch add VLIW DSP program

© Copyright 2004 Jonah Probellslide 9 SIMD Parallelism frame of macroblocks macroblock of pixels 8x8 block of pixels 4x4 block of pixels

© Copyright 2004 Jonah Probellslide 10 Multiprocessor Parallelism video codec motion estimation prediction transform & compression deblocking software system CPU 0 CPU 1 CPU 2 hardware symmetric parallel multiprocessing video codec motion estimation prediction transform & compression deblocking software system CPU 0 CPU 1 CPU 2 hardware pipelined multiprocessing

© Copyright 2004 Jonah Probellslide 11 Data Bandwidths bitstrea m source SDRAM temporary data storage display devicevideo chip

© Copyright 2004 Jonah Probellslide 12 DRAM Optimal Data Ordering DRAM : 1k byte rows Frame mapped to DRAM rows as a C-style two- dimentional array Frame mapped to DRAM rows as square groups

© Copyright 2004 Jonah Probellslide 13 Deterministic Performance

© Copyright 2004 Jonah Probellslide 14 The Inner Loop Processor Data Aligner IMEM Control Unit 32-bit RISC Program Counter Loads & Stores Vector Unit 64-bit SIMD data Multiply Acc Data packing 3-port Regfile 5-port Regfile Switch Fabric 32 64

© Copyright 2004 Jonah Probellslide 15 The Ultra Data UD3000 Outer Loop Processor 0 Crossbar Switch Fabric System Bus Bridge Inner Loop Processor 1 instruction extensions Inner Loop Processor 0 Inner Loop Processor 2 Smart 2-D DMA Controller 2-port DMEM 2-port DMEM … FIFO … Test & Set Outer Loop Processor 1 instruction extensions

© Copyright 2004 Jonah Probellslide 16 A Killer Video Processor Architecture Software programmability Parallelism –VLIW –SIMD –Multiprocessing Appropriate use of on- and off-chip memory –Optimal organization of data structures in DRAM Deterministic performance –5-port regfile –2-port on-chip memory –DMA controller instead of caches

© Copyright 2004 Jonah Probellslide 17 Acknowledgements This presentation is © Copyright 2004 Jonah Probell ALL RIGHTS RESERVED. Certain information for this document was derived from publicly available documents of Ultra Data Corp., UB Video Inc., On2 Technologies Inc., and Wikipedia. All trademarks mentioned in this document are property of their respective owners and are hereby acknowledged. Jonah Probell (781)