1. TM Paramvir Bahl bahl@microsoft.com Microsoft Corporation Adaptive Region-Based Multi-Scaled Motion- Compensated Video Coding for Error Prone Communication Channels Wei-Lien Hsu whsu@mail.dec.com Digital Equipment Corporation SPIE ‘97, Dallas, USA November 4, 1997

2. TM Outline  Video encoder description  Transmission model  Simulation methodology  Performance (experimental) results  Conclusions

3. TM Objective and Approach  Objective: –design a low complexity video compression algorithm for robust transmission over hostile communication channels.  Approach: –Spatially segment video frames into video regions, decompose video regions into sub-bands, apply unequal error protection to different regions and different sub-bands, carry out prioritized transmission and apply novel reconstruction to guarantee a minimum spatial and temporal resolution at the receiver.

4. TM Description of the Video Encoder  Our codec.vs. ITU’s H.263 –Spatial Segmentation (Split-and-Merge algorithm) –Frequency Segmentation (Discrete Wavelet Transform)  Characteristics of our algorithm –compression is achieved by  removing temporal redundancy via classical motion estimation and  removing spatial redundancy via DWT, DCT, quantization and entropy coding. –A new spatial region-segmentation map is generated for every intra- frame, and the same map is used until strong changes appear in the incoming frames.

5. TM Proposed Video Encoder + Subband Id. Region Id. Motion vectors - Write 2D Discrete Wavelet Transformatoio and DCTn Quantizer Quantizer -1 Spatial Segmentation Intra Inverse 2D WT/DCT Picture Type Motion Estimator Motion Compensation Predictor Future Picture Store Previous Picture Store + Inter-Frame Picture Type Inter/Intra RLE + Huffman Picture Type Quantizer Adapter Picture Type Region Map Network Subsystem Inter/Intra Classifier

6. TM Video Frame Segmentation  Intra-Frame Region Segmentation –The (Intra) frame is first partitioned into blocks of size 16 x 16. –The variance of each block i is computed. –All adjacent blocks of similar variances are merged.  Inter-Frame Region Segmentation –Assign the index to each inter-block based on motion estimation. –If the map generated for the frame is different from the one for the previous frame or if the frame contains intra blocks, then performs intra frame segmentation.

7. TM Video Frame Segmentation Threshold (T) = 0.278 Threshold (T) = 0.293

8. TM Discrete Wavelet Transformation  A two-tap Harr filter decomposes each region of the luminance (Y) component into 4 bands –low complexity –capability to decompose arbitrary shaped regions which are multiples of macro-blocks without causing any undesirable boundary effect

9. TM Quantization and Bitstream Packing  Quantization –apply different quantization steps to DC and AC subbands  Bit Stream –five layers:  Picture Layer: HEADER,REGION MAP,REGION LAYER  Region Layer  Subband Layer  Macroblock Layer  Block Layer –The DC and AC-subbands for each video frame are transmitted in different slices

10. TM Error Concealment  Classical problems in video reconstruction –Transmission errors due to channel imperfections cause corruption in some of the transmitted video regions rendering them un-decodable –Dynamic reduction in non-reserved bandwidth causes some of the regions not to reach the decoder in a timely manner  Solution –the complete frame is reconstructed at the receiver by using a combination of the current and previous video regions that were received correctly

11. TM Region Reconstruction Video Region (R i,1 ) Video Region (R i,N ) Video Region (R i,3 ) frame i @ Receiver N regions @ Receiver frame i @ Transmitter Video Region (R x,1 ) Video Region (R x,N ) Video Region (R x,3 ) Video Region (R x,2 ) R i,1 R i,N R i,3 R i-1,2 Region Store Substitute Did not reach receiver

12. TM Simulation Methodology Statistics Gathering Error Warning Read-Solomon Coder Interleaver Disk Write QDPSK Modulator Video Sequence Transmitter Model 16-bit CRC Errors Decompressor Disk Read BE RE De-interleaver RS Decoder ber = 10 -2 Hard Decision Receiver Model Channel Model CRC Decoding Compressor

13. TM Simulation Methodology  Transmitter: –video compressor –Read-Solomon forward error correcting encoder –burst error correcting interleaver –CRC error detector  The compressed video data is fragmented into blocks of 48 octets, packaged and transmitted in packets of 53 octets (ATM cell size) –5 octets for header information (2 octets for CRC, 3 octets for miscellaneous information such as connection number, priority,..)

14. TM Simulation Methodology  Error generator –modeled as a modified Gilbert model with the two states representing the Burst Error State (BE) and the Random Error State (RE).

15. TM Experimental Results  Performance

16. TM Experimental Results  Software Performance

17. TM Experimental Results  Bounded Error Propagation Corruption in the H.263 bitstream Corruption in the AC Sub- bands of the unprotected Regions  25.34 dB

18. TM Experimental Results  Improved Temporal Resolution with changing error characteristics (K represents the number of re-transmissions allowed) H263 Region- based H.263 K = 1 K = 3

19. TM Experimental Results  Improved Temporal Resolution with changing bandwidth constraints Region- based H263 Region- based H.263

20. TM Experimental Results  Efficient Bandwidth Utilization –The average bit rate for DC sub-bands was 8 kbps. –The second, third, and forth AC sub-bands had an average bit rate of 3, 2.5, and 2.8 kbps respectively

21. TM Conclusions  Advantage of the proposed region-based multi-resolution video compression algorithm: –allows the transmitter to apply unequal error protection –allows transmitter to dynamically adjust the order and transmission priority of individual regions –allows for improved temporal resolution at the receiver –limits error spreading in both the spatial and the temporal domain and –reduces coding delays as transmission can begin as soon as the first region is compressed. –Good for QoS  Can be used with near optimum reserved bandwidth utilization –Software performance comparable to H.263 Thanks !