Reinventing Compression: The New Paradigm of Distributed Video Coding Bernd Girod Information Systems Laboratory Stanford University

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 2 Outline  Lossless and lossy compression with receiver side information  Shifting the complexity of video encoding to the decoder  Error-resilient video transmission  Image authentication

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 3 Outline  Lossless and lossy compression with receiver side information  Shifting the complexity of video encoding to the decoder  Error-resilient video transmission  Image authentication

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 4 Encoder Decoder Lossless Compression with Side Information R ≥ H(X|Y) Statistically dependent Encoder Decoder R ≥ ? Statistically dependent Side Information

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 5 Encoder Decoder Lossless Compression with Side Information R ≥ H(X|Y) Statistically dependent Encoder Decoder R ≥ H(X|Y) Statistically dependent [Slepian, Wolf, 1973]

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 6 Towards Practical Slepian-Wolf Coding Convolution coding for data compression [Blizard, 1969] Convolutional source coding [Hellman, 1975] Syndrome source coding [Ancheta, 1976] Coset codes [Pradhan and Ramchandran, 1999] Trellis codes [Wang and Orchard, 2001] Turbo codes [García-Frías and Zhao, 2001] [Bajcsy and Mitran, 2001] [Aaron and Girod, 2002] LDPC codes [Liveris, Xiong, and Georghiades, 2002]...

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding H(X|Y) Rate Slepian-Wolf bound Rate = H(X|Y) Rate-adaptive turbo codes Rate-Adaptive Slepian-Wolf Coding Turbo Decoder Turbo Encoder Parity bits Encoder Buffer Request bits L = 8192 bits Total simulated bits = 2 26

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 8 Encoder Decoder Encoder Decoder Lossy Compression with Side Information [Wyner, Ziv, 1976] For MSE distortion and Gaussian statistics, rate-distortion functions of the two systems are the same. [Zamir, 1996] The rate loss R * (d) – R X|Y (d) is bounded. R X|Y (d) R * (d)

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 9 Practical Wyner-Ziv Coding Wyner-Ziv Decoder Quantizer Slepian- Wolf Encoder Wyner-Ziv Encoder Slepian- Wolf Decoder Minimum Distortion Reconstruction

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 10 Non-Connected Quantization Regions  Example: Non-connected intervals for scalar quantization  Decoder: Minimum mean-squared error reconstruction with side information x x

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 11 Outline  Lossless and lossy compression with receiver side information  Shifting the complexity of video encoding to the decoder  Error-resilient video transmission  Image authentication

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 12 Interframe Video Coding Predictive Interframe Decoder Predictive Interframe Encoder Side Information X’

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 13 Wyner-Ziv Interframe Decoder Wyner-Ziv Intraframe Encoder [Witsenhausen, Wyner, 1980] [Puri, Ramchandran, Allerton 2002] [Aaron, Zhang, Girod, Asilomar 2002] Low Complexity Encoder X’ Side Information

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 14

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 15 Pixel-Domain Wyner-Ziv Video Codec Interframe Decoder Scalar Quantizer Turbo Encoder Buffe r WZ frames X Intraframe Encoder Turbo Decoder Request bits Slepian-Wolf Codec Interpolation/ Extrapolation Reconstruction Y Key frames I Conventional Intraframe coding Conventional Intraframe decoding X’ I’ Side information [Aaron, Zhang, Girod, Asilomar 2002]

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 16 Decoder side information generated by motion- compensated interpolation PSNR 30.3 dB After Wyner-Ziv Decoding 16-level quantization – bpp 11 pixels in error PSNR 36.7 dB Pixel-Domain Wyner-Ziv Video Codec

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 17 Pixel-Domain Wyner-Ziv Video Codec Decoder side information generated by motion- compensated interpolation PSNR 24.8 dB After Wyner-Ziv Decoding 16-level quantization – 2.0 bpp 0 pixels in error PSNR 36.5 dB

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 18 YkYk IDCT DCT-Domain Wyner-Ziv Video Codec Request bits Interpolation/ Extrapolation Recon I Conventional Intraframe coding Conventional Intraframe decoding DCT For each transform band k I’ W’ Y Scalar Quantizer DCT Turbo Encoder Buffer Turbo Decoder Side information WZ frames W Key frames XkXk Xk’Xk’ Interframe DecoderIntraframe Encoder

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 19 Rate-Distortion Performance - Salesman Every 8 th frame is a key frame Salesman QCIF sequence at 10fps 100 frames 6 dB 3 dB Interframe 100% Encoder Runtime Pentium 1.73 GHz machine

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 20 Rate-Distortion Performance – Hall Monitor 8 dB 3 dB Every 8 th frame is a key frame Hall Monitor QCIF sequence at 10fps 100 frames

DCT-based Intracoding 149 kbps PSNR Y =30.0 dB Wyner-Ziv DCT codec 152 kbps PSNR Y =35.6 dB GOP=8 Salesman at 10 fps

DCT-based Intracoding 156 kbps PSNR Y =30.2 dB Wyner-Ziv DCT codec 155 kbps PSNR Y =37.1 dB GOP=8 Hall Monitor at 10 fps

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 23 Outline  Lossless and lossy compression with receiver side information  Shifting the complexity of video encoding to the decoder  Error-resilient video transmission  Image authentication

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 24 Systematic Lossy Source/Channel Coding  Information theoretic optimality conditions [Shamai, Verdú, Zamir, 1998]  Enhancing analog image transmission using digital side information [Pradhan, Ramchandran, 2001]  Lossy source-channel coding of video waveforms [Rane, Aaron, Girod, 2004,’05,’06] Encoder Digital Channel Decoder Analog Channel Wyner-Ziv Encoder Side info Wyner-Ziv Decoder Digital Channel Wyner-Ziv Encoder Side info Wyner-Ziv Decoder Digital Channel

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 25 Systematic Lossy Error Protection (SLEP) Video Encoder Video Decoder With Error Concealment Input Video With Errors Channel Wyner-Ziv Encoder Wyner-Ziv Decoder Side Information Output Video “Analog Channel”

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 26 SLEP using H.264/AVC Redundant Slices Encode Redundant Pic (Requantize) Entropy Decoding WYNER-ZIV ENCODER WYNER-ZIV DECODER Error-prone Channel Decode Redundant Slice Motion Vecs + Coding Modes Erasure Decoding Side info Motion Vecs + Coding Modes QP Recovered motion vectors for erroneously received primary slices Encode Primary Pic Q -1 T -1 H.264/AVC DECODER + Entropy Decoding Output Video H.264/AVC ENCODER Input Video Encode Redundant Pic (Requantize) RS Encoder Parity Slices QP MC

408 kbps, error resilience bit rate = 40 kbps Symbol error probability = 5 x QP = dB Error concealment only 40 kbps FEC SLEP with redundant QP = 36 SLEP with redundant QP = 40 SLEP with redundant QP = dB 25.5 dB 30.9 dB 34.2 dB 32.9 dB Error-free After error propagation

100 kbps FEC PSNR: 32.5 dB Recovered 53.7 % of lost macroblocks 100 kbps Wyner-Ziv bit stream PSNR: 38.0 dB Recovered 96.6 % of lost macroblocks 1 Mbps Symbol error probability = 2 x 10 -4

Rally, 1 Mbps, 3% packet loss 80 kbps Wyner-Ziv bit stream 38.1 dB 80 kbps FEC 33.4 dB Recovered 67.5 % of lost macroblocks Recovered 97.1 % of lost macroblocks

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 30 Outline  Lossless and lossy compression with receiver side information  Shifting the complexity of video encoding to the decoder  Error-resilient video transmission  Image authentication

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 31 Media Authentication Problem Legimate degradation (e.g., compression) Illegimate degradation (e.g., compression + tampering) How to distinguish legimate and illegimate signal degradations without access to the original? Original Received

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 32 Image Authentication by Distributed Coding Original Received Coarse approximation or (random) projection Slepian-Wolf coder Slepian-Wolf decoder ? Side information [Lin, Varodayan, Girod, ICIP 2007] [Lin, Varodayan, Girod, MMSP 2007]

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 33 Image Authentication by Distributed Coding Original Received Coarse approximation or (random) projection Slepian-Wolf coder Slepian-Wolf decoder ? Side information [Lin, Varodayan, Girod, ICIP 2007] [Lin, Varodayan, Girod, MMSP 2007]

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 34 Image Authentication by Distributed Coding [Lin, Varodayan, Girod, ICIP 2007]

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 35 Minimum Rate for Successful Decoding Experiment: JPEG or JPEG2000 compression + illegimate text banner [Lin, Varodayan, Girod, ICIP 2007]

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 36 Demo

B. Girod: Reinventing Compression: The New Paradigm of Distributed Video Coding 37 Distributed Image/Video Coding: Why Do We Care?  New Paradigm: Chance to Reinvent Compression from Scratch –Entropy coding –Quantization –Signal transforms –Adaptive coding –Rate control –...  Powerful New Tool in the Compression Tool-Box –Very low complexity encoders –Compression for networks of cameras –Error-resilient transmission of signal waveforms –Digitally enhanced analog transmission –Unequal error protection without layered coding –Image authentication –Random access –Compression of encrypted signals –...

Further interest: B. Girod, A. Aaron, S. Rane, D. Rebollo-Monedero, "Distributed Video Coding," Proceedings of the IEEE, Special Issue on Video Coding and Delivery. January