1. Wyner-Ziv Residual Coding of Video Anne Aaron, David Varodayan and Bernd Girod Information Systems Laboratory Stanford University

2. A. Aaron, D. Varodayan and B. Girod: Wyner-Ziv Residual Coding of VideoApril 26, 2006 2 Outline Wyner-Ziv video coding Pixel-domain Wyner-Ziv residual video codec Simulations Results – Side information from motion-compensated interpolation – Side information from hash-based motion estimation at the receiver

3. A. Aaron, D. Varodayan and B. Girod: Wyner-Ziv Residual Coding of VideoApril 26, 2006 3 “Motion JPEG” Decoder “Motion JPEG” Encoder Wyner-Ziv Interframe Decoder Wyner-Ziv Intraframe Encoder Side Information Video Coding with Low Complexity [Witsenhausen, Wyner, 1980] [Puri, Ramchandran, Allerton 2002] [Aaron, Zhang, Girod, Asilomar 2002]

4. A. Aaron, D. Varodayan and B. Girod: Wyner-Ziv Residual Coding of VideoApril 26, 2006 4 XnXn Wyner-Ziv Residual Coding Wyner-Ziv Decoder Wyner-Ziv Encoder Side Information X’ n Y

5. A. Aaron, D. Varodayan and B. Girod: Wyner-Ziv Residual Coding of VideoApril 26, 2006 5 Wyner-Ziv Residual Coding Wyner-Ziv Decoder Wyner-Ziv Encoder Motion-Compensated Side Information X’ n Previous Frame Side Information - XnXn Frame difference Y

6. A. Aaron, D. Varodayan and B. Girod: Wyner-Ziv Residual Coding of VideoApril 26, 2006 6 Wyner-Ziv Residual Coding Wyner-Ziv Decoder Wyner-Ziv Encoder X’ n - XnXn Frame difference - Y Side Info

7. A. Aaron, D. Varodayan and B. Girod: Wyner-Ziv Residual Coding of VideoApril 26, 2006 7 Wyner-Ziv Decoder Scalar Quantizer LDPC Encoder Buffe r WZ frames X Wyner-Ziv Encoder LDPC Decoder Request bits Slepian-Wolf Codec Interpolation/ Extrapolation Reconstruction Y Key frames I Conventional Intraframe coding Conventional Intraframe decoding X’ I’ Side information Pixel-Domain Wyner-Ziv Residual Video Codec X er - - Q -1 Frame Memory [Varodayan, Aaron, Girod, 2005]

8. A. Aaron, D. Varodayan and B. Girod: Wyner-Ziv Residual Coding of VideoApril 26, 2006 8 Y Set-up 1: Side Information from Motion-Compensated Interpolation WZ frame W W’ Decoded WZ frames Side information Wyner-Ziv Residual Encoder WZ parity bits Interpolation Decoded frames Wyner-Ziv Residual Decoder I I I Previous key frame as encoder reference

9. A. Aaron, D. Varodayan and B. Girod: Wyner-Ziv Residual Coding of VideoApril 26, 2006 9 Rate-Distortion Performance – Salesman Alternating key frames and WZ frames Side information from motion- compensated interpolation Salesman QCIF sequence 15fps, 96 frames 100200300400500600700800900 26 28 30 32 34 36 38 40 42 44 Rate (kbps) PSNR (dB) H.263+ I-B-I-B WZ DCT WZ Pixel Residual Motion-JPEG+

10. A. Aaron, D. Varodayan and B. Girod: Wyner-Ziv Residual Coding of VideoApril 26, 2006 10 Rate-Distortion Performance – Foreman Alternating key frames and WZ frames Side information from motion- compensated interpolation Foreman QCIF sequence 15fps, 96 frames 100200300400500600700800900 26 28 30 32 34 36 38 40 42 44 Rate (kbps) PSNR (dB) H.263+ I-B-I-B WZ DCT WZ Pixel Residual Motion-JPEG+

11. A. Aaron, D. Varodayan and B. Girod: Wyner-Ziv Residual Coding of VideoApril 26, 2006 11 Previous frame as encoder reference Y Set-up 2: Hash-Based Side Information WZ frame W W’ Decoded WZ frames Side information Wyner-Ziv Residual Encoder WZ parity bits Motion- compensated Extrapolation Hash Generator (with previous hash store) Hash Generator (with previous hash store) Hash bits Decoded previous frame Wyner-Ziv Residual Decoder “I-P-P-P structure” I

12. A. Aaron, D. Varodayan and B. Girod: Wyner-Ziv Residual Coding of VideoApril 26, 2006 12 Rate-Distortion Performance – Salesman 100200300400500600 26 28 30 32 34 36 38 40 42 44 Rate (kbps) PSNR (dB) H.263+ I-P-P-P WZ DCT with DCT Hash WZ Pixel Residual with Pixel Residual Hash Motion-JPEG+ Every 8 th frame is a key frame Side information from hash- based motion estimation at the decoder Salesman QCIF sequence 15fps, 96 frames

13. A. Aaron, D. Varodayan and B. Girod: Wyner-Ziv Residual Coding of VideoApril 26, 2006 13 Rate-Distortion Performance – Salesman Every 8 th frame is a key frame Side information from hash- based motion estimation at the decoder Salesman QCIF sequence 15fps, 96 frames

14. A. Aaron, D. Varodayan and B. Girod: Wyner-Ziv Residual Coding of VideoApril 26, 2006 14 Rate-Distortion Performance – Foreman 100200300400500600700800900 26 28 30 32 34 36 38 40 42 44 Rate (kbps) PSNR (dB) H.263+ I-P-P-P WZ DCT with DCT Hash WZ Pixel Residual with Pixel Residual Hash Motion-JPEG+ Every 8 th frame is a key frame Side information from hash- based motion estimation at the decoder Foreman QCIF sequence 15fps, 96 frames

15. A. Aaron, D. Varodayan and B. Girod: Wyner-Ziv Residual Coding of VideoApril 26, 2006 15 Rate-Distortion Performance – Foreman 100200300400500600700800900 26 28 30 32 34 36 38 40 42 44 Rate (kbps) PSNR (dB) H.263+ I-P-P-P WZ Pixel Residual with Pixel Residual Hash Motion-JPEG+ WZ Pixel with Pixel Hash Every 8 th frame is a key frame Side information from hash- based motion estimation at the decoder Foreman QCIF sequence 15fps, 96 frames

16. Motion-JPEG+ Rate Y =194 kbps PSNR Y =29.9 dB Wyner-Ziv Pixel Residual Coding Rate Y =192 kbps PSNR Y =36.6 dB Salesman at 15 fps

17. A. Aaron, D. Varodayan and B. Girod: Wyner-Ziv Residual Coding of VideoApril 26, 2006 17Summary Wyner-Ziv coding on residual video frames Encoder can use simple reference frame while decoder applies more sophisticated motion compensation Good rate-distortion performance for pixel-domain system – Up to 8 dB improvement over independently encoding the frames – Comparable performance to transform-domain Wyner- Ziv coding scheme