1. The SP- and SI-Frames Design for H.264/AVC Marta Karczewicz and Ragip Kurceren IEEE Trans. on Circuit and System for Video Technology, Vol.13, No. 7, July 2003 (Nokia Research Center)

2. Outline What’s SP- and SI-Frames Motivation Decoding and Encoding Processes for SP- and SI-Frames Results

3. What ’ s SP- and SI-Frames (1/2) SP-frames make use of motion compensated predictive coding to exploit temporal redundancy in the sequence similar to P- frames. The difference between SP- and P-frames is that SP-frames allow identical frames to be reconstructed even when they are predicted using different reference frames.

4. What ’ s SP- and SI-Frames (2/2) bitstream switchingsplicingrandom accessfast forwardfast backwarderror resilience/recovery SP-frames can be used instead of I-frames in such applications as bitstream switching, splicing, random access, fast forward, fast backward, and error resilience/recovery. SP-frames require significantly fewer bits than I- frames to achieve similar quality. In some of the mentioned applications, SI-frames are used in conjunction with SP-frames. An SI-frame uses only spatial prediction as an I-frame and still reconstructs identically the corresponding SP-frame, which uses motion-compensated prediction.

5. Frame types in the existing standards I-, P-, and B-Frames I PPP PP I P … IB P IBB P … B

6. Applications for SP- and SI-Frames (1) Bitstream switching P 2, n-2 P 2, n-1 S 2, n P 2, n+1 P 2, n+2 S 12, n P 1, n-2 P 1, n-1 S 1, n P 1, n+1 P 1, n+2 Bitstream 1 Bitstream 2 Secondary SP-frame Primary SP-frame

7. Applications for SP- and SI-Frames (2) Splicing and Random Access P 2, n-2 P 2, n-1 S 2, n P 2, n+1 P 2, n+2 SI 2, n P 1, n-2 P 1, n-1 S 1, n P 1, n+1 P 1, n+2 Bitstream 1 Bitstream 2

8. Applications for SP- and SI-Frames (3) Error Recovery/Resiliency P 1, n-2 P 1, n-1 S 1, n P 1, n+1 S 21, n SI 1, n P 1, n-3

9. Applications for SP- and SI-Frames (4) Video Redundancy Coding Sync frame 12345678910111213 3456789101112131415 567891011121314151617

10. Decoding and Encoding Processes for SP- and SI-Frames Demultiplexing Inverse Quantization Inverse Transform Frame Memory MC Prediction Intra Predition Motion Information Intra Prediction Mode Generic block diagram of decoding process. + PQP

11. Generic block diagram of decoding process for secondary SP- and SI-frames Demultiplexing + Inverse Quantization Inverse Transform Frame Memory MC Prediction Intra Predition Transform Quantization SPQP Motion Information Intra Prediction Mode d rec l rec l pred l err lclc R P

12. Generic block diagram of decoding process for primary SP-frames Demultiplexing + Inverse Quantization Frame Memory MC Prediction Transform PQP Motion Information c rec c pred d err lclc R P Inverse Transform Inverse Quantization SPQPd rec l rec SPQP l err

13. Generic block diagram of encoding process for nonintra blocks in SP-frames Multiplexing Inverse Quantization Transform Motion Estimation Frame Memory + SPQP c orig Quantization c err d pred PQP Quantization Transform l pred P Motion Information R Inverse Quantization & Inverse Transform Quantization Inverse Quantization + SPQPl rec c pred d err PQP l err -

14. To generate secondary representation of the primary SP-frame l c Demultiplexing + Inverse Quantization Frame Memory MC Prediction Transform PQP Motion Information c rec c pred d err lclc R P Inverse Transform Inverse Quantization SPQPd rec l rec SPQP l err F 1,n => F 2,n l err,2 = l rec – l pred,2

15. Generic block diagram of encoding process for nonintra blocks in SP-frames Multiplexing Inverse Quantization Transform Motion Estimation Frame Memory + SPQP c orig Quantization c err d pred PQP Quantization Transform l pred P Motion Information R Inverse Quantization & Inverse Transform Quantization Inverse Quantization + SPQPl rec c pred d err PQP l err - l err,2 = l rec – l pred,2 l pred,2 l err,2 lclc

16. Generic block diagram of decoding process for secondary SP- and SI-frames Demultiplexing + Inverse Quantization Inverse Transform Frame Memory MC Prediction Intra Predition Transform Quantization SPQP Motion Information Intra Prediction Mode d rec l rec l pred l err lclc R P l pred,2 F 2,n l err,2 l err,2 + l pred,2 = (l rec – l pred,2 ) + l pred,2 = l rec l err,2 = l rec – l pred,2

17. Results – coding efficiencies

18. Results – coding efficiencies when inserted periodically

19. Results – Comparison with S-Frame (Drifting effect)

20. Results – Comparison with S-Frame (Size) The SP-frame is 3.4 times smaller than the S-frame in average. 6.2 times

21. Results – PSNR and Total bits over 100 frames (4 switches) SP or I Frame rate = 1fs

22. Results – PSNR and Total bits over 100 frames (no switches) SP or I Frame rate = 1fs

23. ~ The End ~ Thank you!!