1. Variable Bit Rate Video Coding April 18, 2002 (Compressed Video over Networks: Chapter 9)

2. Topics  Introduction  Variable rate compression of video  Delay constraints  Impact of transmission modes  Encoder rate constraints  Rate control algorithms  Conclusions

3. Introduction  VBR/CBR Coding modes (no. of bits per frame) Transport modes (no. of bits that can be transmitted during certain periods of time)  Video quality (end-to-end), distortion Depends on decisions at encoder (if lossy and online) Depends on transport (loss, bandwidth, delay) Tolerance depends on application

4. Variable rate compression  Variable rate nature of compressed video: Input video characteristics (scene changes, redundancy varies) Coding parameters (quantizers, motion compensation)  Constant quality requires variable bits per frame  Offline encoding (two-pass approach for DVD)  Distortion measurement (MSE)

6. Variable rate compression  Ways of creating variable rate video Quantizer selection (with each block/macroblock; complicates for video since RD of future frames depend on motion vectors) Frame type selection (I, P, B; I frames may be placed at fixed interval, eg. every 0.5 sec) Frame skipping (use more bits/frame but less frame/s; use interpolation at receiver)

7. Delay constraints  Prevent decoder buffer underflow R i = no. of bits assigned to frame i C i = no. of bits recd. by decoder buffer during ith frame interval T d = I corresponds to ith frame intervals having passed

9. Delay constraints  We assumed no frame skipping, and const. frame rate  Encoding and decoding delay constant.  Delays not considered: Transmitter buffer delay  tb Transmission channel delay  ch  tb +  ch should not violate decoder buffer underflow constraint  Constraints vary on Pre-encoded video Live video Interactive video

12. Impact of transmission modes  CBR vs VBR transmission Better video quality, shorter delay, increased call capacity  QoS, best-effort model of networks  Constrained vs. unconstrained transmission Eg. Transmission rate subject to conditions (peak rate/sustainable rate in QoS) or congestion control in TCP/IP within best-effort transmission  Feedback vs No Feedback Encoders can modulate data(quality/resolution/rate) based on feedback  Modes of operation: U-VBR / U-SVBR/ C-VBR / F-VBR

13. Encoder Rate constraints  Derive rate constraints that encoder/transmitter must meet Avoid violating delay constraints Use channel information Consider type of application (PEV/LIV/LNIV) Memory restrictions (PDA)

15.  To prevent decoder buffer underflow, encoder must avoid the encoder buffer size exceeding CBR transmission mode VBR with known channel rate VBR with unpredictable channel rates Encoder Rate constraints

16.  Video caching Proxy caching of web objects Efficient sharing of resources Reduce initial latency Less likelihood of packet losses Replicate entire sequence store prefix (for  t d seconds) Store intermediate samples so that decoder does not starve Coarse layers cached

17. Rate control algorithms  Constraints placed so as to guarantee that decoder always has data to decode  Video encoder should produce bitstream that meets these constraints  To accomplish this, we need rate control algorithms  H.263, MPEG-2 do not define operation of encoder, so free to use any rate control techniques

18. Rate control algorithms  Problem formulation Rate constraints for set of frames, not of individual frame Decision on how to allocate bits among the set, so that the average distortion is minimized – RD optimization  CBR algorithms Each coding parameter is associated with a rate and distortion pair. Find coding parameters within a finite set (for each frame) such that distortion is lowest First calculate RD at all points and then optimize Gets complicated when dependency exists among frames Allow change of frame rate (find optimum, no jerky motion, redundancy decreases among frames at low frame rate)

19. Rate control algorithms  VBR algorithms Many parameters (rate, delay, etc.) Algorithms monitor the long-term average transmission rate and keep it close to a value  Real-time adaptation to channel conditions Encoder should react to channel changes (channel memory is long, channel feedback is fast)  Layered/Scalable video Incorporated in MPEG-2 standard, high complexity.

20. Conclusions  VBR coding is the natural representation of video  Constraints need to be placed at encoder for best quality  Transmission medium and service plays a critical role  Optimization algorithms used to find ideal rate at encoder  Rate control gets even more challenging with wireless mediums