1. Statistical Multiplexer of VBR video streams 31.12.2002 By Ofer Hadar Statistical Multiplexer of VBR video streams 31.12.2002 By Ofer Hadar

2. Outline of the talk Introduction and motivation for Statistical Multiplexing VBR vs. CBR coding Statistical multiplexing of video streams Joint rate control Current Piecewise CBR multiplexer Future true VBR multiplexer Conclusions

3. Case in Point: Movies Huge bandwidth consumer Without compression: Unthinkable (up to 1 Gbps for HDTV stream) With compression: highly bursty temporal and spatial information: frames vary in sizes, scenes vary in bandwidth rate Burstiness complicates server & network provisioning

4. MPEG Compression Fixed GoP pattern such as IBBPBBP IBBPBBP I frames are JPEG encoded B and P frames exploit temporal redundancy I B B P B B P B B I

5. Burstiness One simple definition: Ratio between peak and average bit rate (PAR)

6. What to do ? (1) Conservative rich: reserve peak required bandwidth. Wasteful, costly: ratio of peak to average rate may be over 5:1. Conservative poor: Compress to specified bandwidth

7. Peak Bandwidth Reservation 020040060080010001200 0.8 1 1.2 1.4 1.6 1.8 2 2.2 x 10 4 Reserved peak- rate Wasteful of Bandwidth

8. CBR Video Encoding b = target encoding rate q(n) is a function of buffer contents quantization becomes very coarse as buffer contents approaches B Why is it always blurry exactly when it starts getting interesting ? Encoder Buffer Digitized video B q(n) = quantization scale XnXn Storage b/F bits per frame

9. What to do ? (2) Practical solution: trade bandwidth for memory. With large storage space: trickle the stream, playback from memory need huge space how about live broadcast?

10. What to do ? (3) Using Statistical Multiplexing l Multiple channels fill a CBR pipe l They fill it completely l Each component is VBR or Piecewise CBR l Not really statistical multiplexing, but dynamic, centrally controlled bitrate time What is Statistical Multiplexing ?

11. Properties of Video Communication Applications  Constant bit rate (CBR) or variable bit rate (VBR) coding  Constant bit rate leads to variable quality  Variable bit rate can enable constant quality  Example of CBR: Digital TV, videoconferencing over ISDN Example of VBR: DVD, StatMux

12. Basic Video Coding Question: VBR vs. CBR coding  Question: How many bits should we allocate to code each frame?

13. How to Allocation Bits Among Frames ?  Digitized (uncompressed) video has a constant rate  Question: Compress at a constant bit rate? Variable rate?  Observations: Some frames are more complex than others, or are less predictable than others, and therefore require more bits E.g. to achieve constant quality for every frame, a high complexity frame would require more bits than a low complexity frame

14. Program complexity and bits 1 2 3

15. VBR vs. CBR Coding

16. Ordinary fixed rate encoding is insufficient

17. Statistical multiplexing into CBR channel Stat Mux Channel buffer RcRc MPEG-2 Enc. 1 Bitstream, R1 Statistics Source 1 MPEG-2 Enc. 3 Bitstream, R3 Statistics MPEG-2 Enc. 2 Bitstream, R2 Statistics...... MPEG-2 Enc. 3 Bitstream, Rn Statistics Joint rate control Bit rates … Statistics Source 2 Source 3 Source n

18. Main steps of a Statistical Multiplexer Joint rate control algorithm is based on the feedback concept Statistics are produced by the encoders along with the compressed bitstream. These statistics are continuously fed into the joint rate controller from each encoder after compression of a picture. These coding statistics, together with the information on channel buffer fullness, are used to dynamically compute the bit rate allocation for the individual encoders. The bit rate of program i is obtained according to the complexity of that program and the sum of the complexities of all programs :

19. Considerations Decoder buffers must not overflow or underflow. How to distribute the total bitrate Channels are not synchronous - turn down before up.

20. Joint rate control Sum of elementary stream rates may not exceed allowed total Distortion measured as a weighted average over the channels should be minimized Distortion measure: average value of quantizer_scale over a frame.

21. Joint rate control (Cont.) One simple assumption: q(i)/k(i)=q(j)/k(j), all i,j Assumption leads to both TM5 and StatMux solution: r(j) = R c (X(j)/k(j) /  X(i)/k(i)) where X(i) = b(i)*q(i)

22. Description of the communication commands between each encoder and the StatMux ack Command (R new_i ) Response (R i, Q i ) query StatMuxEncoder i

23. Program bit-rate for encoding the first set of video sources at a channel bit rate of 16 Mb/s Frame number 0.1 s 4 Mbps Mbit/s Enc. 1 Enc. 2 Enc. 3 Enc. 4 Total bit-rate requirements CBR channel bandwidth 16 Mbps Frame number Summation 

24. Full VBR multiplexer design The expectations are that variable bit-rate (VBR) encoding with joint bit-allocation will give some gain in quality and bandwidth efficiency over the previous scheme of piecewise constant bit-rate (CBR) encoding. The global control system controls the bit rate on a global level, i.e. for each picture it computes a global target using the concept of picture complexity.

25. Multiplexing and Intelligent rate control In a stat-mux system, we wish to distribute the channel bandwidth among the program such that The bit rate allocation can be achieved by two schemes: Changing the quantization factor (Q i ). The StatMux sends a new vector of Q for all the encoders. changing the video resolution i.e., reducing the picture resolution of some streams in the case when the aggregated bit rate is above the channel bit rate, and increasing the resolution when the aggregated bit rate is bellow the channel bit-rate

26. Criterions for StatMux What is better ? To allocate bits so that the picture quality is the same for all programs. To allocate bits so that the picture quality of each program is the same as function of time.

27. Evaluation the PSNR statistics both as function of time and as function of the stream index i. PSNR_1 Histogram_1 PSNR_1 Histogram_T 1 Time PSNR_1 Time PSNR_2 Histogram Time PSNR_N PSNR_2 Histogram_2 Histogram_1 Histogram PSNR_n Histogram_T n

28. Conclusions Statistical Multiplexer improves video quality. Statistical Multiplexer improves data efficiency Overall Bandwidth Wasted. The most important feature of VBR coding is its ability to achieve similar or better video coding quality over random complexity video content while using fewer bits compared to CBR coding. This amount of bit saving is a very significant gain for any digital communication system.