Improved input-state linearization in video bitrate controllers Noam Korem

Presentation outline Video encoding and rate control Classic models Suggested improvement Simulation results Summary and conclusions Video encoding and rate control Classic R-Q models Suggested improvement Simulation results Summary / Conclusions

Generic video encoder* IP i-2i-1 P i P i+1 DCT VLC q i Iq i-2 IDCT DCT VLC q i-1 Iq i-1 IDCT Iq i IDCT DCT qiqi VLC DCT q i+1 VLC IntraPredicted Discrete Cosine Transform Video encoding and rate control Classic R-Q models Suggested improvement Simulation results Summary / Conclusions

Discrete Cosine Transform (DCT) Resembles Discrete Fourier Transform Purely real (real transform to real) Allows representation in the frequency domain, usually more compact Spatial domainFrequency domain

Quantization Quantization scale = 1Quantization scale = 31

Generic video encoder with rate control mechanism Frame encoder bits target bit rate Non-linear Bit rate controller qiqi SiSi Linear controller q i =Q(T i,..) TiTi Non linear bitrate controller (Input-state linearization)

The problem P i-1 P i Iq i-1 IDCT + + Iq i IDCT DCT qiqi VLC DCT q i+1 VLC Energy of difference frame i is dependent on the q(i-1) Coded information of frame i is dependent on q(i), q(i-1) Video encoding and rate control√ Classic R-Q models Suggested improvement Simulation results Summary / Conclusions

The problem, cont. Classic Rate-Quantization models do not depend on reference frame quantization scale

Improved R-Q model i-1 i-1(q) i Q i-1 Δ Qi-1 Δ RiRi QiQi Q i-1 Video encoding and rate control√ Classic R-Q models√ Suggested improvement Simulation results Summary / Conclusions

Previous frame quantizer dependency R i (Q i-1 ), Q i =8R i (Q i )

Improved R-Q model, cont.

Simulations Encode real videos, compare accuracy of traditional model (TM5) against improved model ffmpeg open-source video encoder with enhanced quantization and bit rate control used as encoding model (MPEG4). All test videos are YUV 4:2:0, QVGA (320x240), encoded at 370kbps (74CR) Video encoding and rate control√ Classic R-Q models√ Suggested improvement√ Simulation results Summary / Conclusions TM5 model:

Simulations Absolute prediction error is calculated for each frame, for both models Delta of absolute prediction errors is used as performance measurement

Simulation results Mean Abs Err* TM5: 7.18% TM5i: 4.8% (33% improve) Mean Abs Err* TM5: 17.07% TM5i: 15.03% (12% improve)

Simulation results, cont. Mean Abs Err* TM5: 22.3% TM5i: 19% (14.7% improve) Mean Abs Err* TM5: 16.7% TM5i: 14.1% (15.6% improve)

Simulation results, cont. Mean Abs Err* TM5: 34.2% TM5i: 32.2% (6% improve)

Simulation results, degradation Mean Abs Err* TM5: 13.1% TM5i: 13.2% (1% degrade)

Summary & Conclusions Video encoding with rate controller scheme presented (camcorders, live streaming) Video rate and distortion depend on quantization of reference frame, classic models ignore. Improved R-Q model allows more accurate input-state linearization, demands more calculations Improved R-Q model does not deliver when scene complexity changes rapidly Video encoding and rate control√ Classic R-Q models√ Suggested improvement√ Simulation results√ Summary / Conclusions

Future work Prediction quality degrades due to input noise – changes in scene complexity More accurate complexity estimation (more accurate prediction for X i ) based on:  frame content  motion estimation would improve R-Q model accuracy Try the model improvement on other R-Q models A complete model will include past quantization parameters R i =R i (Q i,Q i-1,Q i-2,…)

Related work Video Group, "Test Model 5" JTC1/SC29/WG11 Coding of Moving Pictures and associated Audio MPEG 1994, section 10 Tihao Chiang and Ya-Qin Zhang "A New Rate Control Scheme Using Quadratic Rate Distortion Model", IEEE International Conference on Image Processing, /ICIP Liang-Jin Lin; Ortega, A. “Bit-rate control using piecewise approximated rate-distortion characteristics” Circuits and Systems for Video Technology, IEEE Transactions on, Volume 8, Issue 4, Aug 1998 Page(s): / Saw, Y.-S.; Grant, P.M.; Hannah, J.M., "Rate-distortion analysis of nonlinear quantisers for MPEG videocoders: sigmoidal and unimodal quantiser control functions" Vision, Image and Signal Processing, IEE Proceedings- Volume 145, Issue 4, Aug 1998 Page(s):249 – 256 Ma, S.; Wen Gao; Yan Lu; "Rate-distortion analysis for H.264/AVC video coding and its application to rate control“ Circuits and Systems for Video Technology, IEEE Transactions on Volume 15, Issue 12, Dec Page(s): /TCSVT

Many thanks: Shai Mazor Orly Wigderson Kobi Kohai For the guidance, patience and assistance on all aspects of the project