Flow Control in Compressed Video Communications #2 Multimedia Systems and Standards S2 IF ITTelkom

Agenda 1.Improved Quality Rate Control Using ROI Coding 2.Rate Control Using Prioritized Information Drop 3.Rate Control using Internal Feedback Loop 4.Reduced Resolution Rate Control 5.Rate Control Using Multi-layer Coding 2

1. ROI Coding In some case: a priori knowledge about the content coding the regions of interest (ROI) more accurately than the rest of the video content  improving coding efficiency Quest: identify the ROI ISO MPEG-4 3

Example: head-and-shoulder TMN5 (to be presented) 14.4 kbit/s: conventional variable-Qp TMN5 ROI coding for enhanced-face rate control 4

Rate Control Initial: face region: a smaller quantization step size background: a coarser quantization parameter During encoding process: generated bits < the target number of bits then Qp (Quantization parameter) will decrease generated bits > the target bit rate can handle then Qp will increase 5

For 150 frames sample 6

2. Prioritized Information Drop After encoded, video codewords are sent to a local buffer before transmission In the case of network congestion: the delayed video codewords are arbitrarily dropped off the local buffer if their buffering time exceeds a certain pre-defined threshold Q: which data should be dropped? 7

Random drop MVs are more sensitive to errors than AC coefficients H.263 at 55 kbit/s: error-free 5% drop rate of MV and AC coefficients of P- frames 8

5% drop on… dropping AC coefficients would have a minor effect on the reconstructed quality as compared to motion data …the sub-stream of DCT coefficients of a P-frame …the MV sub-stream 9

Levels of priority Some levels of priority could be assigned to various types of video parameters based on their sensitivity to loss and their contribution to overall video quality a dynamic process  takes into account both the sensitivity of video parameters and the reported channel conditions 10

3. Internal Feedback Loop Previous: an open loop structure the locally reconstructed picture memory is not updated leads to the accumulative damage of the decoded video frames Feedback loop in the encoder: to stop the accumulation of errors reduce their effects on the perceptual quality 11

feedback-controlled H

Example 20 per cent MV drop with feedback 13

4. Reduced Resolution Rate Control where the bit rate budget is extremely low, the quantization step size cannot be increased beyond an upper bound the variable Qp rate control techniques could be used in conjunction with the reduced resolution scheme 14

Reduced resolution technique down-sampling each MB in the prediction error before it is encoded up-sampling the reduced-resolution reconstructed block at the decoder in order to produce the motion compensated picture 15

bilinear pixel interpolation Motion estimation is performed on an MB basis Prediction error image is reduced by half in each dimension 16

Encoding Each down-sampled 8x8 luminance block is transformed using an 8x8DCT 4x4 chrominance blocks are transformed using a 4x4 DCT. The transformed coefficients are quantized, raster-scanned in a zigzag order and then run-length encoded 17

Decoding the transformed coefficients are recovered and inverse-transformed to reconstruct the 8x8 luminance blocks Each reconstructed 8x8 block is then up- sampled to produce the 16x16 MB using the bilinear filter the up-sampled MB is added to the motion compensated MB to obtain the final reconstructed MB 18

Up-sampling 19

H.263 decoder with reduced resolution 20

Example (40 kbit/s and 10 f/s) 21

5. Multi-layer Coding forming the output stream from a number of layers of different bit rates, frame rates and possibly resolutions to achieve a scalable output video layers = one base layer (BL) + a number of enhancement layers (EL) 22

BL & EL BL: essential for the reconstruction of the video sequence EL: help improve the perceptual quality their absence causes a graceful deterioration of the received video quality. Could be used as a trade-off between quality and compression efficiency 23

Scheme of the multilayer video coder RF = Reference frame LRF = Last encoded reference frame CF = Current frame LPF = Low pass filter DS = Down-sampling JPEG E = JPEG encoder, low resolution JPEG L = JPEG encoder medium resolution JPEG H = JPEG encoder high resolution JPEG D = JPEG decoder INT = Interpolation A simple multi-layer digital video coder for multimedia network applications Grattarola, A.; Iscra, A.; Zappatore, S.; DIST, Genoa International Conference on Image Analysis and Processing,

L1L1 + L2 + L3 L1 + L2 + L3 + L4 + L5 25