Fine Granularity Scalability in MPEG-4 Video by Weiping Li Presentation by Warren Cheung

Fine Granularity Scalability ● Amendment to MPEG-4 ● Bitplane coding of the DCT coefficients

Motivation ● Video optimised for bit rate range 1. Limited Channel Capacity 2. Partial Decoding

Nonscalable Encoding ● Fixed bit rate ● Bandwidth efficiency depends on: 1. Channel bit rate 2. Encoded bit rate

Techniques in MPEG ● MPEG-2, MPEG-4 ● incorporate layered scalability techniques 1. SNR scalability 2. Temporal scalability 3. Spatial scalability

SNR scalability ● Signal-to-noise ratio (SNR) ● Encode two layers at same frame rate and spatial resolution ● Use different quantization accuracy

SNR scalability(2) ● Two layers: base layer, enhancement layer ● Base layer decodes DCT coefficients as before ● Can decode and add enhancement layer DCT coefficients ● Drift can occur

Drift ● Drift occurs in the base layer if ● Enhancement layer information used in motion prediction loop and ● Enhancement layer is not decoded by the decoder 1. Drift occurs in the enhancement layer if 1. Enhancement layer information not used in motion prediction loop and 2. Enhancement layer is not decoded

Temporal Scalability ● Two layers at different frame rates ● Base layer with only P-type prediction ● Enhancement layer adds 1. Extra P and B frames based on base layer 2. Extra P frames based on enhancement layer

Spatial Scalability ● Two layers same frame rate but different spatial resolution ● Base layer has lower resolution ● Enhancement layer upsamples the base layer and for prediction ● No drift occurs if enhancement layer is not decoded (but unavailable for prediction of base layer)

Performance ● Enhancement layer must be completely decoded, otherwise it is useless ● Jagged “stair” performance curve ● Want to be able to truncate the enhancement bitstream and still provide partial enhancement

FGS streaming

Bit-Plane Coding ● Two Layers: base and enhancement ● DCT coefficients normally encoded using run- level encoding ● Use Bit-Plance Coding for enhancement layer ● Reorganise the bitstream such that even if it is truncated, we get a usable set of DCT coefficients

Sending Numbers ● Given a set of numbers ● Which bits are the most significant?

Bit-Plane Coding Example ● Suppose we wish to send: ● 10,0,6, is the largest number and requires 4 bits so we have 4 bit-planes 1. 1, 0, 0, 0 (MSB) 2. 0, 0, 1, , 0, 1, , 0, 0, 0

Bit-Plane Coding of DCT Coefficient ● We apply Bit-Plane coding for each 8x8 block of DCT coefficients ● Bit-Planes are encoded using (RUN, EOP) symbols 1. RUN: Number of zeroes before next one 2. EOP: Is this the last one in the plane

Additional Details ● Each of the YUV color components may have a different number of bit-planes in the same frame 1. Y = luminance (black and white) 2. U, V = chrominance (color difference) 1. MSB (most significant bit plane) is the first plane that is not all zeroes

Advanced Features ● Frequency Weighting: Low frequency DCT coefficients are more important than high- frequency coefficients and should be transmitted first ● Shift the bits for DCT coefficients up/down 1. Parts of the picture may be more significant 1. Shift the bit-planes for these elements up

Additional Features(2) ● Error Resilience: ● Resynchronization markers ● bit-plane start code 1. Temporal Scalability 1. Add additional frames that can be decodedto improve bitrate 2. Predictions not allowed in the additional frames, but quantization can be scaled and bit-plane coding can also be used

References ● Scalability Extensions (brief overview) ● node133.html ● FGS and applications ● _scalability/

Questions and Comments Advantages/Disadvantages of FGS (and the other techniques) Advantages/Disadvantages of FGS (and the other techniques) Applications and Related Issues Applications and Related Issues Future Directions for improvement Future Directions for improvement Wavelet based methods Wavelet based methods

Discussion Topics ● Overhead ● Noticeable degradation of image quality ● Quantization is only a coarse way to manipulate the image quality (powers of two) ● Many people share similar connections ● Vary due to path through network, physical location ● Wireless connection ● Other people on the network ● Unreliable bandwidth

Discussion(2) ● New MPEG standard ● Soliciting solutions to scalability issues ● Future work may be based on wavelates ● No current mainstream video format currently uses wavelets ● Used in static image format JPEG-2000