1. ENEE 631 Project Video Codec and Shot Segmentation
Aravind Sundaresan
Vikas Raykar
11/27/2018
ENEE 631 Project: Video Codec

2. Main features/ functionality
Can encode monochrome video with frame dimensions (16*M, 16*N).
Codes the sequence as a series of I/P frames. The I/P decision is made according to suitability of each method. (Example: when a scene change is detected, the subsequent frame is coded INTRA).
The frames are periodically coded as INTRA according to the INTRA refresh rate parameter.
Temporal prediction is closed loop. Performs full-pel Motion Estimation in a window of of dimensions 48 x 48. Macroblocks within an I-frame are coded as INTRA/ INTER according to the compression achieved.
Has a resynchronization marker at frame level.
11/27/2018
ENEE 631 Project: Video Codec

3. ENEE 631 Project: Video Codec
Video Codec Structure
The Video Codec is split into two programs Encoder and Decoder. Both of them have three layers.
Top Layer: Takes care of interface and I/O.
Performs the necessary Initializations.
Splits the input into frames and feeds them to the next layer sequentially.
11/27/2018
ENEE 631 Project: Video Codec

4. Video Encoder - Block diagram
Top Layer
Intermediate Layer
Input
Raw frame
Recon. frame
Bottom Layer
Bitstream
Residue frame
Recon. Residue frame
Encoder
Blocks
ME, MC
11/27/2018
ENEE 631 Project: Video Codec

5. ENEE 631 Project: Video Codec
Video Codec Structure
Intermediate Layer. This Layer performs the frame level manipulations and also takes care of the frame-level and macroblock-level decision making in the encoder.
Performs Motion Estimation and Compensation or removes 128 from the frame to get Residue Frame.
Feeds the residue frame to the frame encode layer.
The reconstructed residue frame is used to reconstruct the current frame for future prediction.
Bottom Layer: This layer performs the actual coding.
A hybrid coding technique, that employs both predictive coding to remove temporal redundancy and transform coding to remove spatial redundancy is used.
The frame is split into macroblocks and each macroblock is coded separately. The bits generated are put in the bitstream.
11/27/2018
ENEE 631 Project: Video Codec

6. Top Layer (Interface Layer)
Performs the necessary initializations (such as Loading Huffman tables).
Serves as interface between user and the actual encoder.
Input sequence is read and passed as frames to the lower layer.
Very first frame is forced to be INTRA. Subsequent frames are by default directed to be coded as INTER. The lower layer may dynamically decide to code such a frame AS INTRA according to various parameters (scene change / INTRA refresh).
11/27/2018
ENEE 631 Project: Video Codec

7. Intermediate Layer (Control Layer)
This is an important layer in that most of the decision making is performed here. These decisions are aimed at selecting the best coding technique according to the input frame. The decisions made and the functions performed are listed below.
Intra / Inter Decision. The top layer has the ability to force the Intra Option. The Intermediate layer has the option to change the INTER option to INTRA. If the number of consecutive frames coded as P-frames equals a certain parameter (INTRA REFRESH RATE), the next frame is coded as an I-frame.
11/27/2018
ENEE 631 Project: Video Codec

8. Intermediate Layer (Control Layer)
Motion Estimation and Compensation. In case of Intra Macroblocks, 128 is subtracted from the Macroblock. Based on the output of the Motion Estimation a decision is made whether to code the frame as INTER.
The frame to be coded is split into a MC frame and Residue frame (in case of INTRA frame, the MC frame comprises of pixels with value 128).
The Residue frame is passed to the Encode frame layer. The layer returns the reconstructed residue frame which is added to the MC frame to be used for future prediction. (Closed Loop prediction to avoid error accumulation)
11/27/2018
ENEE 631 Project: Video Codec

9. Bottom Layer (Encoder Layer)
The frame is split into Macroblocks each of which comprises of 4 blocks. The macroblocks are read in raster scan order and coded sequentially and the blocks in the Macroblock are also similarly coded. Each macroblock consists of a header followed by the coefficient data. The header contains
Coded Information (Coded/ Not coded, INTRA/ INTER, etc)
Motion Vector (for INTRA MBs)
Coded Block Pattern
Optionally include the Quantizer (or differential Quantizer).
11/27/2018
ENEE 631 Project: Video Codec

10. Bottom Layer (Encoder Layer)
The coding Procedure is described below.
Each block is transform coded using the DCT. The DCT coefficients are quantized. The Quantization tables for INTRA and INTER blocks are different.
Resulting matrix is split into DC and AC coefficients which are scanned in a zigzag manner and coded using fixed Huffman tables and run-length coding techniques.
For run-length values not found in the table the run-length and level values are coded using an ESCAPE code and fixed length codes. If none of the blocks contain any coefficients, the MB is 'not coded'.
If only some of the blocks are coded, the corresponding bit is set in the CBP.
11/27/2018
ENEE 631 Project: Video Codec

11. Video Decoder - Block diagram
Top Layer
Intermediate Layer
Output
Ref. frame
Cur. Recon. frame
Bottom Layer
Bitstream
Recon. Residue frame
Decoder
Blocks MC
11/27/2018
ENEE 631 Project: Video Codec

12. ENEE 631 Project: Video Codec
Decoded Frames
11/27/2018
ENEE 631 Project: Video Codec

13. ENEE 631 Project: Video Codec
Results
30 frames encoded
Size of compressed stream = bytes
Size of 30 frames: = bytes
Compression = 17%
Greater Compression can be achieved by
Increasing quantizer step
Increase ratio of P frames to I frames (current ratio 10:1)
11/27/2018
ENEE 631 Project: Video Codec