Decision Trees for Error Concealment in Video Decoding Song Cen and Pamela C. Cosman, Senior Member, IEEE IEEE TRANSACTION ON MULTIMEDIA, VOL. 5, NO. 1, MARCH 2003

Outline Introduction Error Concealment Methods  Panning  Top/botMV Classification tree design  CART algorithm Experiments and Results conclusions

Introduction(1/2) Error control or concealment  Forward error correction added at the encoder  Post-processing methods employed by the decoder Three main approaches for error concealment  Frequency ： using the corresponding DCT coefficient of neighboring blocks (low frequency)  Spatial ： bilinear interpolation from the nearest MBs (computational complexity is large)  Temporal ： searching for a block from other frames

Introduction(2/2) Hybrid methods ： combine more than one of the three approachs  Ex ： In temporal concealment, the referenced block can be improved by spatial smoothing at its edges (additional complexity) Adaptive methods ： using different approachs in different situations  Ex ： Temporl concealment is used for most blocks while spatial concealment is used when scene changes (additional complexity)

Error Concealment Methods(1/3) spatial ： interpolate linearly from boundary pixels in top/bottom MBs frequency ： weighted average of first 9 DCT coefficients of top/bottom MBs panning ： use the camera panning motion vector top/botMV ： use top MV for top 8*16 sub- MB,use bottom MV for bottom 8*16 sub-MB

Error Concealment Methods(2/3) averageMV ： use the average motion vectors of top and bottom MBs useonlyMV ： top or bottom MB is Intra- coded =>use the only MV available spat+onlyMV ： use only available MV for nearest half, spatial interpolation for rest copyPmb ： copy co-sited MB from previous P frame if it’s Intra-coded or has MV=0

Error Concealment Methods(3/3)

Classification tree design(1/4) A classification tree was built at the encoder. It was then transmitted to the receiver. The true class of every MB is defined to be the best concealment method in table I, which provides the minimum MSE. The decoder will classify each lost MB using the decision trees sample

Classification tree design(2/4) The classification parameters are measurements which describe the spatial, temporal, and frequency domain context of a MB. They must be parameters which are available to the decoder even if the MB is lost. The parameters include both ordinal and categorical variables.

Classification tree design(3/4) CART algorithm was used to design the tree at the encoder. x ： the vector of measurements associated with the missing MB. C ： 1,2,…8 be the set of eight EC methods d(x) ： assigns to every vector x a class j from C L ： a training sequence consists of data (x 1,j 1 ), (x 2,j 2 ),…,(x n,j n ) The root node of the tree contains all the N training cases Each split depends on the value of only a single variable.

Classification tree design(4/4) For each variale, we find the split which provides the greatest decrease in node impurity. (using the Gini index to measure the purity of a set of data) Comparing all of these,and find the best overall split of the data. Assigning an EC method to every terminal node.

Experiments and Results(1/5) For any node, 1 bit to indicate which type of this node. For a internal node, 5 bits to specify which variable to split on, and 7 bits to specify the splitting threshold. For a terminal node, 2~3 bits to specify which EC methods to use.

Experiments and Results(2/5) If the tree grows large enough, eventually the classification will be perfect => “omniscient minimum” MSE “maximum” MSE is the MSE that results from using a single fixed and best method from table I. “relative MSE” is the MSE compared to that of the best of the fixed concealment methods. (best fixed = 1)

Experiments and Results(3/5) panning spatial copyPmb 0.08%

Experiments and Results(4/5) 0.45%

Experiments and Results(5/5)

Conclusions Two new temporal EC methods  Panning  Top/botMV The use of a decision tree provided lower distortion than any fixed method alone. The memory and computational requirements are quite asymmetric