Fast motion estimation and mode decision for H.264 video coding in packet loss environment Li Liu, Xinhua Zhuang Computer Science Department, University of Missouri-Columbia, USA IEEE ICME(International Conference on Multimedia & Expo) 2008 June 2008 Presenter : SeungDae Jeong

IntroductionIntroduction  Packet loss environment Practical video communications Videos are usually coded without considering varying channel conditions Error propagation 2

IntroductionIntroduction  “Intra refresh” algorithm Using intra frames that doesn't refer to previous frames  The decision of intra mode Rate distortion optimization Quantization distortion Channel distortion  Simulation based estimation  Model based estimation 3 Previous frameInter frame with packet loss Intra frame

ContentsContents  RD Optimized intra fresh for error resilience Lagrange Cost Function in Packet Loss Environment Model Based Channel Distortion Estimation  Use channel distortion map for fast mode decision and motion estimation  Channel distortion map based fast mode decision and motion estimation algorithm Fast Mode Decision Fast Motion Estimation  Experiment results  Conclusion 4

Lagrange Cost Function in Packet Loss Environment  Set of all defined coding options for macroblock m in frame n 5 The source distortion by quantization The result bit rate The relation between distortion and bit rate

Lagrange Cost Function in Packet Loss Environment  In packet loss environment 6 The quantization distortion The error concealment distortion when a macrobloak is lost The error propagation distortion if reference frames in motion estimation are erroneous The estimated channel packet loss rate

Lagrange Cost Function in Packet Loss Environment  The modified Lagrange cost function for RD optimization mode decision  The rate distortion cost for intra mode 7

Model Based Channel Distortion Estimation  Define a channel distortion map on 4x4 block basis To track potential error propagation 8

Model Based Channel Distortion Estimation  Block based channel distortion map 9 The optimal mode decision The error propagation distortion of ith block The error concealment distortion The estimated error propagation distortion

Use channel distortion map for fast mode decision and motion estimation The error propagation distortion tends to surpass the quantization distortion as frames contain more motions (e.g., from frame 150 to 250) loss rate increases (e.g., from 0.05 to 0.10) 10

Use channel distortion map for fast mode decision and motion estimation  Statistical information from coded video sequence using original MBDE algorithm 11

Channel distortion map based fast mode decision and motion estimation algorithm  Fast mode decision Computational cost of estimating D ep (o) is quite small By using D ep (o) as assistance to decide early termination of possible inter coding choices, encoding speed can be accelerated significantly  Fast Motion Estimation Choosing the best motion vector Minimizing a Lagrange cost function 12

Fast mode decision  Computational cost Inter coding cost > Intra coding cost  The calculation of minD ep Dependent on optimal motion vector decision from motion estimation process Computational cost can not be reduced  Computing D ep (o) based on the predicted motion vector 13

Fast mode decision  Computing D ep (o) based on the predicted motion vector 14

Fast motion estimation  Minimizing a Lagrange cost function Fast motion estimation algorithm selects best motion vector from a set of candidates, and terminates those macroblocks that may have large error propagation distortion associated 15 The sum of absolute difference The number of bits to code motion information

Fast motion estimation 16

Experiment results  Implemented using H.264 reference software JM  Video sequences Carphone, Foreman, Coastguard QCIF format, 7.5 fps Fixed quantization parameter (QP)  Packet loss simulation S. Wenger, “Internet Error Patterns” 17

Experiment results  PSNR Peak Signal-to-Noise Ratio  Bit rate  Speed Very fast 18

ConclusionConclusion  Two stage fast mode decision and motion estimation algorithm for H.264 video coding in packet loss environment  Advantage of estimated channel distortion as assistance to decide early termination of possible inter coding options  Reduce coding complexity significantly while maintaining similar average picture quality 19