Real time H.264 decoder implementation in robot control Saurabh Ghorpade

H.264 decoder [1] algorithms for evaluating the inverse transforms and packet analysis high speed real time implementation in C/C++ for robot control feature recognition to be implemented

Existing functionality Java implementation- successful in terms of functionality but slow speed. Reasons[2]: -Class loading at startup -Unnecessary array validations -Excessive use of heap -Garbage collection

C for real time control No Java Virtual Machine so direct interaction with hardware Better memory management using pointers Compiler optimizations, macros can be used. Easy and fast control system, navigation and socket.

Environment setup Robot communication protocol [3], router. Operating system: Linux C Compiler.

Implementation Multithreaded socket programming -To send commands to robot -To decode the video packet

Image decoding Image divided into groups of blocks (GOB) and each GOB is further divided into macroblocks Each macroblock contains 16x16 image in Y CbCr format, type 4:2:0

Steps Entropy decoding Inverse zigzag, followed by inverse quantization and inverse transformation. Forming the picture by the picture format given in fig 6. Extracting the motion vector information from the packet followed by motion compensation. Finally getting the video.

(Fig 1) Client server [3]

(Fig 2) packet [3]

(Fig 3) Modified JPEG [3]

H.264 Encoder (a) and decoder (b) [15]

(Fig 4) Motion vector estimation [3]

(Fig 5) 16x16 intra prediction modes [3]

(Fig 6) Picture format [3]

(Fig 7) Hardware [3]

Hardware description [3] A.R. Drone: Quadrotor. Batteries: The AR.Drone uses a charged 1Ah, 11.1V LiPo batteries to fly Motion sensors ultrasound telemeter camera aiming downwards Video streaming: The frontal camera is a CMOS sensor with a 90 degrees angle lens.

References [1] F. Pescador, M.J.Garrido, C.Sanz, E.Juarez, M.C.Rodriguez and D.Samper, “A real-time H.264 MP decoder based on a DM642 DSP”, 14th IEEE International Conference on Electronics, Circuits and Systems, Madrid, Spain, ICECS 2007, Vol. 11, pp.1248 – 1251, Dec ICECS 2007, [2] Java is slower than C: [3] A.R. Drone Developer’s guide: one.org%252Fattachments%252Fdownload%252F365%252FARDrone_SDK_1_7_D eveloper_Guide.pdf’s guide: one.org%252Fattachments%252Fdownload%252F365%252FARDrone_SDK_1_7_D eveloper_Guide.pdf [4]A.R.Drone [5] Y-C b -C r format: [6] H.264 Reference : [7] W.T.Staehler and A.A.Susin, “Real-time 4x4 intraframe prediction architecture for a H.264 decoder.” UFRGS, Alegre Telecommunications Symposium, 2006 International, pp. 416 – 421, Sept Telecommunications Symposium, 2006 International

References [8]Wifi wiki : [9] make wiki : [10] Introduction to make (linux make for building C/C++ sources) [11] JPEG wiki: [12] Shih-Tse Wei, Chia-Wei Tien, Bin-Da Liu and Jar-Ferr Yang, “Adaptive Truncation Algorithm for Hadamard-Transformed H.264/AVC Lossless Video Coding”. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 21, pp , May IEEE Transactions on [13] Runlength coding wiki: length_encodinghttp://en.wikipedia.org/wiki/Run- length_encoding [14] Huffman coding wiki: [15] Soon-kak Kwon *, A. Tamhankar and K.R. Rao, “ Emerging H.264/AVC video coding standard”, J. Visual Communication and Image Representation, vol. 17, pp , April H.264 review review

Thank you! Any questions?