1. UNDER THE GUIDANCE DR. K. R. RAO SUBMITTED BY SHAHEER AHMED ID : 1000836047 Encoding H.264 by Thread Level Parallelism

2. Objective In this project the encoding time of a baseline profile H.264 will be reduced using Thread Level Parallelism.( Parallel Programming)

3. About H.264 H.264[1] is an industry standard for video compression, the process of converting digital video into a format that takes up less capacity when it is stored or transmitted. The H.264/AVC standard was first published in 2003. It builds on the concepts of earlier standards such as MPEG-2 and MPEG-4 Visual and offers the potential for better compression efficiency (i.e. better-quality compressed video) and greater flexibility in compressing, transmitting and storing video [1]

4. About H.264 (contd) The H.264 baseline encoder is estimated to be 5x to 10x more complex than the H.263 encoder [2] while the decoder is 2x to 2.5x more than H.263 baseline decoder H.264 comprises of the following profiles 1. Baseline 2. Extended 3. Main 4. High

5. About H.264(contd.) -Baseline Profile is used for real- time conversational services such as video conferencing and video calling. [4] [11] -Extended profile is for the multimedia services over the internet. [4] [11] -Main profile is designed for digital storage media and television broadcasting. [4] [11] -High profile is used in the fidelity range extensions for applications such as content contribution, content distribution and studio editing. [5] The H.264 comprises of both encoder and decoder i.e. it is a codec. The encoder compresses the video into.264 formats and the decoder converts its back to the uncompressed format. This decoder is lossy.

6. H.264 Profiles H.264 Profiles [4]

7. H.264 Encoder [6]

8. H.264 Decoder [6]

9. Prediction Modes Prediction exploits the spatial or the temporal redundancies of a video sequence so that only the difference between actual and predicted image is encoded instead of encoding the entire image. There are two types of prediction Intra Prediction Inter Prediction -Intra Prediction is done on the Intra frame. Since there exists a high similarity among neighboring blocks in a video frame, a block can be predicted and reconstructed using the neighboring pixels [7] -Inter Prediction is done on the Predictive frame and Bi- directional frame. Since the frame rate of a video sequence is 30 to 60 fps (Nominal), the probability of having two successive frames to be similar is very high. The goal of inter prediction is to utilize this temporal redundancy and reduce the data to be encoded. [7]

10. Need for Parallel Programming to implement H.264 Gordon Moore [8] had stated that the transistors available on a semiconductor would double approximately every 18 to 24 months. Over the years the CPU frequency has tended to follow Moore’s law but since past few years the CPU clock speeds have not increased much. This has happened because heat dissipation has increased as well.[8]

11. Need for Parallel Programming to implement H.264 (Contd) Thus in order to make the most efficient use of the processor resources, computer architects have developed ways to implement many instructions in parallel during the same clock cycle

12. Serial Programming

13. Parallel processing

14. Definitions A macro block is an image compression component. It consists of four block of luminance and the spatially corresponding two blocks of chrominance. A slice is a spatially distinct region of a frame that is encoded separately from any other region in the same frame.

15. Data Domain Decomposition H.264 encoder treats a video sequence as many group of pictures (GOP). Each GOP includes a number of frames. Each frame is divided into slices. Each slice is an encoding unit and is independent of other slices in the same frame. The slice can be further decomposed into a Macro Block (MB), which is the unit of motion estimation and entropy coding. All these are places to parallelize the encoder

16. Data Domain Decomposition

17. Frame Level Decomposition

18. Slice Level Decomposition Partitioning of a frame to multiple independent slices enables parallel processing of slices. However, slicing the image and compressing each slice independently reduce the amount of spatial redundancy that can be exploited. Thus more slices =>higher the bit rate of the compressed video, for a fixed desired quality If the bit rate is kept fixed and the degradation of the compressed video quality is also fixed then the number of slices will be reduced and hence the number of threads are reduced.

19. Slice Level Decomposition (Contd.)

20. On applying parallel programming using Open MP Encoding Time : 3.554 sec Y- PSNR : 37.131dB U – PSNR : 41.335dB V- PSNR : 43.063dB PSNR : 38.398dB Bit Rate : 333.04 kbits/sec After applying parallel programming Encoding Time : 3.376 sec Y- PSNR : 37.131dB U – PSNR : 41.335dB V- PSNR : 43.063dB PSNR : 38.398 dB Bit Rate : 333.04 kbits/sec Foreman.qcif for 3 frames

21. On applying parallel programming using Open MP Encoding Time : 3.037 sec Y- PSNR : 37.87dB U – PSNR : 37.864dB V- PSNR : 10.633dB PSNR : 34.46dB Bit Rate : 334.24 kbits/sec After applying parallel programming Encoding Time : 2.906 sec Y- PSNR : 37.87dB U – PSNR : 37.864dB V- PSNR : 10.633dB PSNR : 34.46dB Bit Rate : 334.24 kbits/sec news.qcif for 3 frames

22. Conclusion Parallel programming will lead to better encoding times. The performance boost of on these test samples for more number of frames will be carried out. Parameters for judging the result will be 1.Encoding time 2. Peak Signal to Noise Ratio

23. REFERENCES [1] Iain Richardson, White Paper: “An overview of H.264 Advanced Video Coding”, Vcodex, 2007- 2011 [2] V. Iverson, J. McVeigh, and B. Reese, “Real-time H.264/AVC codec on Intel architectures”, IEEE Int, Conf. Image Processing, vol. 2, pp 757 -760, Oct 2004. [3] M. Horowitz, A. Joch, F. Kossentini, and A. Hallapuro, “H.264/AVC baseline profile decoder complexity analysis”, IEEE Transactions on Circuits and Systems for Video Technology, vol 13 issue 7, pp 704-716,July 2003 [4]I. E. G. Richardson, “H.264 and MPEG-4 video compression: video coding for next generation multimedia”, Wiley 2nd edition, Aug. 2010. [5]G. Sullivan, P. Topiwala, and A. Luthra, “The H.264/AVC advanced video coding standard: overview and introduction to the fidelity range extensions”, SPIE Conference on Applications of Digital Image Processing XXVII, vol. 5558, pp. 53-74, 2004. [6] T.Bhatia, “Optimization for H.264 high profile decoder for Pentium 4 processors”, M.S. Thesis, EE Dept., University of Texas at Arlington, Dec. 2005.

24. REFERENCES [7] Jian- Wen Chen, Chao- Yang Kao, and Youn- Long Lin, "Introduction to H.264 Advanced Video Coding", Department of Computer Science, National Tsing Hua University. [8] S. Akhter and J. Roberts, "Multi Core Programming, Increasing Performance through Software Multi- Threading", Intel Press, April 2006 [10] G. Amit and A. Pinhas,"Real Time H.264 Encoding by Thread Level Parallelism: Gain and Pitfalls", Corporate Technology Group, Intel Corp [11]T. Saxena, “Reducing the encoding time of H.264 baseline profile using parallel programming techniques”, M.S., Thesis EE, UTA, Dec. 2012. [12] A. Rodrigues, N. Roma, and L. Sousa, “p264 Open Platform for Designing Parallel H.264/AVCee Video Encoder on Multi Core Systems”, Dept. of Electrical and Computer Engineering, Technical University of Lisbon Websites Referred: 1. Open Software of Parallel Design : http://sips.inesc-id.pt/~nfvr/prototypes/p264/ 2. EE 5359 Projects : http://www-ee.uta.edu/Dip/Courses/EE5359/index.html 3. http://www.wikipedia.org/ 4. Optimizing Video Coding: http://www.drdobbs.com/parallel/optimizing-video-encoding-using- threads/225600370