1. Application of OpenMP in complexity reduction of inter frame coding in HEVC
Karthik Suresh
Department of Electrical Engineering
University of Texas at Arlington
Advisor: Dr. K.R.Rao

2. Outline Introduction to video coding and HEVC Inter prediction in HEVC
Proposed solution
Parallel Programming
Experimental conditions and results
Conclusion
Future work
References
Acronyms

3. Introduction to video coding and HEVC
The problem of memory space and the necessity of video coding
HEVC [1] is the latest standard in video coding developed by JCT-VC
HEVC has more parallel processing options and 50% bitrate reduction over H.264.
Supports Ultra HD, 3D video

4. HEVC Encoder

5. HEVC inter prediction

6. HEVC inter prediction

7. HEVC inter prediction Motion prediction is done using reference frames
Motion parameters consist of motion vector, reference picture index

8. HEVC inter prediction
Search is done in certain patterns: Square search and Diamond search pattern
Small diamond pattern
Large diamond pattern

9. Proposed solution Steps Start motion estimation
Get predicted motion block and start diamond search
if iDist == 1, calculate SAD for immediate neighboring blocks at distance = 1
Now for conditions iDist <= 8 and iDist > 8, iDist keeps increasing two fold and SAD is calculated for neighboring blocks at these distances until we get the best SAD. This is done in parallel

10. Parallel programming Concept of serial and parallel computing
Need for parallel computing
Ways to achieve parallel processing; Using dedicated hardware, GPU etc. or software like OpenMP [40], CUDA
Threads are the smallest sequence of program that can be managed independently

11. Parallel programming

12. Parallel programming
Introduction to OpenMP API

13. Parallel programming OpenMP directives [15]:
parallel pragma – used to convert a serial part of the code to parallel
barrier pragma – used to make a thread wait for other threads
for pragma – used to run ‘for’ loops in parallel
sections pragma – used to divide certain code sections and run them in parallel

14. Test conditions
Software: HEVC reference software HM 13.0 [12] running on Windows 7 64 bit operating system
Hardware: Intel core i7 running at 2.3 GHz with 8 GB memory
2 cores, 4 threads
Profile: random access profile
Quantization parameters: 22, 27, 32, 37

15. Standard test sequences [16] used
No.
Sequence
Resolution
Type
No. of frames
Frame rate (Hz) 1.
RaceHorses_416x240_30.yuv
416x240
WQVGA 50 30 2.
BasketballDrillText_832x480_50.yuv
832x480
WVGA 3.
BQMall_832x480_60.yuv 60 4.
KristenAndSara_1280x720_60.yuv
1280x720 SD 5.
Kimono1_1920x1080_24.yuv
1920x1080
WQHD 24

16. Standard test sequences used

17. Standard test sequences used

18. Results 1. Encoding time

19. Results 1. Encoding time

20. Results 2. Percentage reduction in encoding time

21. Results 2. Percentage reduction in encoding time

22. Results 3. BD-PSNR

23. Results 3. BD-PSNR

24. Results 4. BD-bitrate

25. Results 4. BD-bitrate

26. Results 5. PSNR vs bitrate

27. Results 5. PSNR vs bitrate

28. Conclusions
The proposed method of using OpenMP has resulted in reducing the encoding time by 10-25% over the original source code.
PSNR change is negligible showing minimal loss in quality
Bitrate increase is negligible

29. Future work Use of several other ways to achieve parallel processing
- CUDA
- GPU
- Custom hardware
- P-threads
Implement parallel processing in more functions and handle inter-dependencies

30. References
[1] G.J. Sullivan et al, “Overview of the high efficiency video coding (HEVC) standard”, IEEE Trans. CSVT, vol. 22, pp , Dec.2012.
[2] C.C.Chi et al, “Parallel scalability and efficiency of HEVC parallelization approaches”, IEEE Trans. CSVT, vol. 22, pp , Dec.2012.
[3] J. Lainema et al, ”Intra coding of the HEVC standard”, IEEE Trans. CSVT, vol.22, pp , Dec.2012.
[4] F. Bossen et al, “HEVC complexity and implementation analysis”, IEEE Trans. CSVT, vol. 22, pp , Dec.2012.
[5] P.Hanhart et al, “Subjective quality evaluation of the upcoming HEVC video compression standard” SPIE Applications of digital image processing XXXV, vol.8499, pp , Aug.2012.
[6] J.-R Ohm et al, “Comparison of the coding efficiency of video coding standards- including high efficiency video coding (HEVC)” , IEEE Trans. CSVT , vol.22, pp , Dec.2012.

31. References
[7] X. Zhang, S. Liu and S. Lei,”Intra mode coding in HEVC standard”, Visual Communications and Image Processing, VCIP 2012, pp. 1-6, San Diego, CA, Nov.2012.
[8] Y.Duan, “An optimized real time multi-thread HEVC decoder”, Visual Communications and Image Processing, VCIP 2012, San Diego, CA, Nov.2012.
[9] G. Correa et al, “Performance and computational complexity assessment of high efficiency video encoders”, IEEE Trans.CSVT, vol.22, pp , Dec.2012.
[10] A.Saxena, F. Fernandes and Y. Reznik, "Fast transforms for intra- prediction-based image and video coding,” in Proc. IEEE Data Compression Conference (DCC’13), pp.13-22, Snowbird, UT, March 2013.
[11] K.R.Rao, D.N.Kim and J.J.Hwang, "Video Coding Standards: AVS China, H.264/MPEG-4 Part10, HEVC, VP6, DIRAC and VC-1", Springer, 2014.

32. References
[12] HEVC open source software (encoder/decoder)
[13]Introduction to parallel computing
[14] Information about quad tree structure of HEVC tb/
[15] Guide into OpenMP: Easy multithreading programming for C++
[16] Website for downloading test sequence for research purposes
[17] Information on developments in HEVC NGVC- Next generation video coding
[18] H.265 standard finalized
could-finally-replace-mpeg-2-and-usher-in-uhdtv

33. References
[19] F. Bossen, D. Flynn and K. Suhring(July 2012), “HEVC reference software manual online available:
1/JCTVC-F634-v2.zip
[20] JCT-VC documents are publicly available at arch/jctvc-site and
[21] T.L Silva et al,”HEVC intra coding acceleration based on tree inter-level mode correlation”, SPA 2013 Sep.2013, Poznan, Poland
[22] A. Saxena and F. Fernandes, “Mode dependent DCT/DST for intra prediction in block based image/video coding”, IEEE ICIP, pp , Sept
[23] H. Zhang and Z. Ma, ”Fast intra prediction for high efficiency video coding ”, Pacific Rim Conf. on Multimedia, PCM2012, Singapore, Dec.2012.

34. References
[24] M. Zhang, C. Zhao and J. Xu, ”An adaptive fast intra mode decision in HEVC ”, IEEE ICIP 2012, pp , Orlando, FL, Sept.- Oct.2012.
[25] K. Chen et al,”Efficient SIMD optimization of HEVC encoder over X86 processors”, APSIPA, pp , Los Angeles, CA, Dec
[26] Y. Kim et al, “A fast intra-prediction method in HEVC using rate- distortion estimation based on Hadamard transform”, ETRI Journal, vol.35, #2, pp , Apr.2013.
[27] T. Wiegand et al., ”Overview of the H.264”, IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp , Jul.2003.
[28] M. Khan et al, “An adaptive complexity reduction scheme with fast prediction unit decision for HEVC Intra encoding”, IEEE ICIP, pp , Sept
[29] Il-Koo Kim et al, "HM9: High Efficiency Video Coding (HEVC) Test Model 9 Encoder Description", JCTVC-K1002-vl, 11th Meeting: Shanghai, CN, October, 2012.

35. References
[30] "Introduction to the issue on video coding: HDTV and beyond IEEE journal of selected topics in Signal Processing", vol 7, Dec 2013.
[31] G.J.Sullivan, et al, "HEVC Range Extensions Draft 5", JCT-VC, version 1, Geneva, Nov
[32] Official website for information on OpenMP
[33] M.Mrak, A. Gabriellini and D.Flynn, “Parallel processing for combined intra prediction in high efficiency video coding”, IEEE ICIP, pp , Sept
[34] J.Rehman and Y. Zhang,”Fast Intra Prediction mode decision using parallel processing”, Proceedings of the fourth international conference on machine learning and cybernetics, pp , Aug. 2005
[35] Overhead in openMP parameters socs/ /Using-OpenMP-for-programming-parallelthreads-in- multicore-applications-Part-2

36. References
[36] X. Li et al, “Rate-Complexity-Distortion evaluation for hybrid video coding”, IEEE Transactions on CSVT, vol. 21, pp , July 2011.
[37] V.Sze, M.Budagavi and G.J. Sullivan, “High Efficiency Video Coding (HEVC) - Algorithms and Architectures”, Springer, 2014.
[38] S.N.Agathos, P. Hadjidoukas and V. Dimakopoulos,“Task based execution of Nested OpenMP loops”, Springer, 2012.
[39] R.Chandra et al, “Parallel programming in OpenMP”, Academic Press,
[40] R.Eigenmann and B. Supinski, “OpenMP in a New Era of Parallelism”, 4th International Workshop, IWOMP 2008 West Lafayette, IN, USA proceedings, Springer 2008.
[41] Getting Started with CUDA UDA_Training_NVISION08.pdf

37. References [42] Multithreaded programming guide
[43] Information about Pthread
[44] Introduction to parallel programming
[45] Introduction to parallel programming and MapReduce
tionToParallelProgrammingAndMapReduce.pdf
[46] Information about shared memory
shm.html
[47] Definition of thread

38. References [48] Information about message passing interface
[49] Information about Data parallel programming model
[50] Information about Load imbalance
[51] Description about race around condition and deadlocks
[52] Information and example for deadlock
[53] Open MP components
04_06_2012.pdf

39. References [54] OpenMP Run time library routines
ompiler/cpp-lin/GUID-D3FC1F0B-DD B9B5-56EEE72E81A7.htm
[55] OpenMP environment variable
[56] OpenMP tutorial
[57] Basics of video:
[58] I.E. Richardson, “the H.264 advanced video compression standard”, Wiley, 2010.
[59] K. Choi et al, “Fast coding unit decision method based on coding tree pruning for high efficiency video coding”, Proc. SPIE Optical Engineering, vol. 51, , March 2012.

40. References
[60] JCT-VC documents can be found in JCT-VC document management system
[61] Thesis by S.Gangavati on “Complexity reduction of H.264 using parallel programming” which describes significant speed-up in encoding time on GPU using CUDA and CPU combined than on CPU by data and task parallelization, Access from
[62] Thesis by T.Saxena on “Reducing the encoding time of H.264 Baseline profile using parallel programming techniques” which describes task based parallelism using OpenMP software without any degradation of quality, Access from
[63] Thesis by S.Muniyappa on “Implementation of complexity reduction algorithm for intra mode selection in H.264/AVC Video coding” which describes complexity reduction algorithm which is much faster than JM reference software, Access from

41. References
[64] Thesis by T.Sathe on “Complexity reduction in H.264 encoder using OpenMP” which basically makes use of parallel processing approach using threads that are managed by OpenMP, Access from
[65] Thesis by P.K.Gajalla on “Efficient HEVC lossless coding using sample based angular intra prediction” which describes use of sample based angular prediction approach which can be used for better intra prediction accuracy compared to HEVC block angular intra prediction, Access from

42. References
[66] Thesis by Parashar Nayana Karunakar on Implementation of an out-of-the-loop post-processing technique for HEVC compressed depth maps, Access from
[67] Thesis by Sapna Vasudevan on Implementation of ROT and fast intra prediction in HEVC, Access from
[68] G. Bjontegaard, “Calculation of average PSNR differences between RD-curves”, Q6/SG16, VCEG, April 2013.
[69] Parallel computing the-need-for-speed/

43. Acronyms API: Application Programming Interface
AVC: Advanced Video Coding
BD: Bjontegaard Delta
CABAC: Context Adaptive Binary Arithmetic Coding
CB: Coding Block
CPU: Central Processing Unit
CSVT: Circuits and Systems for Video Technology
CTB: Coding Tree Block
CTU: Coding Tree Unit
CU: Coding Unit
CUDA: Compute Unified Device Architecture
DCC: Data Compression Conference
DCT: Discrete Cosine Transform
DST: Discrete Sine Transform

44. Acronyms FPGA: Field Programmable Gate Array GOP: Group Of Pictures
GPU: Graphic Processing Unit
HEVC: High Efficiency Video Coding
ICIP: International Conference on Image Processing
ISO: International Organization for Standardization
ITU-T: International Telecommunication Union – Telecommunication Standardization Sector
JCT-VC: Joint Collaborative Team on Video Coding
MC: Motion Compensation
MCP: Motion Compensated Predication
MPEG: Moving Picture Experts Group
MPL: Multimedia Processing Lab

45. Acronyms NGVC: Next Generation Video Coding
OPENMP: Open Multiprocessing
PB: Prediction Block
POC: Phase-Only Correlation
PCM: Pulse Code Modulation
PU: Prediction Unit
SAO: Sample Adaptive Offset
SIMD: Single Instruction Multiple Data
TB: Transform Block
VCEG: Video Coding Experts Group
VCIP: Visual Communication and Image Processing

46. THANK YOU