1. PROJECT INTERIM REPORT HEVC DEBLOCKING FILTER AND ITS IMPLEMENTATION RAKESH SAI SRIRAMBHATLA UTA ID: EE 5359 Under the guidance of DR. K. R. RAO

2. Contents
Objective
HEVC overview
Video compression standards
Working of HEVC
Deblocking filter
SAO filter
Scope of the project
Acronyms
References

3. OBJECTIVE
Scope of this project is to primarily focus on the study of in-loop deblocking filter in HEVC.
Besides this study, modification is done on the deblocking filter which results in the performance improvement of the codec.
To compare the HEVC performance after implementing in HM codec with H.264/AVC codec [22] using processor clock cycle, mean square error (MSE) and PSNR [25]and BD bitrate [26] as metrics.

4. HEVC OVERVIEW
High Efficiency Video Coding (HEVC) is finalized as the newest video coding standard of the ITU-T Video Coding Experts Group and the ISO/IEC Moving Picture Experts Group. [2]
Joint collaborative team on video coding (JCT-VC) developed HEVC video coding standard. JCT-VC is a joint partnership team between video coding experts group (VCEG) and moving picture experts group (MPEG). [2]
Main , main 10 and main intra profiles have been finalized in Extensions such as 3D video, scalable HEVC, 4:4:4, 4:2:2 formats are to be finalized in [3]

5. HEVC has many advantages over previous coding standards H
HEVC has many advantages over previous coding standards H.264 [22] and latter versions, like significant rate distortion efficiency achieving higher bit rate reduction, error resilience and is most network friendly compared to other standards. [2]
The main goal of the HEVC standardization effort is to enable significantly improved compression performance relative to existing standards—in the range of 50% bit-rate reduction for equal perceptual video quality. [2]

6. While H.264/AVC[22] divides a picture into fixed size macroblocks of 16×16 samples, HEVC divides a picture into coding tree units (CTU) of 16 × 16, 32 × 32 or 64 × 64 samples.
The coding tree units can be further divided into smaller blocks using a quadtree structure; such a block, called a coding unit (CU), can further be split into prediction units (PUs)
HEVC has been designed to address essentially all existing applications of H.264/MPEG-4 AVC [22] and to particularly focus on two key issues: increased video resolution and increased use of parallel processing architectures.

7. Subdivision of a CTB into CBs [and transform block (TBs)].
Solid lines indicate CB boundaries and dotted lines indicate TB boundaries.
CTB with its partitioning. (b) Corresponding quadtree. [2]
(a) [b]

8. HIGH EFFICIENCY VIDEO CODING
VIDEO COMPRESSION STANDARDS
STANDARD
SOFTWARE
MAIN APPLICATION
YEAR
JPEG2000
JasPer [21]
IMAGE
2000
H.264/MPEG-4 part 10
JM software[22]
ADVANCED VIDEO CODING
2003
H.265
HM Software [23]
HIGH EFFICIENCY VIDEO CODING
2013

9. GENERATIONS OF VIDEO CODING STANDARDS [6]

10. H.264 functional block diagram [28]

11. BLOCK DIAGRAM OF HEVC ENCODER [2]

12. HEVC DECODER BLOCK DIAGRAM [27]

13. GOP - Group Of Pictures ( Video Frames ). [7]
I-Intra Frame : P – Predicted frame : B – Bidirectional predicted frame
Order in which different types of frames are placed makes a GOP structure
A classic GOP structure is shown below

14. STRUCTURE OF HEVC CODEC [5]

15. Encoder
Partitioning each unit into multiple units.
Predicting each unit using intra or inter prediction and subtracting the prediction from the unit
Transforming and quantizing the residual.
Entopy encoding the transform output, prediction information and headers.
Decoder
Entropy decoding and extracting the elements from the coded sequence.
Rescaling and inverting the transform stage.
Predicting each unit and adding the prediction to the output of the inverse transform.
Reconstructing a decoded video image.

16. DEBLOCKING FILTER AND ITS WORKING
The deblocking filter performs detection of the artifacts at the coded block boundaries and attenuates them by applying a selected filter.
Compared to the H.264/AVC deblocking filter, the HEVC deblocking filter has lower computational complexity and better parallel processing capabilities while still achieving significant reduction of the visual artifacts. [13]
In HEVC, two processing steps, namely a deblocking filter (DBF) followed by an SAO filter, are applied to the reconstructed samples before writing them into the decoded picture buffer in the decoder loop.
The deblocking filter is intended to reduce the blocking artifacts due to block-based coding.

17. Sample adaptive offset filter
A nonlinear amplitude mapping is introduced within the interpicture prediction loop after the deblocking filter.
Its goal is to better reconstruct the original signal amplitudes by using a look-up table that is described by a few additional parameters that can be determined by histogram analysis at the encoder side.
The key idea of SAO is to reduce sample distortion by first classifying reconstructed samples into different categories, obtaining an offset for each category, and then adding the offset to each sample of the category.
The latter consideration is important since the deblocking filter of the H.264/AVC [22] standard, constitutes a significant part of the decoder complexity. [2]

18. The deblocking filter is applied to all samples adjacent to a PU or TU boundary except the case when the boundary is also a picture boundary
Unlike H.264/MPEG-4 AVC [22], where the deblocking filter is applied on a 4×4 sample grid basis, HEVC applies the deblocking filter only to the edges that are aligned on an 8×8 sample grid, for both the luma and chroma samples.

19. Schematic showing the edges of Prediction unit, Transform unit and picture boundary [11]
V- vertical edge
H- horizontal edge

20. Test sequences [18]

21. Basketball.yuv [18]
Picture of test sequence Basketball.yuv with resolution 832×480 with the frame rate = 50 frames/sec.

22. BQMall.yuv [18]
Picture of test sequence BQmall.yuv with resolution 832×480 with the frame rate = 60
frames/sec.

23. Racehorses.yuv [18]
Picture of test sequence Racehorses.yuv with resolution 416×240 with the frame rate =
30 frames/sec.

24. Kirsten&Sara.yuv [18]
Picture of test sequence Kirsten&Sara.yuv with resolution 1280×720 with the frame rate
= 60 frames/sec.

25. Scope of this project
To study deblocking filter operation in HEVC and implement for reduction of blocking visual artifacts using the filter.
To compare the performance of HEVC after enabling the deblocking filter and SAO (sample adaptive offset) with a default HEVC setting using the metrics : complutaional time, BD bitrate and PSNR [25][26].
To compare the HEVC performance in HM codec with H.264/AVC [22] codec performance using metrics: BD bitrate [26], BD PSNR [25] and MSE.

26. Implementation
Test sequence and test environment details
In the initial implementation, Kirsten&Sara.yuv [18] is considered as the test sequence.
Picture of test sequence Kirsten&Sara.yuv [18] with resolution 1280×720 with the frame rate = 60 frames/sec.
Total of 10 frames are considered for the test.
HM.13 HEVC codec software has been used for execution of the test.
HEVC main profile is considered for the initial implementation of encoder.

27. Deblocking filter implementation
Enable/Disable Deblocking Filter [23]
LoopFilterDisable parameter – under Deblock Filter – alone would not do the job of disabling the deblocking filter [23]
Have to use combination of following deblocking filter parameters: [23]
DeblockingFilterControlPresent
LoopFilterDisable
LoopFilterBetaOffset_div2
LoopFilterTcOffset_div2

28. Metrics for the test
BD PSNR is plotted for all the test sequences for different quantization parameters.
Total encoding time is plotted for all the test sequences for different quantization parameters.
BD bitrate is also plotted for all the test sequence for different quantization parameters.
[1]

29. Results and observations Tables
Table 1 represents the metrics when the deblocking filter disabled.
Kristen & Sara
Bitrate
Computational
Time
PSNR
QP = 32
Kbps
secs
38.879
Table 1
Table 2 represents the metrics when the deblocking filter is enabled.
Kristen & Sara
Bitrate
Computational
Time
PSNR
QP = 32
Kbps
secs
38.879
Table 2

30. Computational time (in seconds)
Kirsten&Sara.yuv [18] with resolution 1280×720 with the frame rate = 60 frames/sec. 10 frames are considered.

31. BD Bitrate ( in Kbps) [26]
Kirsten&Sara.yuv [18] with resolution 1280×720 with the frame rate = 60 frames/sec. 10 frames are considered.

32. BD PSNR ( in dB) [25]
Kirsten&Sara.yuv [18] with resolution 1280×720 with the frame rate = 60 frames/sec. 10 frames are considered.

33. Conclusion
The computational time for the test sequence has increased from secs to secs when the deblocking filter was applied to the encoder since the complexity has increased because of the addition of deblocking filter to the elements of the HEVC block.
PSNR and bitrate are the performance metrics. Both PSNR and bitrate have increased with the addition of deblocking filter which shows improvement in performance of the codec.
In the final implementation, these metrics are calculate for main and low delay profiles and HEVC deblocking filter performance would be compared with the deblocking filter in H.264. All the tests are repeated for four test sequences and an aggregated result will be given in the final report

34. Acronyms AVC Advanced Video Coding ALF Adaptive loop filter
B Bi-predictive
CB Coding Block
DBF Deblocking filter
CSVT Circuits and Systems for Video Technology
CTB Coding Tree Block
CTU Coding Tree Unit
CU Coding Unit
GOP Group of pictures
HEVC High Efficiency Video Coding
I Intra frame
ISO International Standardization Organization
ITU-T International Telecommunication Union - Telecommunication Standardization Sector
JCT-VC Joint Collaborative Team on Video Coding
LCU – Largest coding unit
MPEG Moving Picture Experts Group
MSE – Mean square error
PB Prediction Block
PSNR – Peak signal to noise ratio
QP – Quantization parameter
SAO – Sample adaptive offset
 SNR – Signal to noise ratio
 SPS – Sequence parameter set
TB Transform Block
TU Transform Unit
VCEG Video Coding Experts Group

35. REFERENCES
[1] G.J Sullivan et al, “Overview of the H.264/AVC Video Coding Standard”, IEEE Trans. CSVT, vol.13, pp , July 2003
[2] G.J. Sullivan et al, “Overview of the high efficiency video coding (HEVC) standard”, IEEE Trans. CSVT, vol.22, pp , Dec. 2012
[3] G.J. Sullivan et al, “Standardized extensions of High Efficiency Video Coding (HEVC)”, IEEE Journal of Selected topics in signal processing, vol. 7, pp , Dec. 2013
[4] G.J. Sullivan et al, “Comparison of the Coding Efficiency of Video Coding Standards-Including High Efficiency Video Coding (HEVC)”, IEEE Trans. CSVT, Vol. 22, pp , Dec.2012

36. [5] http://www. vcodex. com/images/uploaded/342512928230717
[5] , URL for document on introduction to HEVC
[6 ] Wikipedia URL for scalable video coding which has the definitions of video coding standards.
[7 ] URL for NPTEL video on basics of video coding standards.
[8] Wikipedia search URL for types of frames in video compression.
[10] N. Ling, “High efficiency video coding and its 3D extension: A research perspective," 7th IEEE, Trans. on ICIEA, pp , July 2012.
[11] M. Li et al, “De-blocking Filter Design for HEVC and H.264/AVC”, PCM 2012, Lecture Notes in Computer Science 7674, pp. 273–284, 2012.
[12] M. Naccari et al, “Low Complexity Deblocking Filter Perceptual Optimization For The HEVC Codec”, 18th IEEE International Conference on Image Processing, pp , 2011.

37.   [13] A. Norkin et al, “HEVC Deblocking Filter”, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp , Dec
[14] JM software download for H.264/AVC:
[15] HM codec download for H.265:
[16] HEVC standard test video sequences: ftp://ftp.tnt.uni-hannover.de/testsequences
[18] Test sequence source :
[19] HEVC Reference Software HM
[20] 10.2 JM H.264 software
[21] JPEG2000 latest reference software (jasper version ) Website:

38. [22] JM reference software, Fraunhofer institute for telecommunications Heinrich Hertz institute.
[23] HM reference software, Fraunhofer institute for telecommunications Heinrich hertz institute.
[24] C. M Fu et al, “Sample Adaptive Offset in the HEVC Standard”, IEEE Trans. CSVT, Vol.22, pp – 1764, Dec 2012.
[25] PSNR reference
[26] BD bitrate reference Open source article, “Bit rate”. [Online]. Available:
[27] B. Bross et al, “High efficiency video coding (HEVC) text specification draft 10”, 12th Meeting Geneva January [Online]. Available:
[28] Yu-Wen Huang et al, “Architecture design for deblocking filter in H.264/JVT/AVC”, 2003 International Conference on Multimedia and Expo, ICME '03. Proceedings.  Pages I , vol.1, July 2003

39. [29] K.R. Rao, D.N. Kim and J.J. Huang, ” Video coding standards”, Springer, 2014.
[30] Special issue on emerging research and standards in next generation video coding, IEEE transactions on circuits and systems for video technology (CSVT), vol. 22, pp , Dec
[31] Special issue on emerging research and standards in next generation video coding, IEEE transactions on circuits and systems for video technology (CSVT), vol. 22, pp , Dec
[32] IEEE Journal of Selected Topics in Signal Processing, vol. 7, pp , Dec