Download presentation

Presentation is loading. Please wait.

Published byValentina Lightfoot Modified over 4 years ago

1
Image Deblocking Using Local Segmentation Lukasz Kizewski Supervisor:Dr. Peter Tischer Second Examiner:Dr. Andrew Paplinski

2
Presentation Outline Introduction Lossy Image Compression - JPEG Discrete Cosine Transform (DCT) Subbands Present Research Research gap DCT coefficient study Deblocking Filter Conclusions Further Research

3
Lossy Image Compression JPEG, MPEG is lossy Maintaining high image quality and high compression ratios is a major, widespread issue Used everywhere! Internet, digital photography, digital camcorders, DVDs, Digital TV, video telephony/conference, mobile phones, etc. JPEG/MPEG uses ‘Transform Coding’ technique coupled with data quantization JPEG/MPEG uses Discrete Cosine Transform (DCT) DCT is energy-preserving and reversible Quantization step is the lossy part

4
Discrete Cosine Transform Image divided into 8x8 pixel blocks DCT applied to each block independently Block decomposed into basis functions First basis function (0,0) termed ‘DC coefficient’ Remaining 63 basis functions termed ‘AC coefficients’ [1] DC coefficient - Average brightness AC coefficients -White: Add to average -Black: Subtract from average

5
DCT - Subbands For every 8x8 pixel block, output of DCT is 64 DCT coefficients Each coefficient corresponds to one basis function “Image” of 1 DCT coefficient termed a “subband” [1] 2x2 block DCT example

6
Subbands First 4 (out of 64) subbands of ‘Lenna’ [1]

7
After performing the DCT, resulting coefficients are quantized Divided by quantum value, rounded to integer Quantum value dictated by ‘quality’ parameter and quantization table High-order DCT coefficients more severely quantized (usually to zero) During de-quantization, mid-point of quantization interval is chosen Usually incorrect DCT Coefficient Quantization Q=10 514 1 5 10

8
JPEG – Decompression Quality JPEG decoded image at different ‘quality’ parameter settings: Q = 100Q = 50Q = 1

9
JPEG Image artifacts Incorrectly reconstructing DCT coefficients results in unwanted image artifacts: Smooth regions are blocky, edges are jagged, discontinuities appear near edges Aim of project – decrease severity of artifacts Smooth region (shoulder) Staircase effect (edge of hat) Ringing effect (edge of mirror)

10
Present Research Dozens of filters exist to increase the quality of highly-compressed images Some filter DCT coefficients (subbands) Some filter reconstructed pixel values Others filter both Most filters target only one type of image artifact Some filters reduce one type of artifact whilst making another more prominent

11
Research Gap Natural/photographic images have high correlation between neighbouring pixels Neighbouring pixels are similar in brightness Property fails when edge in image is encountered Lack of image segmentation in most filters Results in blurred/smoothed edges Encountering an edge implies more than one segment Segments should be filtered independently

12
Filling the Research Gap This project differs in that: Local segmentation is used Pixels possibly split into 2 groups Each segment filtered independently “Do No Harm” policy used – avoid further image quality degradation If filtering model fails, it’s filtered value is disregarded Worst-case scenario – image is not filtered at all

13
Filling the Research Gap (cont.) Filter can be applied to DCT Coefficients (filtering subbands) Pixels (filtering an image) Pixels may be filtered after subbands are reconstructed as accurately as possible Quality loss occurs at subband level First filtering step should be reconstructing DCT coefficients better

14
DCT Coefficient Study Determine each subband’s contribution to overall image quality Selected subbands were not quantized Simulates subbands being reconstructed perfectly Subband’s contribution measured by increase in Peak Signal-to-Noise Ratio (PSNR) PSNR: Logarithmically-scaled, mean-squared-error metric NOTE: PSNR value doesn’t always reflect viewer-subjective image quality assessment

15
DCT Coefficient Study (cont.) Results of study:

16
Image Deblocking Filter Main goal is to reconstruct DCT coefficients better to reduce severity of image artifacts DC subband filtered DCT study shows DC subband is best filter candidate A DCT coefficient is a subband “pixel” Uses 3x3 weighted mask Mask center is pixel being filtered Mask scans entire image, filtering each pixel 121 242 121 3x3 filtering mask DC subband of ‘Lenna’

17
“Do No Harm” Filter employs “Do No Harm” (DNH) policy If a filtered value is implausible, reject it and leave value unfiltered Implausible pixel value is one that falls outside the quantization interval Quantization interval: midpoint +/- ½ Quantum Guarantees image quality cannot degrade further

18
Sequential Filter Try 1-segment filtering Replace pixel with mask average If DNH not triggered, accept Otherwise try 2-segment filtering Use average-value thresholding to classify pixels in mask Replace pixel with average of segment to which it belongs to If DNH not triggered, accept Otherwise trigger DNH and leave unfiltered value 100 15 100105 Weighted average = 50.94 100 15 100105 Blue class average =12.78

19
Sequential Filter - Results Blocky effect reduced to 8x8 pixels Contrast between blocks minimised Filtered image can be used as input to another filter

20
DC Subband Expansion Resolution of DC subband is increased 8-fold To match the resolution of the original image Inverse-DCT modified to use expanded DC subband Linear-interpolation used to fill the gaps between original DC subband values Applies a “gradient” to DC subband 128x128 pixel image128x128 expanded DC subband DC subband DCTDC expansion

21
DC Subband Expansion - Results Smooth regions completely void of blocky artifacts Expansion applied to filtered DC subband

22
What about the edges? ‘Staircase’ and ‘ringing’ effects still visible in all results Sequential deblocking filter inappropriate for AC subband filtering AC subbands have very low inter-pixel correlation Filtering AC subbands with this filter introduces ringing DNH isn’t triggered because AC quantization intervals are very wide

23
What about the edges? (cont.) AC subbands describe edges, with respect to DC value Modifying AC coefficients adds/removes edges/textures Filtered image (far right) shows a light edge added next to the actual edge – a ringing artifact! AC filtering – topic for further research OriginalReconstructedFiltered-reconstructed

24
Unsuccessful Additions Always use 2-segment filtering Contrast between blocks increased ‘Chessboard’ effect Threshold segmenting If class representatives are close, treat mask as 1 segment Finding correct threshold value impossible 3-segment filtering If 2-segment filtering triggered DNH, 3-segment filtering was attempted Little-to-no improvement in image quality Concluded that 2-segment model is sufficient for 95% of cases

25
Unsuccessful Additions (cont.) Overlapping mask filtering All pixels’ filtered values in a mask were recorded Filtered value equal to average of all possible recorded reconstructions (most pixels had up to 9 possible filtered values) Little-to-no improvement in image quality Applying sequential filter to reconstructed pixel values “Quantization interval” parameter not known Guessing interval resulted in: Blurry images – blurred edges Overly-sharpened images – “cardboard cut-out” effect with flat colors

26
Conclusions Filtering DC subband has been successful in improving overall image quality As predicted by the DCT coefficient study Due to high inter-pixel correlation AC subbands must be filtered in some other way This filter produces ringing artifacts Due to averaging Due to little inter-pixel correlation Due to extremely wide quantization intervals – allowing large change to an AC coefficient’s value Little or no information left in subband Perhaps try median-filtering, instead of averaging May introduce blurring of edges

27
Conclusions (cont.) Knowledge of a subband’s quantization interval aids in segmentation Maximum quantization error is known DNH “switches” between 1- or 2-segment filtering Sequential (adaptive) filtering model proved successful DNH policy ensured no further image quality loss Sometimes too strict Some blocks “don’t fit” in with surrounding pixels Conforms to JPEG specifications Critical for video sequence coding (MPEG) – reconstruction errors propagate to subsequent frames

28
Future Research AC subbands must be filtered in a different manner altogether Perhaps use DC subband segmentation to drive AC subband filtering Resulting image from filtered, non-expanded DC subband filter can be used as a starting point for a secondary filter This filter should target edge artifacts Once edges are also filtered, DC expansion could be applied as a third step Try median-filtering, instead of averaging

29
For More Information… Consult Thesis See Website http://www.csse.monash.edu.au/~kizewski/ E-mail me kizewski@mail.csse.monash.edu.au Read referenced material Read filter source code Ask a question [1]: Rabbani, M. and Jones, P. W. (1991). Digital Image Compression Techniques, Vol. TT7 of SPIE Tutorial Texts. [2]: http://www.mat.univie.ac.at/~kriegl/Skripten/CG/node53.html

30
Thank-You Questions? …and now, for some eye-candy!

Similar presentations

OK

5. 1 JPEG “ JPEG ” is Joint Photographic Experts Group. compresses pictures which don't have sharp changes e.g. landscape pictures. May lose some of the.

5. 1 JPEG “ JPEG ” is Joint Photographic Experts Group. compresses pictures which don't have sharp changes e.g. landscape pictures. May lose some of the.

© 2019 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google