1. 1 Pixel Interpolation By: Mieng Phu Supervisor: Peter Tischer

2. 2 Outline Pixel Interpolation and Background Scenarios Interpolation Techniques Test Data Results and Discussions Future Work Summary

3. 3 Pixel Interpolation Predicting/interpolating missing values occurs in many areas of image processing, especially in lossless image coding. The idea of this project is to take the prediction techniques from lossless image coding and apply them to other area of image/video processing.

4. 4 Scenarios in Video/Image Processing Deinterlacing within a single field (spatial) Deinterlacing in two fields (spatial and temporal) Deinterlacing in three fields (spatial and temporal) Convert from SDTV to HDTV (Magnification)

5. 5 xxx xxx ??? Current field Time t i ??? ??? xxx xxx xxx ??? Current fieldPrevious field Scenario Two (2 fields) Scenario One (1 field) X- known value ? - unknown value

6. 6 Scenario Three (3 fields) ??? ??? xxx xxx xxx ??? ??? ??? xxx t i - 1 titi t i +1 Previous fieldCurrent fieldFuture field

7. 7 Terminology Neighboring pixels: NWNNE SWSSE W ? E ABC DEF ?

8. 8 Interpolation Techniques(1) Scenario One Predictors (Prediction within a single field or on still image). –Line doubling –Averaging, e.g. (N+S)/2, (NW+N+NE+SW+SE)/2 –Median filter. –Pseudo Median (PMED) - H-Shape, A-Shape, Adaptive. –ELA, Adaptive ELA, ELA (Tao Chen). –Some of my proposed algorithms.

9. 9 Interpolation Techniques(2) Edge Line Averaging (ELA) Adaptive - ELA (A-ELA) –The concept is the same as ELA, but it uses a unique way to detect a horizontal edges. 11050100 105010 ? xxx xxx ?

10. Interpolation Techniques(3) ELA – Chen, Henry, et al. Used two additional measurements to determine the direction correlations. Hence, good predictor for a 63 0 edges from the horizontal. 11050100 105010 ? xxx xxx ? xxx xxx ?

11. 11 Interpolation Techniques(4) Median Filter –Median{10,10,10,10,100,100} Pseudo Median (by definition) 10 100 10 100 ? Segment 1Segment 2 PMED{ a, b, c, d, e, f} = 0.5 x max [min of each sub window] + 0.5 x min [max of each sub window]

12. 12 Interpolation Techniques(5) For scenario two and three predictors: –Interpolation techniques in situation scenario one can be generalized in situation two and three. ghi abc def ? Current fieldPrevious field ? = h ? = (b + h + e)/3 ? = median (b, h, e,) ? = PMED {a, b, c, d, e, f, h}

13. 13 jkl abc def ?ghi t i - 1 titi t i +1 Previous fieldCurrent fieldFuture field ? = (h +k)/2? = Median {h, k, (b+e)/2} ? = PMED {a, b, c, d, e, f, h, k}

14. 14 Test Data Standard natural Images Synthetic Images –Lines with different orientations –Different textures Video sequences –Different speeds of motion between fields. –Textures – edges, lines etc.

15. 15 Scenario1 :Results(1) Rank AlgorithmAverage PSNR (dB) 1HSHAPE PMED29.55 2Median (Proposed)29.51 3Adaptive PMED29.45 4CHEN PMED (Proposed)29.39 5Window 1 (Proposed)29.25 6Median filter29.22 7Average 229.17 8ELA (CHEN)29.06 9A-Shaped PMED28.83 10Average 628.59 11A-ELA28.18 12ELA28.08 13LD25.47

16. 16 Scenario1 :Results(2) H-Shaped PMED –Best predictor overall –Superb in detection of vertical edges. H-Shaped PMED {a, b, c, d, e, f} = 0.5 x max[min{a,b,c,},min{d,e,f},min{b,e}] + 0.5 x min[max{a,b,c},max{d,e,f},max{b,e}] abc def ? Current field Time t i

17. 17 Scenario1 :Results(3) LD- 21.61 dB ELA - 19.26 dB A-ELA- 19.16 dB ELA (CHEN) - 24.99 dB But even better results… H-Shaped- 25.27 dB CHEN PMED- 25.22 dB

18. 18 Scenario1 :Results(4) CHEN PMED is a combination of the PMED and ELA (Chen). It use ELA(Chen) to select the PMED subwindows. Like ELA(Chen), it predict well for edges at an orientation of 63 0. This algorithm perform better than ELA(Chen) in wide range of images. 11050100 105010 ? xxx xxx ? xxx xxx ?

19. 19 Scenario1 :Results(5) The median {a, b, c, d, e, f, (b+e)/2} Can have the –median {a, b, c, d, e, f, (b+e)/2, (a+b)/2, (b+c)/2, (d+e)/2, (e+f)/2} Furthermore, you can further subdivide into half-pel, quarter-pel, or even further. This approach is proven to be better than the generic median filter. But more computation. abc def ?

20. 20 Scenario1 :Results(6) A-ELA can detect horizontal lines incredible well. A-ELA – 39.7 dB H-Shaped – 25.7 dB ELA(CHEN) - 25.7dB A-ELA - 13.15 dB H-Shaped - 8.47 dB ELA(CHEN) – 8.45 dB

21. 21 Scenario1 :Results(7) Flower Garden (64 frames). 352x288 Football (64 frames). 352x240 Akiyo (10 frames). 352x288

22. 22

23. 23 Conclusions ELA (Chen) is good for detection of diagonal edges. ELA (Chen) can be improved by using PMED or Median filter. H-Shaped PMED can detect the vertical edges well, and perform best overall. A-ELA is really good at detect the horizontal lines. Median filter can be improved by using the right group of pixels. Average 2, give good results and for little computation.

24. 24 Future Work Potential algorithms in scenario one can be improved. Combine H-Shaped, ELA(CHEN) and A- ELA together, to form the best predictor. More generalization can be made on two and three fields. Magnifications

25. 25 Questions?