1. Block Loss Recovery Techniques for Image Communications Jiho Park, D-C Park, Robert J. Marks, M. El-Sharkawi The Computational Intelligence Applications (CIA) Lab. Department of Electrical Engineering University of Washington May 29, 2002

2. 2 Projections based Block Recovery – Motivation Conventional Algorithms use information of all surrounding area. Using only highly correlated area

3. 3 Alternating Projections is projecting between two or more convex sets iteratively. Alternating Projections Converging to a common point

4. 4 Projections based Block Recovery – Algorithm 2 Steps Pre Process : 1) Edge orientation detection 2) Surrounding vector extraction 3) Recovery vector extraction Projections : 1) Projection operator P1 2) Projection operator P2 3) Projection operator P3

5. 5 Edge orientation in the surrounding area( S ) of a missing block( M ). In order to extend the geometric structure to the missing block. Simple line masks at every i, j coordinate in surrounding area( S ) of the missing block( M ) for edge detection. Pre Process 1 – Edge Orientation Detection Horizontal Line Mask Vertical Line Mask

6. 6 Pre Process 1 – Edge Orientation Detection Responses of the line masks at window W : Total magnitude of responses : T h > T v ; Horizontal line dominating area T h < T v ; Vertical line dominating area

7. 7 Pre Process 2 – Surrounding Vectors Surrounding Vectors, s k, are extracted from surrounding area of a missing block by N x N window. Each vector has its own spatial and spectral characteristic. The number of surrounding vectors, s k, is 8N.

8. 8 Pre Process 3 – Recovery Vector Recovery vectors are extracted to restore missing pixels. Two positions of recovery vectors are possible according to the edge orientation. Recovery vectors consist of known pixels(white color) and missing pixels(gray color). The number of recovery vectors, r k, is 2. Vertical line dominating areaHorizontal line dominating area

9. 9 Projections based Block Recovery – Projection operator P 1 Recovery vectors, r i, for i = 1, 2 Surrounding vectors, s j, for j = 1 ~ 8N Surrounding vectors, S, form a convex hull in N 2 - dimensional space Recovery vectors, R, are orthogonally projected onto the line defined by the closest surrounding vector, s i, j : Projection Operator P 1.

10. 10 Projections based Block Recovery – Projection operator P 1 Projection operator P 1 : Convex hull (formed by surrounding vectors, containing information of local image structure)

11. 11 Projections based Block Recovery – Projection operator P 1 Surrounding vectors, s j, for j = 1 ~ 8N Recovery vectors, r i, for i = 1, 2 The closest vertex, s d i, from a recovery vector, r i. or equivalently in DCT domain, P 1 :

12. 12 Convex set C 2 acts as an “identical middle”. Projection operator P 2 : Projections based Block Recovery – Projection operator P 2

13. 13 Convex set C 3 acts as a convex constraint between missing pixels and adjacent known pixels, (f N-1 f N ). where, and is a N x N recovery vector in column vector form. Projections based Block Recovery – Projection operator P 3 f N-1 f N Projection operator P 3 :

14. 14 Projections based Block Recovery – Iterative Algorithm Missing pixels in recovery vectors are restored by iterative algorithm of alternating projections : N x N windows moving : Vertical line dominating areaHorizontal line dominating area

15. 15 Projections based Block Recovery - Summary Edge Orientation Detection Surrounding Vector Extraction Recovery Vector Extraction Projection Operator P 1 Projection Operator P 2 Projection Operator P 3 Iteration=I? All pixels?

16. 16 Simulation Results – Lena, 8 x 8 block loss Original ImageTest Image

17. 17 Simulation Results – Lena, 8 x 8 block loss Ancis, PSNR = 28.68 dBHemami, PSNR = 31.86 dB

18. 18 Simulation Results – Lena, 8 x 8 block loss Ziad, PSNR = 31.57 dBProposed, PSNR = 34.65 dB

19. 19 Simulation Results – Lena, 8 x 8 block loss Ancis PSNR = 28.68 dB Hemami PSNR = 31.86 dB Ziad PSNR = 31.57 dB Proposed PSNR = 34.65 dB

20. 20 Simulation Results – Each Step Lena 8 x 8 block loss (a) (b) (c)

21. 21 Simulation Results – Peppers, 8 x 8 block loss Original ImageTest Image

22. 22 Simulation Results – Peppers, 8 x 8 block loss Ancis, PSNR = 27.92 dBHemami, PSNR = 31.83 dB

23. 23 Simulation Results – Peppers, 8 x 8 block loss Ziad, PSNR = 32.76 dBProposed, PSNR = 34.20 dB

24. 24 Simulation Results – Lena, 8 x one row block loss Original ImageTest Image

25. 25 Simulation Results – Lena, 8 x one row block loss Hemami, PSNR = 26.86 dBProposed, PSNR = 30.18 dB

26. 26 Simulation Results – Masquerade, 8 x one row block loss Original ImageTest Image

27. 27 Simulation Results – Masquerade, 8 x one row block loss Hemami, PSNR = 23.10 dBProposed, PSNR = 25.09 dB

28. 28 Simulation Results – Lena, 16 x 16 block loss Original ImageTest Image

29. 29 Simulation Results – Lena, 16 x 16 block loss Ziad, PSNR = 28.75 dBProposed, PSNR = 32.70 dB

30. 30 Simulation Results – Foreman, 16 x 16 block loss Original ImageTest Image Ziad, PSNR = 25.65 dBProposed, PSNR = 30.34 dB

31. 31 Simulation Results – Flower Garden, 16 x 16 block loss Original ImageTest Image Ziad, PSNR = 20.40 dBProposed, PSNR = 22.62 dB

32. 32 Simulation Results – Test Data and Error 512 x 512 “Lena”, “Masquerade”, “Peppers”, “Boat”, “Elaine”, “Couple” 176 x 144 “Foreman” 352 x 240 “Flower Garden” 8 x 8 pixel block loss 16 x 16 pixel block loss 8 x 8 consecutive block losses Peak Signal – Noise – Ratio

33. 33 Simulation Results – PSNR (8 x 8) LenaMasqrdPeppersBoatElaineCouple Ancis28.6825.4727.9226.3329.8428.24 Sun29.9927.2529.9727.3630.9528.45 Park31.2627.9131.7128.7732.9630.04 Hemami31.8627.6531.8329.3632.0730.31 Ziad31.5727.9432.7630.1131.9230.99 Proposed34.6529.8734.2030.7834.6331.49

34. 34 Simulation Results – PSNR (Row, 16 x 16) (16 x 16)LenaForemanGarden Ziad28.7525.6520.40 Proposed32.7030.3422.62 (8 x Row)LenaMaskrdPeppersBoatElaineCouple Hemami26.8623.1025.4124.5426.8724.30 Proposed30.1825.0928.3126.0630.1126.12