1. ENEE631 Digital Image Processing (Spring'04) Digital Image and Video Processing – An Introduction Spring ’04 Instructor: Min Wu ECE Department, Univ. of Maryland, College Park   www.ajconline.umd.edu (select ENEE631 S’04)   minwu@eng.umd.edu Based on ENEE631 Spring’04 Section 1

2. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [2] Scope of ENEE631 First graduate course on image/video processing Prerequisites: ENEE620 and 624, or by permission –Not assume you have much exposure on image processing at undergraduate level –Random process and DSP are required background Emphasis on fundamental concepts –Provide theoretical foundations on multi-dimensional signal processing built upon pre-requisites –Coupled with assignments and projects for hands-on experience and reinforcement of the concepts –Follow-up courses u image analysis, computer vision, pattern recognition u multimedia communications and security

3. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [3] Textbooks Primary –A.K. Jain: Fundamentals of Digital Image Processing, Prentice-Hall, 1989. (blue cover) –R.C. Gonzalez and R.E. Woods: Digital Image Processing, Prentice Hall, 2001. (black cover) –Y. Wang, J. Ostermann, Y-Q. Zhang: Digital Video Processing and Communications, Prentice-Hall, 2001. References –A.Bovik & J.Gibson: Handbook Of Image & Video Processing, Academic Press, 2000. Other references –Will be announced in lectures

4. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [4] Introduction to Image and Video Processing UMCP ENEE631 Slides (created by M.Wu © 2001)

5. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [5] A picture is worth 1000 words. Rich info. from visual data Examples of images around us natural photographic images; artistic and engineering drawings scientific images (satellite, medical, etc.) “Motion pictures” => video movie, TV program; family video; surveillance and highway/ferry camera A video is worth 1000 sentences? UMCP ENEE631 Slides (created by M.Wu © 2001) http://marsrovers.jpl.nasa.gov/gallery/press/opportunity/20040125a.html JPL Mars’ Panorama captured by the Opportunity

6. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [6] Why Do We Process Images? Enhancement and restoration –remove artifacts and scratches from an old photo/movie –improve contrast and correct blurred images Transmission and storage –images from oversea via Internet, or from a remote planet Information analysis and automated recognition –providing “human vision” to machines Security and rights protection –encryption and watermarking UMCP ENEE631 Slides (created by M.Wu © 2001)

7. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [7] Why Digital? “Exactness” –Perfect reproduction without degradation –Perfect duplication of processing result Convenient & powerful computer-aided processing –Can perform rather sophisticated processing through hardware or software –Even kindergartners can do it! Easy storage and transmission –1 CD can store hundreds of family photos! –Paperless transmission of high quality photos through network within seconds UMCP ENEE631 Slides (created by M.Wu © 2001)

8. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [8] List of Image and Video Processing Examples Compression Manipulation and Restoration –Restoration of blurred and damaged images –Noise removal and reduction –Morphing Applications –Visual mosaicing and virtual views –Face detection –Visible and invisible watermarking –Error concealment and resilience in video transmission UMCP ENEE631 Slides (created by M.Wu © 2001)

9. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [9] Compression Color image of 600x800 pixels –Without compression u 600*800 * 24 bits/pixel = 11.52K bits = 1.44M bytes –After JPEG compression (popularly used on web) u only 89K bytes u compression ratio ~ 16:1 Movie –720x480 per frame, 30 frames/sec, 24 bits/pixel –Raw video ~ 243M bits/sec –DVD ~ about 5M bits/sec –Compression ratio ~ 48:1 “Library of Congress” by M.Wu (600x800) UMCP ENEE631 Slides (created by M.Wu © 2001)

10. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [10] Denoising From X.Li http://www.ee.princeton.edu/~lixin/denoising.htm UMCP ENEE631 Slides (created by M.Wu © 2001)

11. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [11] Deblurring http://www.mathworks.com/access/helpdesk/help/toolbox/images/deblurr7.shtml UMCP ENEE631 Slides (created by M.Wu © 2001)

12. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [12] Morphing Princeton CS426 face morphing examples http://www.cs.princeton.edu/courses/archive/fall98/cs426/assignments/morph/morph_results.html UMCP ENEE631 Slides (created by M.Wu © 2001)

13. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [13] Visual Mosaicing –Stitch photos together without thread or scotch tape R.Radke – IEEE PRMI journal paper draft 5/01 UMCP ENEE631 Slides (created by M.Wu © 2001)

14. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [14] Face Detection Face detection in ’98 @ CMU CS, http://www.cs.cmu.edu/afs/cs/Web/People/har/faces.html UMCP ENEE631 Slides (created by M.Wu © 2001)

15. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [15] ( Get more vision / image understanding e.g. from Rama’s group? ) UMCP ENEE631 Slides (created by M.Wu © 2004)

16. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [16] Visible Digital Watermarks from IBM Watson web page “Vatican Digital Library” UMCP ENEE631 Slides (created by M.Wu © 2001)

17. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [17] UMCP ENEE631 Slides (created by M.Wu © 2001) Invisible Watermark –1st & 30th Mpeg4.5Mbps frame of original, marked, and their luminance difference –human visual model for imperceptibility: protect smooth areas and sharp edges

18. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [18] Data Hiding for Annotating Binary Line Drawings original marked w/ “01/01/2000” pixel-wise difference UMCP ENEE631 Slides (created by M.Wu © 2001)

19. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [19] UMCP ENEE631 Slides (created by M.Wu © 2001) Error Concealment 25% blocks in a checkerboard pattern are corrupted corrupted blocks are concealed via edge-directed interpolation (a) original lenna image(c) concealed lenna image(b) corrupted lenna image Examples were generated using the source codes provided by W.Zeng.

20. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [20]

21. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [21] What is An Image? Grayscale image –A grayscale image is a function I(x,y) of the two spatial coordinates of the image plane. –I(x,y) is the intensity of the image at the point (x,y) on the image plane. u I(x,y) takes non-negative values u assume the image is bounded by a rectangle [0,a]  [0,b] I: [0, a]  [0, b]  [0, inf ) Color image – – Can be represented by three functions, R(x,y) for red, G(x,y) for green, and B(x,y) for blue. y x y x I(x,y) UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

22. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [22] Sampling and Quantization Computer handles “discrete” data. Sampling –Sample the value of the image at the nodes of a regular grid on the image plane. –A pixel (picture element) at (i, j) is the image intensity value at grid point indexed by the integer coordinate (i, j). Quantization –Is a process of transforming a real valued sampled image to one taking only a finite number of distinct values. –Each sampled value in a 256-level grayscale image is represented by 8 bits. 0 (black) 255 (white) UMCP ENEE631 Slides (created by M.Wu © 2001)

23. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [23] UMCP ENEE631 Slides (created by M.Wu © 2001) Examples of Sampling 256x256 64x64 16x16

24. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [24] UMCP ENEE631 Slides (created by M.Wu © 2001) Examples of Quantizaion 8 bits / pixel 4 bits / pixel 2 bits / pixel

25. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [25] Summary of Today’s Lecture Course organization and policies Background and examples of digital image processing Sampling and quantization concepts for digital image Next time –Color and Human Visual System UMCP ENEE631 Slides (created by M.Wu © 2001)

26. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [26] Readings and Assignment E-Handout (Sec.I of Bovik’s Handbook) Introductory sections in Matlab Image Processing Toolbox –http://www.mathworks.com/access/helpdesk/help/toolbox/images/images.shtml Go over mathematical preliminaries –Linear system and basics of 1-D signal processing –FT and ZT –Matrix and linear algebra –Probability UMCP ENEE631 Slides (created by M.Wu © 2001,2004)

27. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [27] Questions for Today (QFT) Why “seeing yellow without yellow”? –mix green and red light to obtain the perception of yellow, without shining a single yellow light 520nm630nm 570nm = “Seeing yellow” figure is from B.Liu ELE330 S’01 lecture notes @ Princeton; primary color figure is from Chapter 6 slides at Gonzalez/ Woods DIP book website

28. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [28] Reference Documentation of Matlab Image Processing Toolbox, http://www.mathworks.com/access/helpdesk/help/toolbox/images/images.shtml

29. ENEE631 Digital Image Processing (Spring'04) Lec1 – Introduction [29]