1. Image Processing 1 Digital Image Processing Teacher Assistant: Elhanan Elboher course email : impr@cs.huji.ac.il personal email : elhanan.elboher@mail.huji.ac.ilimpr@cs.huji.ac.il Reception hour: Thursday 14:00-15:00 Web site: www.cs.huji.ac.il/~impr

2. Image Processing 2 Administration All exercises are mandatory 3 Exams and Moed B Please register online asap!

3. Image Processing 3 Today Outline Matlab Basics Intensity transform and Histogram Equalization Exercise #1 – Basic Image Processing

4. Image Processing 4 Matlab Desktop To operate Matlab : type matlab from linux xterm Matlab 7.8 is the current default version

5. Image Processing 5 Matlab Advantages Every variable in Matlab is a multidimensional matrix. Highly modular. No memory allocation is necessary. Matlab enables its own garbage collection. Simple interface for complex mathematical concepts.

6. Image Processing 6 Getting Help >> help help (explain how to get help) >> helpbrowser / doc (open the online Matlab documentation) >> help images (list of all commands in the Image Processing Toolbox) >> demo >> lookfor read (display list of function with ‘read’ in the name or help text)‏ >> type imread >> help imread (function name + % block)‏ >> doc imread (function documentation in help browser)‏ Using [tab] is useful to auto-complete function names and variables Mathworks website: http://www.mathworks.com/products/matlab

7. Image Processing 7 Matlab Basics Digital image representation : 2D function f(x,y) -> finite discrete quantities Coordinate Conventions img(r,c) r – rows (height) c – cols (width)‏ >> size(img) The first pixel: img(1,1)‏

8. Image Processing 8 Image Types Intensity images scaled to represent intensities (uint8 – [0,255], double [0,1]) Binary images logical array of 0s and 1s Indexed images Look up table [x, map] RGB images truecolor, array of (m*n*3)‏ Checking the image type : isind, isbw, isgray, isrgb Converting image types: rgb2ind, rgb2gray, gray2ind, ind2gray,….

9. Image Processing 9 Reading Images >> f = imread(‘filename’); filename is a string including the file type (jpg, tiff, bmp, gif,…)‏ ; is used for suppressing output >> [height, width] = size(f); >> whos f display additional information about an array Name Size Bytes Class f 512x512x3 786432 uint8 array Grand total is 786432 elements using 786432 byte

10. Image Processing 10 Displaying Images >> imshow(f)‏ display the image f according to its type >> imshow(f, [low high]) display as black all values less than ‘low’ and as white all values greater or equal to ‘high’ (in grayscale images)‏ >> imshow(f, []) set low and high as the minimal and maximal values of array f useful for low dynamic range images or that have negative values >> impixelinfo display intensity value of individual pixel interactively >> figure(2), imshow(g)‏ Open a new figure before displaying the image (the default – using the same figure)‏

11. Image Processing 11 Writing Images >> imwrite(f, ‘filename’) f is an image array ‘filename’ must include the file format (tif, jpg, bmp,..)‏ >> k = imfinfo(‘test1.jpg’) Filename: 'test1.jpg' FileModDate: '22-Oct-2005 13:07:36' FileSize: 3464 Format: 'jpg' FormatVersion: '' Width: 256 Height: 256 BitDepth: 24 ColorType: 'truecolor' FormatSignature: '' Comment: {} The answer is a structure variable with different fields: k.Width

12. Image Processing 12 Data Classes Converting between types : B = data_class_name(A) for example: B = double(A)‏

13. Image Processing 13 Conversions When converting between data classes and types it is important to keep the value range for each data class >> img = double(img)/255; >> img = im2double(img);

14. Image Processing 14 Variable Decleration and Memory issues Matlab variables do not need to be declared in advance. ‘ans’ is a defined variable containing the last result Memory is allocated and freed automatically. >> A = [1 2 3; 2 3 4; 3 4 5]; >> A = double(A); >> A = 0.0005 ans = 0.0005 >> A = logical(ans);

15. Image Processing 15 Vector indexing row vector (1xN) >> v = [1 3 5 7]; (elements separated by space or comma (,))‏ >> v(2) = 3; column vector (MX1)‏ >> w = [1;3;5;7]; (elements separated semi-comma (;))‏ >> w = v’ (transpose operation) w = 1 3 5 7 To Access blocks of elements we use colon notation >> v(2:4)‏ ans = 3 5 7 >> v(1:end) end is the last element in the vector >> v(:) produce a column vector >> v(1:2:end) enables steps (jumps)‏ >> v(end:-2:1) steps can be negative as well Vector can be used as an index into another vector >> v([1 3 4])‏ ans = 1 5 7

16. Image Processing 16 Matrix indexing Image – 2D array, matrix Matrix can be represented as a sequence of row vectors >>A = [1 2 3; 4 5 6; 7 8 9] A = 1 2 3 4 5 6 7 8 9 To access an element, 2 indexes are used – row index and column index >> A(2,3)  6 >> A(:,3) 3 6 9 >> A(2,:) 4 5 6 >> a(1:2,1:3) 1 2 3 4 5 6 >> B = A; >> B(:,3) = 0 B = 1 2 0 4 5 0 7 8 0 Using vectors to index into a matrix provide a powerful tool for element selection A([1 3], [2 3]) 2 3 8 9

17. Image Processing 17 Matrix indexing cont. Image – 2D array, matrix A matrix is also represented as a long vector >> A = [1 2 3; 4 5 6; 7 8 9] A = 1 2 3 4 5 6 7 8 9 To access sequential elements in a matrix a single index can also be used: >> A(1:9)‏ A = 1 4 7 2 5 8 3 6 9 Sometimes changing the shape of the matrix is of use: >> reshapse(A(1:2,1:3), 3, 2)‏ A = 1 5 4 3 2 6 The size of the output matrix is the size of the index matrix!

18. Image Processing 18 Matrix Addressing A very useful approach is to use logical matrix as an index to the matrix >> D = logical([1 0 0; 0 0 1; 0 0 0])‏ D = 1 0 0 0 0 1 0 0 0 >> A(D)‏ ans = 1 6 The use of column operation on a matrix produce a single column vector from the matrix (on a column by column basis). It is very useful for image operations like sum or max >> s = sum(f(:)) (equivalent to: sum(sum(f)))‏

19. Image Processing 19 Operators Arithmetical operators have their algebraic meaning: >> A = [1 2 3; 4 5 6; 7 8 9]; >> A + A ans = 2 4 6 8 10 12 14 16 18 >> A * [1 1 1] ??? Error using ==> times Matrix dimensions must agree >> A * [1 1 1]’ ans = 6 15 24

20. Image Processing 20 Operators cont. Vector multiplication depends on the order of the two vectors >> [1 2 3] * [2 3 4] ??? Error using ==> times Matrix dimensions must agree >> [1 2 3] * [2 3 4]’ ans = 20 >>[1 2 3]’ * [2 3 4] ans = 2 3 4 4 6 8 6 9 12

21. Image Processing 21 Operators cont. Element by element operators -.*.^./ >> [1 2 3].* [2 3 4] ans = 2 6 12 >> [1 2 3].^ 2 ans = 1 4 9 >> [1 2 3]./ (2:2:6)‏ ans = 0.5000 0.5000 0.5000

22. Image Processing 22 Ambiguity Since each variable is a multi dimensional matrix, matlab solves ambiguity on its own – variables with no dimension can be seen as vectors/matrices >> [1 2] + 1 ans = 2 3

23. Image Processing 23 Array dimensions Matlab arrays can be of any dimensions It is useful to operate on specific dimension of the array, for example: >> height = size(i,1); Usually we deal with 2D arrays but there are cases we need to address higher dimensions (such as color images)‏ >> i(200:300, 200:400, 3)‏ To get the number of dimensions of an array >> d = ndims(f)‏

24. Image Processing 24 Standard Arrays Generating simple array enables trying out simple ideas and test the syntax of a function during development >> zeros(m,n)‏ >> ones(m,n)‏ >> true(m,n)‏ >> false(m,n)‏ >> magic(m) >> rand(n)‏ >> randn(n)‏ >> pascal(n)‏

25. Image Processing 25 Additional Operators Arithmetic operators (numeric computations)‏ –matrix arithmetic (linear algebra A*B)‏ –array arithmetic (element by element A.*B) +, -,./,.^,:.. Relational operators (compare)‏ –Compare corresponding elements of arrays of equal dimensions (, =, ==, ~=) or an array to scalar Logical operators can operate both on logical and numeric data (and: &, or: |, not: ~) true: logical 1 or non-zero numeric quantity false: logical or numerical 0 logical functions : xor, any, all

26. Image Processing 26 Examples - Matrix indexing >> i = imread('sowrds0040.bmp'); >> i = rgb2gray(double(i)/255); >> imshow(i)‏

27. Image Processing 27 Examples - Matrix indexing >> i = imread('sowrds0040.bmp'); >> i = rgb2gray(double(i)/255); >> imshow(i)‏

28. Image Processing 28 Examples - Matrix indexing >> i = imread('sowrds0040.bmp'); >> i = rgb2gray(double(i)/255); >> imshow(i)‏ >> s = i(end:-1:1, :); >> imshow(s);

29. Image Processing 29 Examples - Matrix indexing >> i = imread('sowrds0040.bmp'); >> i = rgb2gray(double(i)/255); >> imshow(i)‏ >> s = i(end:-1:1,:); >> imshow(s);

30. Image Processing 30 Examples - Matrix indexing >> i = imread('sowrds0040.bmp'); >> i = rgb2gray(double(i)/255); >> imshow(i)‏ >> s = i(end:-1:1,:); >> imshow(s); >> a = i(200:300,200:400); >> imshow(a);

31. Image Processing 31 Examples - Matrix indexing >> i = imread('sowrds0040.bmp'); >> i = rgb2gray(double(i)/255); >> imshow(i)‏ >> s = i(end:-1:1,:); >> imshow(s); >> a = i(200:300,200:400); >> imshow(a);

32. Image Processing 32 Examples - Matrix indexing >> i = imread('sowrds0040.bmp'); >> i = rgb2gray(double(i)/255); >> imshow(i)‏ >> s = i(end:-1:1,:); >> imshow(s); >> a = i(200:300,200:400); >> imshow(a); >> t = i(1:2:end, 1:2:end); >> imshow(t);

33. Image Processing 33 Examples - Matrix indexing >> i = imread('sowrds0040.bmp'); >> i = rgb2gray(double(i)/255); >> imshow(i)‏ >> s = i(end:-1:1,:); >> imshow(s); >> a = i(200:300,200:400); >> imshow(a); >> t = i(1:2:end, 1:2:end); >> imshow(t);

34. Image Processing 34 Examples - Matrix indexing >> i = imread('sowrds0040.bmp'); >> i = rgb2gray(double(i)/255); >> imshow(i)‏ >> s = i(end:-1:1,:); >> imshow(s); >> a = i(200:300,200:400); >> imshow(a); >> t = i(1:2:end, 1:2:end); >> imshow(t); >> i(200:300, 200:400) = 0; >> imshow(i);

35. Image Processing 35 Examples - Matrix indexing >> i = imread('sowrds0040.bmp'); >> i = rgb2gray(double(i)/255); >> imshow(i)‏ >> s = i(end:-1:1,:); >> imshow(s); >> a = i(200:300,200:400); >> imshow(a); >> t = i(1:2:end, 1:2:end); >> imshow(t); >> i(200:300, 200:400) = 0; >> imshow(i);

36. Image Processing 36 Examples - Matrix indexing >> i = imread('sowrds0040.bmp'); >> i = rgb2gray(double(i)/255); >> imshow(i)‏ >> s = i(end:-1:1,:); >> imshow(s); >> a = i(200:300,200:400); >> imshow(a); >> t = i(1:2:end, 1:2:end); >> imshow(t); >> i(200:300, 200:400) = 0; >> imshow(i); >> imshow(i/2);

37. Image Processing 37 Examples - Matrix indexing >> i = imread('sowrds0040.bmp'); >> i = rgb2gray(double(i)/255); >> imshow(i)‏ >> s = i(end:-1:1,:); >> imshow(s); >> a = i(200:300,200:400); >> imshow(a); >> t = i(1:2:end, 1:2:end); >> imshow(t); >> i(200:300, 200:400) = 0; >> imshow(i); >> imshow(i/2);

38. Image Processing 38 Examples - Matrix indexing >> i = imread('sowrds0040.bmp'); >> i = rgb2gray(double(i)/255); >> imshow(i)‏ >> s = i(end:-1:1,:); >> imshow(s); >> a = i(200:300,200:400); >> imshow(a); >> t = i(1:2:end, 1:2:end); >> imshow(t); >> i(200:300, 200:400) = 0; >> imshow(i); >> imshow(i/2); >> imshow((i>0.8).*i);

39. Image Processing 39 Examples - Matrix indexing >> i = imread('sowrds0040.bmp'); >> i = rgb2gray(double(i)/255); >> imshow(i)‏ >> s = i(end:-1:1,:); >> imshow(s); >> a = i(200:300,200:400); >> imshow(a); >> t = i(1:2:end, 1:2:end); >> imshow(t); >> i(200:300, 200:400) = 0; >> imshow(i); >> imshow(i/2); >> imshow((i>0.8).*i);

40. Image Processing 40 M-Files M-Files can be one of two: Scripts – A series of commands that are performed on the global scope. No input and output variables. Functions – A set of commands performed over a given input, with a required output. Functions have a scope of their own. (accessing the global scope can be done by defining the variables to be ‘globals’). In both types – the m-file must be in the current directory, or in a previously added path (added with the function addpath)

41. Image Processing 41 M-Function Programming Components of m files: Function definition line function [out1 out2] = name(in1, in2, in3)‏ H1 line - a single comment line that follows the function definition line. % SQUARESUM compute the sum of the square of the matrix elements This line appears when user writes >> help function_name >> lookfor keyword - display all functions where the keyword appeared in H1 line

42. Image Processing 42 M-Function Programming Components of m files (cont.): Help Text - text block following the H1 line without any blank line in between the two Function body – the Matlab code Comments – lines starting with % Note: – add short and clear comments to your code!

43. Image Processing 43 Flow control if, else, elseif, end switch, case, otherwise, end return try...catch…end for i=start:increment:end, end while, end break (used with for or while)‏ continue (used with for or while)‏ Try not to use

44. Image Processing 44 Code optimization – vectorizing loops Convert for / while loops to equivalent vector or matrix operations 1D indexing >> for x = 1:k ff(x) = 5*sin((x-1)/(2*pi)); end 2D indexing Try to avoid 2D loops >> x = 0:k-1 >> ff = 5*sin(x/(2*pi));

45. Image Processing 45 Code optimization – vectorizing loops 2D indexing meshgrid – convert rows vectors to arrays C and R that can be used for evaluating function with two variables >> for r = 1:10 >> for c = 1:10 >> b(r,c) = r.^2+ c.^2 >> end Vectorzing code accelerates the computation significantly For Example: compute 2D sin using meshgrid runs on the order of 30 times faster the same computation based on loops on Image of 512x512 pixels >> [C, R] = meshgrid(1:c, 1:r)‏ >> h = R.^2 + C.^2;

46. Image Processing 46 Code Optimization – Pre-allocating large arrays Simple way to improve code execution is to pre-allocate the size of the arrays in the program. The preallocation also helps reduce memory fragmentation when working with large matrices >> f = zeros(1024);

47. Image Processing 47 Cell arrays and Structures Cell array is multidimensional array whose elements are copies of other arrays >> c = {‘gauss’,[1 0;0 1], 3} >> c{1} ans = gauss Structures are similar to cell arrays (allow grouping of a collection of dissimilar data) but they addressed by fields rather than by numbers >> params.nimgs = 100; >> params.jump = 2; >> params.baseStr = ‘testImg’

48. Image Processing 48 Arguments Matlab arguments are always passed by value Checking whether an argument exist >> exist(a,’var’)‏ Checking number of arguments to the functions >> nargin, nargout, nargchk Getting variable number of arguments >>varargin, varargout

49. Image Processing 49 Gui >> guide (Graphic User Interface Development Environment) Start the GUI Layout Editor. Guide create –fig file: complete description of the gui elements and their arrangements –gui m-file: the code that controls the gui operations, initializations functions, callback functions

50. Image Processing 50 Image Enhancement

51. Image Processing 51 Intensity Transformation Spatial Domain – image plane itself direct manipulation of pixels in the image. g(x,y) = T(f(x,y))‏ T is an operator of f, defined over a specific neighborhood around a pixel. Principle approach – use a square around the pixel. In the simplest case the square size is one pixel

52. Image Processing 52 Intensity Transformation function In the case of single pixel transformation – we can use a lookup table s = T(r) 0 1 2 3........ 255 s1s1 s0s0 s2s2 s3s3 s 255 r s Dark light

53. Image Processing 53 Simple Examples for Intensity Transformations Image negative s = L-1-r r s Dark light

54. Image Processing 54 Simple Examples for Intensity Transformations Image negative s = L-1-r r s Dark light

55. Image Processing 55 Examples for Intensity Transformations Log Transform s = c*log(1+r)‏ r s Dark light

56. Image Processing 56 Examples for Intensity Transformations Power Law Transform

57. Image Processing 57 Example - Gamma Correction What kind of correction can we try here ?

58. Image Processing 58 Example - Gamma Correction γ=3, 4, 5

59. Image Processing 59 Histogram Processing h(r k ) = n k Normalized: p(r k ) = n k /N 01255 in Matlab: imhist(img)‏ +

60. Image Processing 60 Sample of Image Histogram

61. Image Processing 61 Histogram Equalization We are interested in equal use of all gray level N pixels, range 0,..K-1 n k – number of pixels in level k 255 graylevels #p in level 0 histogram 1 graylevels #p < level 0 Normalized Accumulative histogram graylevels #p < level 0 Accumulative histogram 255 r k =k/K sksk 1

62. Image Processing 62 Histogram Equalization Equalize :For every original gray level k –Calculate the image histogram –Find the accumulative sum of the histogram values - y k (in Matlab – cumsum(vec))‏ –Normalize the values of the histogram accumulative sum by dividing in the total number of pixels –Multiply the normalized vector in the maximal gray level value (K-1) and round (shift back to the original gray level range)‏ –Map the gray level values to the result of step (3)‏ –Stretch the values back to the range 1,..K (improve contrast in the end of this process)‏ 1 #p < level 0 Normalized Accumulative histogram r k =k/K sksk rkrk sksk 1

63. Image Processing 63 Histogram Equalization In most discrete images it is impossible to obtain uniform histogram, we want to get as close as possible to uniform. The distribution of gray levels in areas along the histogram will result in approximately same number of pixels The number of bins can only decrease: bins can never split and small bins in the histogram may be united (but the effect is not visibly disturbing since it involves a very small number of pixels)‏ The histogram equalization process is monotonic (as the process involves accumulative sum) and therefore the relative brightness of a pixel is preserved

64. Image Processing 64 When will this fail ? In General : on images with gray level distribution that is not ‘standard’ we may get unwanted results from the histogram equalization process.

65. Image Processing 65 Equalization results

66. Image Processing 66 Exercise 1 Targets: –Getting familiar with the Matlab environment –Learning image formats and conversions between them –Argument error checking –Working with image histograms –Writing an efficient code

67. Image Processing 67 A few more important commands for image operations (look in the help) ‏ find repmat reshape clear save plot, subplot disp