Download presentation

Presentation is loading. Please wait.

Published byLauryn Allmand Modified about 1 year ago

1
National Center for Supercomputing Applications University of Illinois at Urbana-Champaign Image Feature Descriptors Kenton McHenry, Ph.D. Research Scientist

2
Raster Images [Hoiem, 2012] image(234, 452) = 0.58

3
Neighborhoods of Pixels For nearby surface points most factors do not change much Local differences in brightness [Hoiem, 2012]

4
Features Interest points Points or collections of points that are somehow relevant to identifying the contents of an image Subset of the image Far less data than the total number of pixels Repeatability A feature detector should be likely to detect the feature in a scene under a variety of lighting, orientation, and other variable conditions. Descriptors A relatively short numerical representation that can be used to identify a specific feature instance

5
Descriptors, Why? Matching: Match an object in two images based on similar features 3D Reconstruction: Stereopsis Tracking: Follow an object in a video by following its features Object Recognition: Find objects based on known features it will posses Segmentation: Break an image up into more meaningful regions based on the seen features.

6
Feature Descriptors

7
Feature Descriptors for Edges Lines: (m, b) (r, ) Circles: r 2 = (x 2 - a 2 ) + (y 2 - b 2 ) (a, b, r) Other shapes Follow lines to make curves

8
Feature Descriptors for Curves

9
Curve length Curvature Curves smoothness

10
Feature Descriptors for Curves

11

12
Curve length Curvature Curves smoothness Corners Concave polygons Convex polygons

13

14
Feature Descriptors for Curves Curve length Curvature Curves smoothness Corners Concave polygons Convex polygons Periodicity Frequency …

15
Feature Descriptors for Regions

16
Matching Regions Two views of the same object Find matching regions

17
Matching Regions

18
Template Matching If regions are the same size we can again treat them as vectors and compare them Sum of Squared Differences (SSD) d = (a 1 -b 1 ) 2 + (a 2 -b 2 ) 2 + … Euclidean Distance without the square root Slightly faster and closer things still have smaller values Normalized Cross Correlation d = a/|a|. b/|b| More invariant to changes in intensity Values between -1 and 1

19
Template Matching: SSD >r0 = mean(imread(‘scar1_region2.png’), 3); >imagesc(r0); >r1 = mean(imread(‘scar2_region1.png’), 3); >r2 = mean(imread(‘scar2_region2.png’), 3); >r3 = mean(imread(‘scar2_region3.png’), 3); >sum((r1(:)-r0(:)).^2) >sum((r2(:)-r0(:)).^2) >sum((r3(:)-r0(:)).^2)

20
Template Matching: NCC >r0 = r0(:) / norm(r0(:)); >r1 = r1(:) / norm(r1(:)); >r2 = r2(:) / norm(r2(:)); >r3 = r3(:) / norm(r3(:)); >dot(r0,r1) >dot(r0,r2) >dot(r0,r3)

21
Color Average Color Treat the entire regions as one color Coarse representation Regions don’t have to be the same size

22
Color >r0 = imread(‘scar1_region2.png’); >image(r0); >whos >mean(mean(r0, 1), 2) >c0 = reshape(mean(r0, 1), 2), 1, 3); >image(uint8(reshape(c0, 1, 1, 3)));

23
Color >r1 = imread(‘scar2_region1.png’); >image(r1); >c1 = reshape(mean(r1, 1), 2), 1, 3); >image(uint8(reshape(c1, 1, 1, 3))); >r2 = imread(‘scar2_region2.png’); >image(r2); >c2 = reshape(mean(r2, 1), 2), 1, 3); >image(uint8(mean(r2, 1), 2))); >r3 = imread(‘scar2_region3.png’); >image(r3); >c3 = reshape(mean(r3, 1), 2), 1, 3); >image(uint8(mean(r3, 1), 2)));

24
Color >sqrt(sum((c1-c0).^2)) >sqrt(sum((c2-c0).^2)) >sqrt(sum((c3-c0).^2))

25
Color Average Color Treats the entire regions as one color Coarse representation Regions don’t have to be the same size Color Distributions Build a distribution of colors in a region A bit more detail Regions still don’t have to be the same size

26
Color >r0 = imread(‘scar1_region2.png’); >r = r0(:,:,1); >g = r0(:,:,2); >b = r0(:,:,3); >hr = hist(double(r(:)), 10); >hg = hist(double(g(:)), 10); >hb = hist(double(b(:)), 10); >h0 = [hr hg hb]; >h0 = h0 / norm(h0);

27
Color >r1 = imread(‘scar2_region1.png’); >… >r2 = imread(‘scar2_region2.png’); >… >r3 = imread(‘scar2_region3.png’); >… >sqrt(sum((h1-h0).^2)) >sqrt(sum((h2-h0).^2)) >sqrt(sum((h3-h0).^2))

28
Color Spaces L Human studies to model color similarity Euclidean distances between values better matches our perception HSV Hue: The colors Saturation: The purity of the color Value: The brightness, a black and white version of the image Matlab: rgb2hsv, hsv2rgb

29
Texture

30
Break the universe of textures into individual elements of texture, textons Use these textons as patterns to look for in the image and use convolution to find them Gaussian Filters In particular variations of Gaussian filters

31
Texture: Orientations >filter = fspecial(‘guassian’, 50, 5); >filter = diff(filter, 1, 2); >filter = filter / norm(filter); >imagesc(filter); >filter = imrotate(filter, 45); >imagesc(filter);

32
Texture: Elongations >f = fspecial(‘guassian’, 50, 5); >f = diff(f, 1, 2); >f = imresize(f, [2*size(f,1) size(f,2)]); >f = f / norm(f); >imagesc(f);

33
Texture: Miscellaneous >filter = fspecial(‘guassian’, 50, 5); >filter = diff(filter, 1, 2); >filter = filter / norm(filter); >imagesc(filter); >filter = imrotate(filter, 45); >imagesc(filter); >filter = fspeical(‘log’, 50, 5); Bars Spots

34
Texture: Scales >filter = … >filter = imresize(filter, 0.5); >filter = filter / norm(filter); >imagesc(filter);

35
Texture Filter bank Create many filters spanning: Orientations Elongations Types Scales

36
Texture >Ig = mean(imread(‘scar1.jpg’), 3); >Ig = Ig / norm(Ig); >f = fspecial(‘gaussian’, 16, 2); >f1 = diff(f, 1, 2); >f1 = f1 / norm(f1); >f2 = imrotate(f1, 45, ‘bilinear’, ‘crop’); >f2 = f2 / norm(f2); >f3 = imrotate(f2, 45, ‘bilinear’, ‘crop’); >f3 = f3 / norm(f3); >Ir1 = abs(conv2(Ig, f1)); >Ir2 = abs(conv2(Ig, f1)); >Ir3 = abs(conv2(Ig, f1));

37
Texture >Ig = mean(imread(‘textures.png’), 3); >Ig = Ig / norm(Ig); >f = fspecial(‘gaussian’, 16, 2); >f1 = diff(f, 1, 2); >f1 = f1 / norm(f1); >f2 = imrotate(f1, 45, ‘bilinear’, ‘crop’); >f2 = f2 / norm(f2); >f3 = imrotate(f2, 45, ‘bilinear’, ‘crop’); >f3 = f3 / norm(f3); >Ir1 = abs(conv2(Ig, f1)); >Ir2 = abs(conv2(Ig, f1)); >Ir3 = abs(conv2(Ig, f1));

38
Texture >r1 = Ir1(1:100,1:100); >r2 = Ir2(1:100,1:100); >r3 = Ir3(1:100,1:100); >h1 = hist(r1(:), 10); >h2 = hist(r2(:), 10); >h3 = hist(r3(:), 10); >ht1 = [h1 h2 h3]; >ht1 = ht1 / norm(ht1); >r1 = Ir1(1:100,100:200); >… >r1 = Ir1(1:100,200:300); >…

39
Texture >sqrt(sum((ht1-ht2).^2)) >sqrt(sum((ht1-ht3).^2)) >sqrt(sum((ht2-ht3).^2))

40
Miscellaneous Size Count the number of pixels Statistics Calculate mean and standard deviation in x and y directions Neighbors …

41
Image and Spatial Data Analysis Group Questions?

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google