Presentation is loading. Please wait.

Presentation is loading. Please wait.

0 By: Navid Einackchi Isfahan University of Technology Electrical and Computer Department Spring 2007 Image alignment and stitching using Object Recognition.

Similar presentations


Presentation on theme: "0 By: Navid Einackchi Isfahan University of Technology Electrical and Computer Department Spring 2007 Image alignment and stitching using Object Recognition."— Presentation transcript:

1 0 By: Navid Einackchi Isfahan University of Technology Electrical and Computer Department Spring 2007 Image alignment and stitching using Object Recognition Methods Supervisors: Dr Rasoul Amirfattahi Dr javad Askari Advisor: Dr M. Saraee Master Thesis of Computer Engineering- Artificial Intelligence and Robotic به نام خدا

2 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Presentation Process IntroductionIntroduction –Definition –Aligning concepts –Stitching Concepts Object Recognition ProblemObject Recognition Problem –Salient Feature –What is Descriptor –Matching Proposed Method for AligningProposed Method for Aligning –Using Harris Detector –Using SIFT Detector –Stitching Concluding and SuggestionsConcluding and Suggestions Some slides are from other sources

3 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Presentation Process IntroductionIntroduction –Definition –Aligning Concepts –Stitching Concepts Object Recognition ProblemObject Recognition Problem –Salient Feature –What is Descriptor –Matching Proposed Method for AligningProposed Method for Aligning –Using Harris Detector –Using SIFT Detector –Stitching Concluding and SuggestionsConcluding and Suggestions

4 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Presentation Process IntroductionIntroduction –Definition –Aligning Concepts –Stitching Concepts Object Recognition ProblemObject Recognition Problem –Salient Feature –What is Descriptor –Matching Proposed Method for AligningProposed Method for Aligning –Using Harris Detector –Using SIFT Detector –Stitching Concluding and SuggestionsConcluding and Suggestions

5 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Image Alignment and Stitching DefinitionDefinition ApplicationsApplications –Mosaic Image –Panorama –Virtual Environment –Robotic تصوير حاصل

6 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Image Alignment and Stitching DefinitionDefinition ApplicationsApplications –Mosaic Image –Panorama –Virtual Environment –Robotic

7 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Image Alignment and Stitching DefinitionDefinition ApplicationsApplications –Mosaic Image –Panorama –Virtual Environment –Robotic

8 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Presentation Process IntroductionIntroduction –Definition –Aligning Concepts –Stitching Concepts Object Recognition ProblemObject Recognition Problem –Salient Feature –What is Descriptor –Matching Proposed Method for AligningProposed Method for Aligning –Using Harris Detector –Using SIFT Detector –Stitching Concluding and SuggestionsConcluding and Suggestions

9 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Aligning Concepts Image TransformationsImage Transformations –Applicable Transformations Find transformationsFind transformations –How many parameters? –Calculating Parameters

10 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Image Transformations TransitionTransition –Number of Parameters: 2 –Number of Points: 1

11 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Image Transformations EuclideanEuclidean –Number of Parameters: 3 –Number of Points: 2

12 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Image Transformations SimilaritySimilarity –Number of Parameters: 4 –Number of Points: 2

13 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Image Transformations AffineAffine –Number of Parameters: 6 –Number of Points: 3

14 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Image Transformations PerspectivePerspective –Number of Parameters: 8 –Number of Points: 4 1

15 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Image Transformation Computation DirectDirect –Mapping one image into another using different parameters –Define an Error Function based on pixel intensity difference Using Corresponding pointsUsing Corresponding points –Obtaining Corresponding points –Computing Transformation parameters using corresponding points

16 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Direct Method Error Function

17 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Direct Method Error Function ؟

18 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Direct Method

19 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Using Corresponding points

20 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Using Corresponding points

21 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Using Corresponding points

22 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods How many points?How many points? How to select?How to select? –Using human –Automatically Using Corresponding points

23 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Presentation Process IntroductionIntroduction –Definition –Aligning Concepts –Stitching Concepts Object Recognition ProblemObject Recognition Problem –Salient Feature –What is Descriptor –Matching Proposed Method for AligningProposed Method for Aligning –Using Harris Detector –Using SIFT Detector –Stitching Concluding and SuggestionsConcluding and Suggestions

24 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Stitching StitchingStitching ChoicesChoices –Final Plane –Pixel weighting

25 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Presentation Process IntroductionIntroduction –Definition –Aligning Concepts –Stitching Concepts Object Recognition ProblemObject Recognition Problem –Salient Feature –What is Descriptor –Matching Proposed Method for AligningProposed Method for Aligning –Using Harris Detector –Using SIFT Detector –Stitching Concluding and SuggestionsConcluding and Suggestions

26 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Object Recognition Methods Local Feature DetectionLocal Feature Detection –Edge –Corner –Hole Interest Regions MatchingInterest Regions Matching –Appearance Matching –Geometric Matching Finding Object Relations in ImageFinding Object Relations in Image

27 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Presentation Process IntroductionIntroduction –Definition –Aligning Concepts –Stitching Concepts Object Recognition ProblemObject Recognition Problem –Salient Feature –What is Descriptor –Matching Proposed Method for AligningProposed Method for Aligning –Using Harris Detector –Using SIFT Detector –Stitching Concluding and SuggestionsConcluding and Suggestions

28 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Salient Features Features of Feature!Features of Feature! –Invariants against transformations –Ability to explain the image –Ability to being Matched (Repeatability) Selected FeaturesSelected Features –Corners –Holes

29 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Detection Harris DetectorHarris Detector –Corners –Invariant against intensity and view changes –Invariant against transition and rotation transformations Scale Invariant Feature Transform (SIFT)Scale Invariant Feature Transform (SIFT) –Holes –Invariant against intensity and view changes –Invariant against transition and rotation transformations –Invariant against Scale transformation

30 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Harris Detector Plain No changes in every direction Edge High changes in Edge Direction Corner High changes in every direction

31 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Harris Detector (Continued) Slow change direction Fast change Direction ( max ) -1/2 ( min ) -1/2

32 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Harris Detector (Continued) 1 2 both big “Corner” 1, 2 both big 1 ~ 2 Edge 1 >> 2 Edge 2 >> 1 Plain

33 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Harris Detector (Continued) Harris measurementHarris measurement

34 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Harris Detector (Continued) Harris measurement responseHarris measurement response 1 2 “Corner” “Edge” “Plain” R > 0 R < 0 |R| small Local Maximum bigger than Threshold

35 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Harris Detector (Continued) FeaturesFeatures –Invariant against rotation –Invariant against intensity R x Threshold R x

36 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Harris Detector (Continued) ExampleExample

37 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Scale ProblemScale Problem Scale Invariant

38 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Scale Invariant Finding Appropriate Window Size

39 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Scale Selection Using Differential of Gaussian FiltersUsing Differential of Gaussian Filters –Laplacian of Gaussian (LoG) –Difference of Gaussian (DoG) - = Window thst maximize filter

40 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods

41 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Automatic Scale Selection

42 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Automatic Scale Selection

43 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods SIFT Detector Bulb DetectorBulb Detector Invariant against scaleInvariant against scale Using DoGUsing DoG –Bulb detection –Scale detection مقیاس x y  DoG 

44 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods SIFT Detector 2kσ 2σ kσ σ 2k 2 σ 2kσ 2σ kσ σ DoG PyramidDoG Pyramid

45 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods SIFT Detector Detecting Bulb and Scale SimultaneouslyDetecting Bulb and Scale Simultaneously –Bulb position = maximum in plain –Scale = maximum in third dimension Bulb position Scale position

46 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods

47 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Presentation Process IntroductionIntroduction –Definition –Aligning Concepts –Stitching Concepts Object Recognition ProblemObject Recognition Problem –Salient Feature –What is Descriptor –Matching Proposed Method for AligningProposed Method for Aligning –Using Harris Detector –Using SIFT Detector –Stitching Concluding and SuggestionsConcluding and Suggestions

48 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Descriptor Vector which describes region of interest pointVector which describes region of interest point –Pixels –Histogram –Differential Feature of DescriptorFeature of Descriptor –Invariant against image distortion (view, angle, intensity) –Able to express similarities and differences

49 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods SIFT Descriptor Based on value and direction of gradientsBased on value and direction of gradients 128 dimensional Vector128 dimensional Vector ConstructionConstruction 1.Calculation of value and direction of gradient of each pixel 2.Making direction histogram of gradients 3.Rotating interest region based on dominant direction 4.Dividing the region into 16 region 5.Constructing direction histogram for each region 0 

50 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Construction Example

51 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Descriptor Features Invariant againstInvariant against –Rotation –Position –Affine changes in intensity ( I  Ia + b)

52 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Presentation Process IntroductionIntroduction –Definition –Aligning Concepts –Stitching Concepts Object Recognition ProblemObject Recognition Problem –Salient Feature –What is Descriptor –Matching Proposed Method for AligningProposed Method for Aligning –Using Harris Detector –Using SIFT Detector –Stitching Concluding and SuggestionsConcluding and Suggestions

53 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Matching Appearance MatchingAppearance Matching –Comparing distance of each two descriptors –Euclidean Distance –K nearest neighbor Geometrical MatchingGeometrical Matching –Based on geometrical relations between points –Estimation of image transformation –Rejecting false corresponding points

54 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Example of Geometrical Matching

55 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Automatically image alignment Direct MethodDirect Method Based on interest pointsBased on interest points

56 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Compare Interest points Direct Method Low (Order of number of interest points) High (order of number of pixels) Time and Memory order HighLow (Based on pixel intensity) Resisting against intensity changes YesNo Ability to recognize corresponding image

57 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Presentation Process IntroductionIntroduction –Definition –Aligning Concepts –Stitching Concepts Object Recognition ProblemObject Recognition Problem –Salient Feature –What is Descriptor –Matching Proposed Method for AligningProposed Method for Aligning –Using Harris Detector –Using SIFT Detector –Stitching Concluding and SuggestionsConcluding and Suggestions

58 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Brown and Lowe Algorithm O(n 4 m 4 )

59 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Brown and Lowe Algorithm

60 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Brown and Lowe Algorithm

61 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Similarity Transform rather than perspective WhyWhy –Detector are unable to handle perspective –Small portion of each images is overlapped –Increasing the tolerance of true corresponding So We use Similarity TransformationSo We use Similarity Transformation

62 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Presentation Process IntroductionIntroduction –Definition –Aligning Concepts –Stitching Concepts Object Recognition ProblemObject Recognition Problem –Salient Feature –What is Descriptor –Matching Proposed Method for AligningProposed Method for Aligning –Using Harris Detector –Using SIFT Detector –Stitching Concluding and SuggestionsConcluding and Suggestions

63 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods First Proposed Algorithm 1.Detecting Interest points using Harris detector 2.Defining descriptors for each point 3.Finding corresponding points using appearance Comparing interest points with each other K Nearest Neighbor 4.Finding relation between 2 images Using Similarity Based on voting 5.Rejecting wrong corresponding points based on geometrical relations

64 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods First Proposed Algorithm 1.Detecting Interest points using Harris detector 2.Defining descriptors for each point 3.Finding corresponding points using appearance Comparing interest points with each other K Nearest Neighbor 4.Finding relation between 2 images Using Similarity Based on voting 5.Rejecting wrong corresponding points based on geometrical relations

65 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Similarity Finding 1.Select 2 points in first image 1.Draw hypothesis lines between them 2.Find corresponding points in other image 1.Draw hypothesis lines between them 3.Finding relation between lines of each 2 points. 1.Scale 2.Angle 4.Vote to corresponding relation bin –Scale + Angle Bin

66 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Voting Mechanism

67 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Voting Mechanism Scale Angle از مرتبه O(n 2 m 2 )

68 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Applied Rotation Estimated Rotation Applied Scale Estimated Scale Evaluation

69 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Concluding Very good for rotationVery good for rotation Good for little scalingGood for little scaling Weak for big scalingWeak for big scaling –Due to Harris detector weakness

70 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Presentation Process IntroductionIntroduction –Definition –Aligning Concepts –Stitching Concepts Object Recognition ProblemObject Recognition Problem –Salient Feature –What is Descriptor –Matching Proposed Method for AligningProposed Method for Aligning –Using Harris Detector –Using SIFT Detector –Stitching Concluding and SuggestionsConcluding and Suggestions

71 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Improvement Using less time order algorithmUsing less time order algorithm Using Scale Invariant Feature TransformUsing Scale Invariant Feature Transform

72 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Second proposed Algorithm 1.Detecting Interest points SIFT Detector 2.Finding Corresponding points Based on appearance Using k-d tree Distance ratio threshold 3.Finding relation between images Voting on step 2 based on Appearance corresponding 4.Rejecting wrong corresponding points Based on geometric relations between points

73 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Distance Ration Threshold Find 1NN and its distance = D1Find 1NN and its distance = D1 Find 2NN and its distance = D2Find 2NN and its distance = D2 Find ratio of these distances (D1/D2 = ratio)Find ratio of these distances (D1/D2 = ratio) –If Lower than threshold -> reject –Else accept

74 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Distance Ration Threshold My Improvement Find 1NN and its distance = D1Find 1NN and its distance = D1 Multiply D1 by ratio = rangeMultiply D1 by ratio = range Look for range in k-d treeLook for range in k-d tree

75 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Results of threshold on ratio Threshold Ratio = 0.5Threshold Ratio = 0.5

76 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Results of threshold on ratio Threshold Ratio = 0.7Threshold Ratio = 0.7

77 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Second proposed Algorithm 1.Detecting Interest points SIFT Detector 2.Finding Corresponding points Based on appearance Using k-d tree Distance ratio threshold 3.Finding relation between images Voting on step 2 based on Appearance corresponding 4.Rejecting wrong corresponding points Based on geometric relations between points

78 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Each interest point hasEach interest point has –Dominant Direction –Scale Use this information for votingUse this information for voting Estimating Geometrical Relation

79 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Estimation Histogram Using threshold methodUsing 3NN method Scale Direction Scale O(nm) O(n) Maximum

80 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Method Potential Corresponding points True Corresponding Points Percent of True Corresponding Distance Ratio t= % Distance Ratio t= % Distance Ratio t= % Distance Ratio t= % Distance Ratio t= % 2 Nearest Neighbor 710*2→ % 3 Nearest Neighbor 710*3→ % 4 Nearest Neighbor 710*4→ % Comparing Knn and Ratio Threshold setting Based on Number of true corresponding points

81 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Comparing Knn and Ratio Threshold setting Based on Number of true corresponding points Method Potential Corresponding points True Corresponding Points Percent of True Corresponding Distance Ratio t= % Distance Ratio t= % Distance Ratio t= % Distance Ratio t= % Distance Ratio t= % 2 Nearest Neighbor 710*2→ % 3 Nearest Neighbor 710*3→ % 4 Nearest Neighbor 710*4→ %

82 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Method 1st Maximum2nd MaximumDifferenceRatio Distance Ratio t= Distance Ratio t= Distance Ratio t= Nearest Neighbor Nearest Neighbor Nearest Neighbor Comparing Knn and Ratio Threshold setting (In Tolerating False Corresponding)

83 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Comparing Knn and Ratio Threshold setting (In Tolerating False Corresponding) Method 1st Maximum2nd MaximumDifferenceRatio Distance Ratio t= Distance Ratio t= Distance Ratio t= Nearest Neighbor Nearest Neighbor Nearest Neighbor

84 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Applied Rotation Estimated Rotation Applied Scale Estimated Scale Evaluation

85 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Presentation Process IntroductionIntroduction –Definition –Aligning Concepts –Stitching Concepts Object Recognition ProblemObject Recognition Problem –Salient Feature –What is Descriptor –Matching Proposed Method for AligningProposed Method for Aligning –Using Harris Detector –Using SIFT Detector –Stitching Concluding and SuggestionsConcluding and Suggestions

86 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Stitching Pixel Weights based on their distancePixel Weights based on their distance

87 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Stitching

88 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Stitching

89 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Other examples Adobe Photoshop could not stitch these imagesAdobe Photoshop could not stitch these images

90 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Other Examples

91 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Other Examples

92 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Presentation Process IntroductionIntroduction –Definition –Aligning Concepts –Stitching Concepts Object Recognition ProblemObject Recognition Problem –Salient Feature –What is Descriptor –Matching Proposed Method for AligningProposed Method for Aligning –Using Harris Detector –Using SIFT Detector –Stitching Concluding and SuggestionsConcluding and Suggestions

93 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Concluding IdeasIdeas –Using Harris and SIFT as a detector –Using Similarity transform as a general image transform –Using voting based on direction and scale of each interest points –Introducing Evaluation Method ResultsResults –Harris is more reliable but SIFT is much better in presence of scale transform –Similarity can handle prospective transform for human eyes –Voting can obtain similarity transform with much lower time order –Threshold ratio has a better behavior than Knn

94 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods Thanks Questions?

95 By: Navid Einackchi Spring 2007 Image alignment and stitching using object recognition methods


Download ppt "0 By: Navid Einackchi Isfahan University of Technology Electrical and Computer Department Spring 2007 Image alignment and stitching using Object Recognition."

Similar presentations


Ads by Google