Presentation is loading. Please wait.

Presentation is loading. Please wait.

I A f M 2 0 0 6 Martin J. Moene E.H. van Tol-Homan P.V. Ruijgrok T.H. Oosterkamp J.W.M. Frenken M.J. Rost Kamerlingh Onnes Laboratory Image Processing.

Similar presentations


Presentation on theme: "I A f M 2 0 0 6 Martin J. Moene E.H. van Tol-Homan P.V. Ruijgrok T.H. Oosterkamp J.W.M. Frenken M.J. Rost Kamerlingh Onnes Laboratory Image Processing."— Presentation transcript:

1 I A f M Martin J. Moene E.H. van Tol-Homan P.V. Ruijgrok T.H. Oosterkamp J.W.M. Frenken M.J. Rost Kamerlingh Onnes Laboratory Image Processing for Video-rate Scanning Probe Microscopy

2 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Image Processing for Video-rate Scanning Probe Microscopy Martin Moene Interface Physics Leiden University The Netherlands graphic by Prof.Dr. Richard Berndt, Kiel University 50 x 49 nm300 KAu(110)

3 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Scanning Probe Microscopy 1981 Scanning Tunneling Microscope (STM) [1] 1986 Atomic Force Microscope (AFM) Other variants… graphic by Prof. Dr. Richard Berndt, Kiel University [1]G. Binnig, H. Rohrer, C. Gerber, and E. Weibel, Phys. Rev. Lett. 49, 57 (1982). 20 x 13 nm300 KSi(111)

4 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion 40s per Image 1024 x x 90 nmSi(111)

5 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion 27 IMAGES per second (64 x 64 pixels 2 ) [2]M.J. Rost, L. Crama, P. Schakel, E. van Tol et al.; Rev. Sci. Instrum. 76 (2005) Zoom Rotate Pan 27 Hz r e a l t i m e 80 Hz Au(110) HOPG

6 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Feedback Drivers Scan Generator ADCs LeidenProbeMicroscopy.com STM Head

7 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Stabilizing and Comparing Images thermal drift 50 x 49 nm300 KAu(110)

8 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion pixels AU Apply Image Stabilisation to: Stay Focused Enable Quantitative Analysis (comparing images) A tool for both Image Stabilisation and Quantitative Analysis heightline 1 st Solution: Normalized Cross-correlation (NCC)

9 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion What is Cross-correlation (CC) ? Simplifiednano wire or single-atom row x

10 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion What is Cross-correlation (CC) ? Simplified crystal surface x

11 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion where f is the image and the sum is over x under the window containing the feature t positioned at c: x = c..c+w Cross-correlation CC(c) = x f(x) t(x c) c What is Cross-correlation (CC) ? error CC depends on offset and amplitude x

12 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Mean subtracted Better Correlate Signal Form Cross-correlation CC(c) = x f(x) t(x c) where f is the image and the sum is over x under the window containing the feature t positioned at c: x = c..c+w Normalized Cross-correlation [ - 1,+1]

13 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Symmetric Computation CC(c) = N-1 x=0 f(c + x N/2) t(x) The usual notation to compute symmetrically around the column at hand Values required that are outside the signal

14 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Values required that are outside the image Boundary Conditions CC(c) = N-1 x=0 f(c + x N/2) t(x) Constant Reflect Extend Periodic

15 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion NCC Application 1: determine shift vector template dy dx image

16 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion NCC Application 2: compare images

17 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Qualitative: locate a at global peak Quantitative: a-s can be found at 1 Quantitative: o-s can be found at 0.7 NCC Application 3: locate feature template image

18 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Several Ways to Normalise Cross-correlation [3]J. Martin and J.L. Crowley. Experimental comparison of correlation techniques. In Proc. International Conf. on Intelligent Autonomous Systems, energy zero-mean image mean under template

19 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Numerator computed via FFT as a convolution with the template reversed Fast NCC Implementation [4] [4]J.P. Lewis. Fast normalized cross-correlation. In Vision Interface, pages 120–123, [5]H.Huang, D.Dabiri and M.Gharib. On errors of digital particle image velocimetry. Meas. Sci. Technol. 8 (1997) FFT requires size 2 N, pad with zeros FFT is periodic, prevent errors by padding larger area [5]

20 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Fast NCC Implementation Denominator computed from table containing the integral (running sum) of the image square over the search area. image energy under template

21 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Fast NCC Implementation: Integral Image Using the integral image representation one can compute the value of any rectangular sum in constant time. For example the integral sum inside rectangle D we can compute as: ii(4) + ii(1) ii(2) ii(3) [6]P. Viola and M. Jones. Robust real-time object detection. Second International Workshop on Statistical and Computational Theories of Vision, Def: The integral image at location (x,y), is the sum of the pixel values above and to the left of (x,y), inclusive.

22 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Results: Timing *) While Measuring, Registrate (Preliminary) Decimate image to 64 x 64 pixels 2 Apply Gaussian sub-pixel interpolation [7] Background subtraction plus fast NCC: 14 ms While Analysing, Registrate and Correlate Spatial Domain NCC: 40 minutes Fast NCC: 300 ms *) timing for images of 512 x 512 pixels 2 on a PC with an AMD Athlon at 2.8 GHz [7]J. Bolinder. On the accuracy of a digital particle image velocimetry system

23 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Results: Stabilisation Au(110) 300 K 39 x 38 nm 26 sec/frame Au(110) 300 K 52 x 55 nm 3.8 sec/frame

24 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Summary: NCC enables finding features NCC enables quantitatively comparing features & images NCC enables tracking to compensate for drift, there is room for improvement Future improvement: Lucas-Kanade [8] Spatial intensity gradient Taylor series expansion, iteration Gaussian Filter ( resolution) Pyramid of images at different resolution [8]B. Lucas and T. Kanade, An iterative image registration technique with an application to stereo vision, in Proc. Imaging Understanding Workshop, 1981, pp

25 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Recognizing Features Coalescence of Vacancy Islands on Cu(100) Paul Ruijgrok 200 x 200 nm300 KCu(100)

26 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Finding the Vacancy Islands Paul Ruijgrok

27 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Leveling the Image Accuracy: Data based number of bins Fit (part of) Gaussian curve Paul Ruijgrok

28 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Finding the Vacancy Islands: threshold Paul Ruijgrok h threshold = h 0 + sa 0, s: 0.1…0.9

29 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Detecting the Island Edges Paul Ruijgrok erosion Island AErosion E(A,N 4 )A = AE(A,N 4 )

30 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Paul Ruijgrok y i = ax i + b or b = - x i a+ y i Transform points to curves in parameter space a = 1, b = 1 y = x + 1 [9]Duda, R. O. and P. E. Hart, "Use of the Hough Transformation to Detect Lines and Curves in Pictures," Comm. ACM, Vol. 15, pp. 11–15 (January, 1972). Finding the Vacancy Lines Hough Transform

31 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Finding the Vacancy Lines Hough Transform Slope-intercept representation: unbounded parameters Want grid of limited size: ρ = x cos(θ) + y sin(θ), or ρ = C cos(θ + δ) Paul Ruijgrok

32 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Summary Paul Ruijgrok Thanks to DIPimage team, Delft University of Technology. DIPimage: a scientific image processing toolbox for MATLAB.

33 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Thanks To: Ph.D. Students drs. K. Schoots (Koen) Undergraduate Students P.V. Ruijgrok (Paul) Technicians L. Crama (Bert) E. van Tol-Homan (Els) R. Koehler (Raymond) P. Schakel (Peter) Staff prof.dr. J.W.M. Frenken (Joost) dr.ir. T.H. Oosterkamp (Tjerk) dr. M.J. Rost (Marcel)

34 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Summary template image Hough Transform

35 Introduction | Stabilizing and Comparing Images | Recognizing Features | Future ? The Future: Superresolution ? template image Hough Transform

36 Introduction | Stabilizing and Comparing Images | Recognizing Features | References [1]G. Binnig, H. Rohrer, C. Gerber, and E. Weibel, Phys. Rev. Lett. 49, 57 (1982). [2]M.J. Rost, L. Crama, P. Schakel, E. van Tol et al.; Rev. Sci. Instrum. 76 (2005) [3]J. Martin and J.L. Crowley. Experimental comparison of correlation techniques. In Proc. International Conf. on Intelligent Autonomous Systems, 1995.Experimental comparison of correlation techniques [4]J.P. Lewis. Fast normalized cross-correlation. In Vision Interface, pages 120–123, 1995.Fast normalized cross-correlation [5]H.Huang, D.Dabiri and M.Gharib. On errors of digital particle image velocimetry. Meas. Sci. Technol. 8 (1997) On errors of digital particle image velocimetry [6]P. Viola and M. Jones. Robust real-time object detection. Second International Workshop on Statistical and Computational Theories of Vision, 2001.Robust real-time object detection [7]J. Bolinder. On the accuracy of a digital particle image velocimetry system On the accuracy of a digital particle image velocimetry system [8]B. Lucas and T. Kanade, An iterative image registration technique with an application to stereo vision, in Proc. Imaging Understanding Workshop, 1981, pp An iterative image registration technique with an application to stereo vision [9]R. Duda and P. Hart. Use of the Hough transformation to detect lines and curves in pictures. Comm. ACM, Vol. 15, pp. 11–15 (January, 1972).Use of the Hough transformation to detect lines and curves in pictures. Comm References

37 Introduction | Stabilizing and Comparing Images | Recognizing Features | References Du-Ming Tsai, Chien-Ta Lin, Fast normalized cross correlation for defect detection, Pattern Recognition Letters, v.24 n.15, p , November 2003Fast normalized cross correlation for defect detection Ian T. Young, Jan. J. Gerbrands and Lucas J. van Vliet. Fundamentals of Image Processing Fundamentals of Image Processing W.H. Press, S.A. Teukolsky, W.T. Vetterling, B.P. Flannery. Numerical Recipes in C: The Art of Scientific Computing, 2nd edition. Cambridge University Press. New York, NY, USA. Numerical Recipes in C: The Art of Scientific Computing, 2nd edition Ullrich Köthe. STL-Style Generic Programming with Images. C++ Report Magazine 12(1), pp , January 2000.STL-Style Generic Programming with Images Leiden Probe Microscopy Interface Physics at Leiden University This presentation from authors web-site Other Information

38 Introduction | Stabilizing and Comparing Images | Recognizing Features | Software Stan Birchfield. Dept. of Electrical and Computer Engineering. Clemson University. KLT: An Implementation of the Kanade-Lucas-Tomasi Feature TrackerKLT: An Implementation of the Kanade-Lucas-Tomasi Feature Tracker. Quantitative Imaging Group at the Faculty of Applied Sciences, Delft University of Technology. The Delft Image Processing library The Delft Image Processing library Quantitative Imaging Group at the Faculty of Applied Sciences, Delft University of Technology. DIPimage, A Scientific Image Processing Toolbox for MATLAB DIPimage, A Scientific Image Processing Toolbox for MATLAB Insight Software Consortium. National Library of Medicine Insight Segmentation and Registration Toolkit (ITK) National Library of Medicine Insight Segmentation and Registration Toolkit (ITK) Cognitive Systems Group, University of Hamburg, Germany. The VIGRA Computer Vision Library The VIGRA Computer Vision Library Chair of Technical Computer Science, RWTH Aachen University. LTI-Lib library for image processing and computer vision LTI-Lib library for image processing and computer vision Software

39 Introduction | Stabilizing and Comparing Images | Recognizing Features | Conclusion Testbeeld


Download ppt "I A f M 2 0 0 6 Martin J. Moene E.H. van Tol-Homan P.V. Ruijgrok T.H. Oosterkamp J.W.M. Frenken M.J. Rost Kamerlingh Onnes Laboratory Image Processing."

Similar presentations


Ads by Google