Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 Autonomous 3D Reconstruction From Irregular LiDAR and Aerial Imagery Takis Kasparis Nicholas S. Shorter.

Similar presentations


Presentation on theme: "1 Autonomous 3D Reconstruction From Irregular LiDAR and Aerial Imagery Takis Kasparis Nicholas S. Shorter."— Presentation transcript:

1 http://www.nshorter.com 1 Autonomous 3D Reconstruction From Irregular LiDAR and Aerial Imagery Takis Kasparis kasparis@mail.ucf.edu Nicholas S. Shorter nshorter@mail.ucf.edu University of Central Florida Orlando, Florida, 32816 USA Research Website: http://www.nshorter.com

2 2 Presentation Layout 1.Overview 2.LiDAR Capture Characteristics 3.Applications 4.LiDAR Noise 5.‘Solved’ LiDAR Problems 6.Current LiDAR Challenges 7.Existing Methodology 8.Nicholas and Takis’ Research

3 http://www.nshorter.com3 LIDAR Overview Data Collection –Plane Equipped with GPS, INS & LIDAR –LIDAR – Light Detection and Ranging (active sensor) LiDAR sensor works day or night, cloud coverage or not Collection of 3D points Laser sent out from Emitter, reflects off of Terrain, Returns to Receiver [1], [2] –Receiver measures back scattered electromagnetic radiation (laser intensity) –In [1], Yu Sun et. al. propose using LS-SVM for denoising LiDAR »sufficiently explains how LiDAR works Time Difference Determines Range to Target http://www.toposys.com/

4 http://www.nshorter.com4 Sampling the following scene…

5 http://www.nshorter.com5 Raw Elevation Plot

6 http://www.nshorter.com6 Angled, Zoomed In View

7 http://www.nshorter.com7 Triangulated, Angled, Zoomed In View

8 http://www.nshorter.com8 LIDAR Captured Characteristics Range to Target (elevation from INS & LiDAR) Longitude and Latitude (GPS) First and Last Return Pulses –First – shrubbery, vegetation, power lines, birds and buildings –Last – buildings (unless vegetation is really dense, then vegetation too) Returned Laser Intensity

9 http://www.nshorter.com9 First and Last Return Pulse Difference

10 http://www.nshorter.com10 LiDAR Returned Intensity Plot

11 http://www.nshorter.com11 LiDAR Characteristics Continued Typically stored as an ASCII File The (x,y,z) coordinates are irregularly distributed – “swarm of angry bees”

12 http://www.nshorter.com12 Applications for 3D Reconstruction Military Applications –Automatic Target Recognition [3] –Reconstructed Models of Opponent Terrain (UAV) [5] Tourism/Entertainment –Virtual Walkthrough of Theme Park Commercial –Change Detection (Natural Disasters) [4] –Network Planning for Mobile Communication –Noise Nuisance (Universal Studios, 408 Expressway - walls) –Urban Planning

13 http://www.nshorter.com13 LiDAR Noise Geo-location results from LiDAR, GPS and INS sensor systems –Accuracy Limitations [1] –Offset and Drift in both GPS and INS [13] –Misalignment between INS and LiDAR [13] Atmosphere – Intensity and Path Distortion Shadowing Effect from Tall buildings Artifacts from non uniform sampling from multiple strips

14 http://www.nshorter.com14 ‘Solved’ LiDAR Challenges LiDAR Triangulation [6] –Wang et. al. in [6] benchmark different triangulation methods on LiDAR LiDAR interpolation to fixed point spacings [7] –Goncalves in [7] compares several interpolation methods and the errors they produce for LiDAR rasterizing DTM and DSM generation [25] –Commercial packages/companies provide DTM and DSM generation from raw LiDAR

15 http://www.nshorter.com15 Current LiDAR Challenges Building Detection [8], [9], [11], [17], [26] –In [8], Dr. Zhang overcomes Morphological Filtering Windowing Problem by using several window sizes (LiDAR only) –[9] unfortunately makes very limiting assumptions (single aerial image only) Parallel building sides, rectilinear buildings only, only considers aerial imagery –[17] assumes roof top colors are within red spectrum of RGB and takes 45 to 70 minutes on 6400x6400 pixel image (single aerial image only) Several buildings in Fairfield dataset have white and grey roof tops –[26] uses thresholding between DTM and DSM No ideal window size for entire data set for Morphological Filtering (as shown in [8])

16 http://www.nshorter.com16 Current LiDAR Challenges Building Reconstruction [14], [15], [16], [18] –[14] reports trouble with small houses and building roof tops not being simple Gable or Hip –[15], [16] Authors assume roofs on all buildings are flat –[18] Some stages during reconstruction process rely on human intervention

17 http://www.nshorter.com17 Current LiDAR Challenges Multiple Source Registration [10] –In [10] Zitova and Flusser refer to over 200 sources when reviewing current Image Registration methods Note [10] covers not only image, but registration between different other sources as well –Most have trouble automating the relation of control points from the source image to the reference image LiDAR Strip Registration [12] LiDAR Noise Removal [1], [2], [13] Detecting Structure under Vegetation Canopy

18 http://www.nshorter.com18 Data Sources All Sources Typically procured by Commercial Geospatial Solution Company and/or Funded Academic Researchers –More commonly researchers receive commercially donated data (instead of procuring themselves) LIDAR (both air and ground) –Overhead – readily available if not procurable –Ground – rare Aerial Imagery (single, stereo pair, video sequences) –Single Aerial Image – readily available if not procurable –Stereo Pair – most existing data sets only have single nadir image but stereo pair procurable –Video Sequence – rare but procurable GIS Ground Plans, Architectural Plans, GIS Models –All rare and very difficult to get updated Ground Truth (extremely rare)

19 http://www.nshorter.com19 LiDAR – What’s the Big Deal? Literally hundreds of commercial companies providing several Geo-spatial solutions developed for previously listed applications Lockheed Martin [22], Harris Corporation [23], and Boeing [24] have all (within the last 5 years) been involved in a LiDAR related project Literally hundreds of publications (many of which are funded), from as early as 90s, being made about LiDAR Hundreds of Geo Spatial Companies using LiDAR technology This is an actively pursued topic by both academia and industry!

20 http://www.nshorter.com20 Existing 3D Reconstruction Research Model Based Reconstruction [19], [20], [21] –Pre-defined models with parameters –Minimize error between models and data –[21] Assumes flat, gable or hip roof –[19] relies on user intervention Data Driven –Group Coplanar Pts Clustering, planar equation thresholds, etc. –Identify Break Lines planar intersections –Derive Model to Minimize Error

21 http://www.nshorter.com21 Other Data Driven Approaches In [27] Chen et. al use Patented Split Shape Merge (SMS) algorithm In [28] Overby et. al. use 3D Hough to extract planes and then use geometric constraints to refine and voting to accept/reject

22 http://www.nshorter.com22 Triangulation Based Methods Triangular Irregular Network (TIN) –Series of Non-Overlapping Triangles Modeling given Surface TIN 3D Reconstruction Methods –Clustering approach [29] Spherical Normal Vectors of Triangles –TIN region growing approach [27], [30] Merge Triangles to Same Region if Normal Vectors within Threshold

23 http://www.nshorter.com23 Nicholas and Takis’ Research Green = 85% to 100% Complete Yellow = 30% to 85% Complete Red = Less than 30% Complete

24 http://www.nshorter.com24 Our Project Publications Modified Greedy Insertion Triangulation –Shorter, N. S.; Kasparis, T. (2006). 3D Reconstruction of Irregular Spaced LIDAR, Proceedings of the 6th WSEAS International Conference on Systems Theory and Scientific Computation, Elounda, Greece, August 21-23 (pp. 19-24). LiDAR Building Detection –Shorter, N.S.; Kasparis, T. (2008). Clustering Irregular Spaced LiDAR TINs for 3D Reconstruction. The 5th International Conference On Cybernetics and Information Technologies, Systems and Applications, Orlando, Florida, June 29 - July 2 (published)

25 http://www.nshorter.com25 Our Project Publications LiDAR to Aerial Image Registration –Shorter, N.S.; Kasparis, T. (2008). Autonomous Registration of LiDAR Data to Single Aerial Image. IEEE International Geoscience & Remote Sensing Symposium, Boston, Massachusetts, July 6 - 11 (published) Building Reconstruction –Shorter, N.S.; Kasparis, T. (2008). Clustering Irregular Spaced LiDAR TINs for 3D Reconstruction. The 5th International Conference On Cybernetics and Information Technologies, Systems and Applications, Orlando, Florida, June 29 - July 2 (published) –Shorter, N. S. (2006). Heuristic 3D Reconstruction of Irregular Spaced LIDAR, Masters Thesis, University of Central Florida, Orlando Florida, 2006. –Shorter, N. S.; Kasparis, T. (2006). Fuzzy SART Clustering for 3D Reconstruction from Irregular LIDAR Data, WSEAS Transactions on Signal Processing, Vol. 2, No. 8 (pp. 1122 to 1129).

26 http://www.nshorter.com26 Triangulation Contributions Modified Greedy Insertion Triangulation –TIN Filtering - Data points within 20 centimeters of pre-existing planes are merged with those planes Implemented procedure removes elongated, distorted triangles existing at TIN edge

27 http://www.nshorter.com27 Triangulated Scene – Distorted Edge Triangles

28 http://www.nshorter.com28 Triangulated Scene

29 http://www.nshorter.com29 Building Detection Contributions Developed unsupervised building detection approach –No parameter adjustment or user intervention –No windows (or morphological filters) –No unreasonable assumptions on building structures Algorithm capable of producing generation plots (region growing age) Algorithm execution completes within seconds 90% Detection Accuracy on Commercial and Residential

30 http://www.nshorter.com30 Scene 1 - Color Aerial Image

31 http://www.nshorter.com31 Scene 1 – Unique Building Detection

32 http://www.nshorter.com32 Scene 2 – Color Aerial Image

33 http://www.nshorter.com33 Scene 2 – Binary Building Detection

34 http://www.nshorter.com34 Aerial Image Contributions Image Segmentation –Use of Fuzzy ART for Color Quantization Used as Preprocessing step for JSEG image Segmentation Line Detection –Implemented line segment detection approach based on radon transform

35 http://www.nshorter.com35 LiDAR to Aerial Image Registration Developed Interpolation Approach –Ties Range Image to Irregular LiDAR and TIN Proposed use Binary Building Images for preprocessing Proposed use of Phase Only Matched Filtering (POMF) for Unsupervised Registration approach

36 http://www.nshorter.com36 Interpolated Range Image

37 http://www.nshorter.com37 Binary Mask and Registration

38 http://www.nshorter.com38 Reconstruction Contributions Fuzzy SART Planar Segment Detection –Proposed Translating Input to Origin –Proposed Scaling each Input Dimension –Transformed TIN Normal Vectors to Polar Coordinates –Proposed use of Fuzzy SART for detecting planar segments within LiDAR TIN Used Planar Regression Procedure to form LMS Best Fit Plane after Fuzzy SART Clustering

39 http://www.nshorter.com39 Aerial Images for Buildings

40 http://www.nshorter.com40 Reconstructed Buildings

41 http://www.nshorter.com41 Acknowledgements Data Contributors –Dr. Simone Clode, Dr. Franz Rottensteiner, AAMHatch –Mr. John Ellis, AeroMap –Mr. Steffen Firchau, TopoSys –Mr. Paul Mrstik, Terra Point Funding Contributors –Harris Corporation Advisor (Committee Chair) –Dr. Takis Kasparis Committee Members –Dr. Michael Georgiopoulos, Dr. Georgios Anagnostopoulos, Dr. Andy Lee, Dr. Abhijit Mahalanobis, and Dr. Wasfy Mikhael

42 http://www.nshorter.com42 References [1] Sun, Bing Yu; Huang, De-Shang; Fang, Hai-Tao; “LiDAR Signal Denoising Using Least Square Support Vector Machine.” IEEE Signal Processing Letters, Vol. 12, No. 2, February 2005 [2] Yu, Shirong; Wang, Weiran; “LiDAR Signal Denoising Based on Wavelet Domain Spatial Filtering.” International Conference on Radar, October 2006 [3] Mahalanobis, Abhijit; “Multidimensional Algorithms for Target Detection in LiDAR Imagery.” University of Central Florida, Electrical and Computer Engineering Seminar Series. Orlando. 28 March 2007 [4] Vu, Tuong Thuy; Matsoka, Matashi; Yamazaki, Fumio; “LiDAR based Change Detection of Buildings in Dense Urban Areas.” Geoscience and Remote Sensing Symposium, 2004. IGARSS '04. Proceedings. 2004 IEEE International [5] Lammons, George; “After The Storm.” Military Geospatial Technology, Volume 4, Issue 1, March 2006. [6] Wang, Kai; Lo, Chor Pang; Brook, George; Arabnia, Hamid; “Comparison of existing triangulation methods for regularly and irregularly spaced height fields.” INT. J. Geographical Information Science, vol. 15, no. 8, pp 743-762, 2001 [7] Goncalves, Gil; "Analysis of Interpolation Errors in Urban Digital Surface Models Created from LiDAR Data." 7th International Symposium on Spatial Accuracy Assessment in Natural Resources and Environmental Sciences, 2006, pp 160-168 [8] Zhang, Keqi; Chen, Shu-Ching; Whitman, Dean; Shyu, Mei-Ling; Yan, Jianhua, Zhang, Chengcui; "A Progressive Morphological Filter for Removing Nonground Measurements from Airborne LIDAR Data." IEEE Transactions on Geoscience and Remote Sensing, Vol. 41, No. 4, April 2003. [9] Zitova, Barbara; Flusser, Jan; “Image Registration Methods: A Survey”; Image and Vision Computing, Vol 21, 2003, pp 977-1000 [10] Katartzis, Antonis; and Sahli, Hichem; “A Stochastic Framework for the Identification of Building Rooftops Using a Single Remote Sensing Image.” IEEE Transactions on Geoscience and Remote Sensing, vol. 46, no. 1, January 2008 [11] Rottensteiner, Franz; Trinder, John; Clode, Simon; Kubik, Kurt; “Building Detection Using LIDAR Data and Multispectral Images.” 2003 [12] Vosselman, G.; Maas, H.-G.; "Adjustment and Filtering of Raw Laser Altimetry Data." Proceedings OEEPE Workshop on Airborne Laserscanning and Interferometric SAR for Detailed Digital Elevation Models, OEEPE Publication No. 40, on CD-ROM, 11 pages.

43 http://www.nshorter.com43 References [13] Filin, S. Elimination of systematic errors from airborne laser scanning data. Geoscience and Remote Sensing Symposium, 2005. IGARSS apos;05. Proceedings. 2005 IEEE International. Volume 1, Issue, 25-29 July 2005 Page(s): 4 pp. [14] Forlani, G.; Nardinocchi, C.; Zingaretti, P.; Scaioni, M. ; "Complete classification of raw LIDAR data and 3D reconstruction of buildings." Pattern Analysis and Applications vol.8, no.4 (2006),p.357-74 [15] Xie, Minghong; Fu, Kun; Wu, Yirong; "Building Recognition and Reconstruction from Aerial Imagery and LIDAR Data." Radar, 2006. CIE '06. International Conference on, Oct. 2006, pp. 1-4. [16] Chen, Liang-Chien; Teo, Tee-Ann; Shao, Yi-Chen; Lai, Yen-Chung; and Rau, Jiann-Yeou; “Fusion of LIDAR Data and Optical Imagery for Building Modeling.” ISPRS XXth Congress - Comission 4, 2004, p732 [17] Muller, Sonke; Zaum, Daniel; "Robust Building Detection in Aerial Images." IAPRS, Vol. XXXVI, Part3/W24, Vienna, Austria, August 29-30, 2005 [18] Vosselman, G.; Gorte, B.G.H.; Sithole, G.; Rabbani, T.; "Recognising structure in laser scanner point clouds." International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. 46, part 8/W2, Freiburg, Germany, October 4-6, (2004) pp. 33-38. [19] Hu, Jinhui; You, Suya; Neumann, Ulrich; Park, Kyung Kook; "Building Modeling From LiDAR and Aerial Imagery." Proceedings of ASPRS, May 2004. [20] Mass, H; and Vosselman, G. “Two Algorithms for Extracting Building Models from Raw Lser Altimetry Data.” ISPRS Journal of Photogrammetry & Remote Sensing, vol. 54, pp. 153-155, 1999 [21] Suveg, Ildiko; and Vosselman, George. “Automatic 3D Building Reconstruction.” Proceedings of SPIE, vol. 4661, pp. 59 – 69, 2002 [22] http://www.laserfocusworld.com/display_article/309074/12/none/none/INDUS/Lockheed-Martin-to-combine-electro-optics,-LIDAR- for-urban-environment-surveillanchttp://www.laserfocusworld.com/display_article/309074/12/none/none/INDUS/Lockheed-Martin-to-combine-electro-optics,-LIDAR- for-urban-environment-surveillanc [23] http://www.harris.com/view_pressrelease.asp?act=lookup&pr_id=2001http://www.harris.com/view_pressrelease.asp?act=lookup&pr_id=2001 [24] http://www.boeing.com/news/releases/2004/q4/nr_041203m.htmlhttp://www.boeing.com/news/releases/2004/q4/nr_041203m.html [13] Filin, S. Elimination of systematic errors from airborne laser scanning data. Geoscience and Remote Sensing Symposium, 2005. IGARSS apos;05. Proceedings. 2005 IEEE International. Volume 1, Issue, 25-29 July 2005 Page(s): 4 pp. [14] Forlani, G.; Nardinocchi, C.; Zingaretti, P.; Scaioni, M. ; "Complete classification of raw LIDAR data and 3D reconstruction of buildings." Pattern Analysis and Applications vol.8, no.4 (2006),p.357-74 [15] Xie, Minghong; Fu, Kun; Wu, Yirong; "Building Recognition and Reconstruction from Aerial Imagery and LIDAR Data." Radar, 2006. CIE '06. International Conference on, Oct. 2006, pp. 1-4. [16] Chen, Liang-Chien; Teo, Tee-Ann; Shao, Yi-Chen; Lai, Yen-Chung; and Rau, Jiann-Yeou; “Fusion of LIDAR Data and Optical Imagery for Building Modeling.” ISPRS XXth Congress - Comission 4, 2004, p732 [17] Muller, Sonke; Zaum, Daniel; "Robust Building Detection in Aerial Images." IAPRS, Vol. XXXVI, Part3/W24, Vienna, Austria, August 29-30, 2005 [18] Vosselman, G.; Gorte, B.G.H.; Sithole, G.; Rabbani, T.; "Recognising structure in laser scanner point clouds." International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. 46, part 8/W2, Freiburg, Germany, October 4-6, (2004) pp. 33-38. [19] Hu, Jinhui; You, Suya; Neumann, Ulrich; Park, Kyung Kook; "Building Modeling From LiDAR and Aerial Imagery." Proceedings of ASPRS, May 2004. [20] Mass, H; and Vosselman, G. “Two Algorithms for Extracting Building Models from Raw Lser Altimetry Data.” ISPRS Journal of Photogrammetry & Remote Sensing, vol. 54, pp. 153-155, 1999 [21] Suveg, Ildiko; and Vosselman, George. “Automatic 3D Building Reconstruction.” Proceedings of SPIE, vol. 4661, pp. 59 – 69, 2002 [22] http://www.laserfocusworld.com/display_article/309074/12/none/none/INDUS/Lockheed-Martin-to-combine-electro-optics,-LIDAR- for-urban-environment-surveillanchttp://www.laserfocusworld.com/display_article/309074/12/none/none/INDUS/Lockheed-Martin-to-combine-electro-optics,-LIDAR- for-urban-environment-surveillanc [23] http://www.harris.com/view_pressrelease.asp?act=lookup&pr_id=2001http://www.harris.com/view_pressrelease.asp?act=lookup&pr_id=2001 [24] http://www.boeing.com/news/releases/2004/q4/nr_041203m.htmlhttp://www.boeing.com/news/releases/2004/q4/nr_041203m.html [25] Arefi, H.; Hahn, M.; “A Morphological Reconstruction Algorithm For Separating Off-Terrain Points from Terrain Points in Laser Scanning Data.” ISPRS Workshop Laser Scanning, 2005

44 http://www.nshorter.com44 References [26] Rottensteiner, F.; and Briese, Ch.; “A New Method for Building Extraction in Urban Areas from High-Resolution LIDAR Data.” IAPRSIS, vol. 34/3A, pp. 295-301, 2002 [27] Chen, Liang Chien; Teo, Tee-Ann; Shao, Yi-Chen; Lai, Yen-Chung; Rau, Jiann-Yeo; “Fusion of LIDAR Data and Optical Imagery for Building Modeling.” International Archives of Photogrammetry and Remote Sensing, vol. 35, no. B4, pp. 732-737, 2004 [28] Overby, Jens; Bodum, Lars; Kjems, Erik; and Ilsoe, Pee M; “Automatic 3D Building Reconstruction from Airborne Laser Scanning and Cadastral Data Using Hough Transform.” Geo-Imagery Bridging Continents 20th ISPRS Congress, 2004 [29] Hofmann, A.D.; “Analysis of TIN-Structure Parameter Spaces In Airborne Laser Scanning Data for 3-D Building Model Generation.” Geo-Imagery Bridging Continents XXth ISPRS Congress, 2004 [30] Morgan, Michel; Habib, Ayman; “Interpolation of LIDAR Data and Automatic Building Extraction.” ACSM-ASPRS2002 Annual Conference Proceedings, 2002


Download ppt "1 Autonomous 3D Reconstruction From Irregular LiDAR and Aerial Imagery Takis Kasparis Nicholas S. Shorter."

Similar presentations


Ads by Google