Download presentation
Presentation is loading. Please wait.
Published byJocelin Harrison Modified over 9 years ago
1
http://www.nshorter.com 1 Autonomous 3D Reconstruction From Irregular LiDAR and Aerial Imagery Takis Kasparis kasparis@mail.ucf.edu Nicholas S. Shorter nshorter@mail.ucf.edu University of Central Florida Orlando, Florida, 32816 USA Research Website: http://www.nshorter.com
2
2 Presentation Layout 1.Overview 2.LiDAR Capture Characteristics 3.Applications 4.LiDAR Noise 5.‘Solved’ LiDAR Problems 6.Current LiDAR Challenges 7.Existing Methodology 8.Nicholas and Takis’ Research
3
http://www.nshorter.com3 LIDAR Overview Data Collection –Plane Equipped with GPS, INS & LIDAR –LIDAR – Light Detection and Ranging (active sensor) LiDAR sensor works day or night, cloud coverage or not Collection of 3D points Laser sent out from Emitter, reflects off of Terrain, Returns to Receiver [1], [2] –Receiver measures back scattered electromagnetic radiation (laser intensity) –In [1], Yu Sun et. al. propose using LS-SVM for denoising LiDAR »sufficiently explains how LiDAR works Time Difference Determines Range to Target http://www.toposys.com/
4
http://www.nshorter.com4 Sampling the following scene…
5
http://www.nshorter.com5 Raw Elevation Plot
6
http://www.nshorter.com6 Angled, Zoomed In View
7
http://www.nshorter.com7 Triangulated, Angled, Zoomed In View
8
http://www.nshorter.com8 LIDAR Captured Characteristics Range to Target (elevation from INS & LiDAR) Longitude and Latitude (GPS) First and Last Return Pulses –First – shrubbery, vegetation, power lines, birds and buildings –Last – buildings (unless vegetation is really dense, then vegetation too) Returned Laser Intensity
9
http://www.nshorter.com9 First and Last Return Pulse Difference
10
http://www.nshorter.com10 LiDAR Returned Intensity Plot
11
http://www.nshorter.com11 LiDAR Characteristics Continued Typically stored as an ASCII File The (x,y,z) coordinates are irregularly distributed – “swarm of angry bees”
12
http://www.nshorter.com12 Applications for 3D Reconstruction Military Applications –Automatic Target Recognition [3] –Reconstructed Models of Opponent Terrain (UAV) [5] Tourism/Entertainment –Virtual Walkthrough of Theme Park Commercial –Change Detection (Natural Disasters) [4] –Network Planning for Mobile Communication –Noise Nuisance (Universal Studios, 408 Expressway - walls) –Urban Planning
13
http://www.nshorter.com13 LiDAR Noise Geo-location results from LiDAR, GPS and INS sensor systems –Accuracy Limitations [1] –Offset and Drift in both GPS and INS [13] –Misalignment between INS and LiDAR [13] Atmosphere – Intensity and Path Distortion Shadowing Effect from Tall buildings Artifacts from non uniform sampling from multiple strips
14
http://www.nshorter.com14 ‘Solved’ LiDAR Challenges LiDAR Triangulation [6] –Wang et. al. in [6] benchmark different triangulation methods on LiDAR LiDAR interpolation to fixed point spacings [7] –Goncalves in [7] compares several interpolation methods and the errors they produce for LiDAR rasterizing DTM and DSM generation [25] –Commercial packages/companies provide DTM and DSM generation from raw LiDAR
15
http://www.nshorter.com15 Current LiDAR Challenges Building Detection [8], [9], [11], [17], [26] –In [8], Dr. Zhang overcomes Morphological Filtering Windowing Problem by using several window sizes (LiDAR only) –[9] unfortunately makes very limiting assumptions (single aerial image only) Parallel building sides, rectilinear buildings only, only considers aerial imagery –[17] assumes roof top colors are within red spectrum of RGB and takes 45 to 70 minutes on 6400x6400 pixel image (single aerial image only) Several buildings in Fairfield dataset have white and grey roof tops –[26] uses thresholding between DTM and DSM No ideal window size for entire data set for Morphological Filtering (as shown in [8])
16
http://www.nshorter.com16 Current LiDAR Challenges Building Reconstruction [14], [15], [16], [18] –[14] reports trouble with small houses and building roof tops not being simple Gable or Hip –[15], [16] Authors assume roofs on all buildings are flat –[18] Some stages during reconstruction process rely on human intervention
17
http://www.nshorter.com17 Current LiDAR Challenges Multiple Source Registration [10] –In [10] Zitova and Flusser refer to over 200 sources when reviewing current Image Registration methods Note [10] covers not only image, but registration between different other sources as well –Most have trouble automating the relation of control points from the source image to the reference image LiDAR Strip Registration [12] LiDAR Noise Removal [1], [2], [13] Detecting Structure under Vegetation Canopy
18
http://www.nshorter.com18 Data Sources All Sources Typically procured by Commercial Geospatial Solution Company and/or Funded Academic Researchers –More commonly researchers receive commercially donated data (instead of procuring themselves) LIDAR (both air and ground) –Overhead – readily available if not procurable –Ground – rare Aerial Imagery (single, stereo pair, video sequences) –Single Aerial Image – readily available if not procurable –Stereo Pair – most existing data sets only have single nadir image but stereo pair procurable –Video Sequence – rare but procurable GIS Ground Plans, Architectural Plans, GIS Models –All rare and very difficult to get updated Ground Truth (extremely rare)
19
http://www.nshorter.com19 LiDAR – What’s the Big Deal? Literally hundreds of commercial companies providing several Geo-spatial solutions developed for previously listed applications Lockheed Martin [22], Harris Corporation [23], and Boeing [24] have all (within the last 5 years) been involved in a LiDAR related project Literally hundreds of publications (many of which are funded), from as early as 90s, being made about LiDAR Hundreds of Geo Spatial Companies using LiDAR technology This is an actively pursued topic by both academia and industry!
20
http://www.nshorter.com20 Existing 3D Reconstruction Research Model Based Reconstruction [19], [20], [21] –Pre-defined models with parameters –Minimize error between models and data –[21] Assumes flat, gable or hip roof –[19] relies on user intervention Data Driven –Group Coplanar Pts Clustering, planar equation thresholds, etc. –Identify Break Lines planar intersections –Derive Model to Minimize Error
21
http://www.nshorter.com21 Other Data Driven Approaches In [27] Chen et. al use Patented Split Shape Merge (SMS) algorithm In [28] Overby et. al. use 3D Hough to extract planes and then use geometric constraints to refine and voting to accept/reject
22
http://www.nshorter.com22 Triangulation Based Methods Triangular Irregular Network (TIN) –Series of Non-Overlapping Triangles Modeling given Surface TIN 3D Reconstruction Methods –Clustering approach [29] Spherical Normal Vectors of Triangles –TIN region growing approach [27], [30] Merge Triangles to Same Region if Normal Vectors within Threshold
23
http://www.nshorter.com23 Nicholas and Takis’ Research Green = 85% to 100% Complete Yellow = 30% to 85% Complete Red = Less than 30% Complete
24
http://www.nshorter.com24 Our Project Publications Modified Greedy Insertion Triangulation –Shorter, N. S.; Kasparis, T. (2006). 3D Reconstruction of Irregular Spaced LIDAR, Proceedings of the 6th WSEAS International Conference on Systems Theory and Scientific Computation, Elounda, Greece, August 21-23 (pp. 19-24). LiDAR Building Detection –Shorter, N.S.; Kasparis, T. (2008). Clustering Irregular Spaced LiDAR TINs for 3D Reconstruction. The 5th International Conference On Cybernetics and Information Technologies, Systems and Applications, Orlando, Florida, June 29 - July 2 (published)
25
http://www.nshorter.com25 Our Project Publications LiDAR to Aerial Image Registration –Shorter, N.S.; Kasparis, T. (2008). Autonomous Registration of LiDAR Data to Single Aerial Image. IEEE International Geoscience & Remote Sensing Symposium, Boston, Massachusetts, July 6 - 11 (published) Building Reconstruction –Shorter, N.S.; Kasparis, T. (2008). Clustering Irregular Spaced LiDAR TINs for 3D Reconstruction. The 5th International Conference On Cybernetics and Information Technologies, Systems and Applications, Orlando, Florida, June 29 - July 2 (published) –Shorter, N. S. (2006). Heuristic 3D Reconstruction of Irregular Spaced LIDAR, Masters Thesis, University of Central Florida, Orlando Florida, 2006. –Shorter, N. S.; Kasparis, T. (2006). Fuzzy SART Clustering for 3D Reconstruction from Irregular LIDAR Data, WSEAS Transactions on Signal Processing, Vol. 2, No. 8 (pp. 1122 to 1129).
26
http://www.nshorter.com26 Triangulation Contributions Modified Greedy Insertion Triangulation –TIN Filtering - Data points within 20 centimeters of pre-existing planes are merged with those planes Implemented procedure removes elongated, distorted triangles existing at TIN edge
27
http://www.nshorter.com27 Triangulated Scene – Distorted Edge Triangles
28
http://www.nshorter.com28 Triangulated Scene
29
http://www.nshorter.com29 Building Detection Contributions Developed unsupervised building detection approach –No parameter adjustment or user intervention –No windows (or morphological filters) –No unreasonable assumptions on building structures Algorithm capable of producing generation plots (region growing age) Algorithm execution completes within seconds 90% Detection Accuracy on Commercial and Residential
30
http://www.nshorter.com30 Scene 1 - Color Aerial Image
31
http://www.nshorter.com31 Scene 1 – Unique Building Detection
32
http://www.nshorter.com32 Scene 2 – Color Aerial Image
33
http://www.nshorter.com33 Scene 2 – Binary Building Detection
34
http://www.nshorter.com34 Aerial Image Contributions Image Segmentation –Use of Fuzzy ART for Color Quantization Used as Preprocessing step for JSEG image Segmentation Line Detection –Implemented line segment detection approach based on radon transform
35
http://www.nshorter.com35 LiDAR to Aerial Image Registration Developed Interpolation Approach –Ties Range Image to Irregular LiDAR and TIN Proposed use Binary Building Images for preprocessing Proposed use of Phase Only Matched Filtering (POMF) for Unsupervised Registration approach
36
http://www.nshorter.com36 Interpolated Range Image
37
http://www.nshorter.com37 Binary Mask and Registration
38
http://www.nshorter.com38 Reconstruction Contributions Fuzzy SART Planar Segment Detection –Proposed Translating Input to Origin –Proposed Scaling each Input Dimension –Transformed TIN Normal Vectors to Polar Coordinates –Proposed use of Fuzzy SART for detecting planar segments within LiDAR TIN Used Planar Regression Procedure to form LMS Best Fit Plane after Fuzzy SART Clustering
39
http://www.nshorter.com39 Aerial Images for Buildings
40
http://www.nshorter.com40 Reconstructed Buildings
41
http://www.nshorter.com41 Acknowledgements Data Contributors –Dr. Simone Clode, Dr. Franz Rottensteiner, AAMHatch –Mr. John Ellis, AeroMap –Mr. Steffen Firchau, TopoSys –Mr. Paul Mrstik, Terra Point Funding Contributors –Harris Corporation Advisor (Committee Chair) –Dr. Takis Kasparis Committee Members –Dr. Michael Georgiopoulos, Dr. Georgios Anagnostopoulos, Dr. Andy Lee, Dr. Abhijit Mahalanobis, and Dr. Wasfy Mikhael
42
http://www.nshorter.com42 References [1] Sun, Bing Yu; Huang, De-Shang; Fang, Hai-Tao; “LiDAR Signal Denoising Using Least Square Support Vector Machine.” IEEE Signal Processing Letters, Vol. 12, No. 2, February 2005 [2] Yu, Shirong; Wang, Weiran; “LiDAR Signal Denoising Based on Wavelet Domain Spatial Filtering.” International Conference on Radar, October 2006 [3] Mahalanobis, Abhijit; “Multidimensional Algorithms for Target Detection in LiDAR Imagery.” University of Central Florida, Electrical and Computer Engineering Seminar Series. Orlando. 28 March 2007 [4] Vu, Tuong Thuy; Matsoka, Matashi; Yamazaki, Fumio; “LiDAR based Change Detection of Buildings in Dense Urban Areas.” Geoscience and Remote Sensing Symposium, 2004. IGARSS '04. Proceedings. 2004 IEEE International [5] Lammons, George; “After The Storm.” Military Geospatial Technology, Volume 4, Issue 1, March 2006. [6] Wang, Kai; Lo, Chor Pang; Brook, George; Arabnia, Hamid; “Comparison of existing triangulation methods for regularly and irregularly spaced height fields.” INT. J. Geographical Information Science, vol. 15, no. 8, pp 743-762, 2001 [7] Goncalves, Gil; "Analysis of Interpolation Errors in Urban Digital Surface Models Created from LiDAR Data." 7th International Symposium on Spatial Accuracy Assessment in Natural Resources and Environmental Sciences, 2006, pp 160-168 [8] Zhang, Keqi; Chen, Shu-Ching; Whitman, Dean; Shyu, Mei-Ling; Yan, Jianhua, Zhang, Chengcui; "A Progressive Morphological Filter for Removing Nonground Measurements from Airborne LIDAR Data." IEEE Transactions on Geoscience and Remote Sensing, Vol. 41, No. 4, April 2003. [9] Zitova, Barbara; Flusser, Jan; “Image Registration Methods: A Survey”; Image and Vision Computing, Vol 21, 2003, pp 977-1000 [10] Katartzis, Antonis; and Sahli, Hichem; “A Stochastic Framework for the Identification of Building Rooftops Using a Single Remote Sensing Image.” IEEE Transactions on Geoscience and Remote Sensing, vol. 46, no. 1, January 2008 [11] Rottensteiner, Franz; Trinder, John; Clode, Simon; Kubik, Kurt; “Building Detection Using LIDAR Data and Multispectral Images.” 2003 [12] Vosselman, G.; Maas, H.-G.; "Adjustment and Filtering of Raw Laser Altimetry Data." Proceedings OEEPE Workshop on Airborne Laserscanning and Interferometric SAR for Detailed Digital Elevation Models, OEEPE Publication No. 40, on CD-ROM, 11 pages.
43
http://www.nshorter.com43 References [13] Filin, S. Elimination of systematic errors from airborne laser scanning data. Geoscience and Remote Sensing Symposium, 2005. IGARSS apos;05. Proceedings. 2005 IEEE International. Volume 1, Issue, 25-29 July 2005 Page(s): 4 pp. [14] Forlani, G.; Nardinocchi, C.; Zingaretti, P.; Scaioni, M. ; "Complete classification of raw LIDAR data and 3D reconstruction of buildings." Pattern Analysis and Applications vol.8, no.4 (2006),p.357-74 [15] Xie, Minghong; Fu, Kun; Wu, Yirong; "Building Recognition and Reconstruction from Aerial Imagery and LIDAR Data." Radar, 2006. CIE '06. International Conference on, Oct. 2006, pp. 1-4. [16] Chen, Liang-Chien; Teo, Tee-Ann; Shao, Yi-Chen; Lai, Yen-Chung; and Rau, Jiann-Yeou; “Fusion of LIDAR Data and Optical Imagery for Building Modeling.” ISPRS XXth Congress - Comission 4, 2004, p732 [17] Muller, Sonke; Zaum, Daniel; "Robust Building Detection in Aerial Images." IAPRS, Vol. XXXVI, Part3/W24, Vienna, Austria, August 29-30, 2005 [18] Vosselman, G.; Gorte, B.G.H.; Sithole, G.; Rabbani, T.; "Recognising structure in laser scanner point clouds." International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. 46, part 8/W2, Freiburg, Germany, October 4-6, (2004) pp. 33-38. [19] Hu, Jinhui; You, Suya; Neumann, Ulrich; Park, Kyung Kook; "Building Modeling From LiDAR and Aerial Imagery." Proceedings of ASPRS, May 2004. [20] Mass, H; and Vosselman, G. “Two Algorithms for Extracting Building Models from Raw Lser Altimetry Data.” ISPRS Journal of Photogrammetry & Remote Sensing, vol. 54, pp. 153-155, 1999 [21] Suveg, Ildiko; and Vosselman, George. “Automatic 3D Building Reconstruction.” Proceedings of SPIE, vol. 4661, pp. 59 – 69, 2002 [22] http://www.laserfocusworld.com/display_article/309074/12/none/none/INDUS/Lockheed-Martin-to-combine-electro-optics,-LIDAR- for-urban-environment-surveillanchttp://www.laserfocusworld.com/display_article/309074/12/none/none/INDUS/Lockheed-Martin-to-combine-electro-optics,-LIDAR- for-urban-environment-surveillanc [23] http://www.harris.com/view_pressrelease.asp?act=lookup&pr_id=2001http://www.harris.com/view_pressrelease.asp?act=lookup&pr_id=2001 [24] http://www.boeing.com/news/releases/2004/q4/nr_041203m.htmlhttp://www.boeing.com/news/releases/2004/q4/nr_041203m.html [13] Filin, S. Elimination of systematic errors from airborne laser scanning data. Geoscience and Remote Sensing Symposium, 2005. IGARSS apos;05. Proceedings. 2005 IEEE International. Volume 1, Issue, 25-29 July 2005 Page(s): 4 pp. [14] Forlani, G.; Nardinocchi, C.; Zingaretti, P.; Scaioni, M. ; "Complete classification of raw LIDAR data and 3D reconstruction of buildings." Pattern Analysis and Applications vol.8, no.4 (2006),p.357-74 [15] Xie, Minghong; Fu, Kun; Wu, Yirong; "Building Recognition and Reconstruction from Aerial Imagery and LIDAR Data." Radar, 2006. CIE '06. International Conference on, Oct. 2006, pp. 1-4. [16] Chen, Liang-Chien; Teo, Tee-Ann; Shao, Yi-Chen; Lai, Yen-Chung; and Rau, Jiann-Yeou; “Fusion of LIDAR Data and Optical Imagery for Building Modeling.” ISPRS XXth Congress - Comission 4, 2004, p732 [17] Muller, Sonke; Zaum, Daniel; "Robust Building Detection in Aerial Images." IAPRS, Vol. XXXVI, Part3/W24, Vienna, Austria, August 29-30, 2005 [18] Vosselman, G.; Gorte, B.G.H.; Sithole, G.; Rabbani, T.; "Recognising structure in laser scanner point clouds." International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. 46, part 8/W2, Freiburg, Germany, October 4-6, (2004) pp. 33-38. [19] Hu, Jinhui; You, Suya; Neumann, Ulrich; Park, Kyung Kook; "Building Modeling From LiDAR and Aerial Imagery." Proceedings of ASPRS, May 2004. [20] Mass, H; and Vosselman, G. “Two Algorithms for Extracting Building Models from Raw Lser Altimetry Data.” ISPRS Journal of Photogrammetry & Remote Sensing, vol. 54, pp. 153-155, 1999 [21] Suveg, Ildiko; and Vosselman, George. “Automatic 3D Building Reconstruction.” Proceedings of SPIE, vol. 4661, pp. 59 – 69, 2002 [22] http://www.laserfocusworld.com/display_article/309074/12/none/none/INDUS/Lockheed-Martin-to-combine-electro-optics,-LIDAR- for-urban-environment-surveillanchttp://www.laserfocusworld.com/display_article/309074/12/none/none/INDUS/Lockheed-Martin-to-combine-electro-optics,-LIDAR- for-urban-environment-surveillanc [23] http://www.harris.com/view_pressrelease.asp?act=lookup&pr_id=2001http://www.harris.com/view_pressrelease.asp?act=lookup&pr_id=2001 [24] http://www.boeing.com/news/releases/2004/q4/nr_041203m.htmlhttp://www.boeing.com/news/releases/2004/q4/nr_041203m.html [25] Arefi, H.; Hahn, M.; “A Morphological Reconstruction Algorithm For Separating Off-Terrain Points from Terrain Points in Laser Scanning Data.” ISPRS Workshop Laser Scanning, 2005
44
http://www.nshorter.com44 References [26] Rottensteiner, F.; and Briese, Ch.; “A New Method for Building Extraction in Urban Areas from High-Resolution LIDAR Data.” IAPRSIS, vol. 34/3A, pp. 295-301, 2002 [27] Chen, Liang Chien; Teo, Tee-Ann; Shao, Yi-Chen; Lai, Yen-Chung; Rau, Jiann-Yeo; “Fusion of LIDAR Data and Optical Imagery for Building Modeling.” International Archives of Photogrammetry and Remote Sensing, vol. 35, no. B4, pp. 732-737, 2004 [28] Overby, Jens; Bodum, Lars; Kjems, Erik; and Ilsoe, Pee M; “Automatic 3D Building Reconstruction from Airborne Laser Scanning and Cadastral Data Using Hough Transform.” Geo-Imagery Bridging Continents 20th ISPRS Congress, 2004 [29] Hofmann, A.D.; “Analysis of TIN-Structure Parameter Spaces In Airborne Laser Scanning Data for 3-D Building Model Generation.” Geo-Imagery Bridging Continents XXth ISPRS Congress, 2004 [30] Morgan, Michel; Habib, Ayman; “Interpolation of LIDAR Data and Automatic Building Extraction.” ACSM-ASPRS2002 Annual Conference Proceedings, 2002
Similar presentations
© 2025 SlidePlayer.com Inc.
All rights reserved.