1. http://www.nshorter.com 1 Autonomous 3D Reconstruction From Irregular LiDAR and Aerial Imagery Nicholas Shorter BSEE, May 2005; MSEE, Aug. 2006 PhD Committee: Dr. Takis Kasparis (Chair) Dr. Georgios Anagnostopoulos, Dr. Michael Georgiopoulos, Dr. Andy Lee, Dr. Wasfy Mikhael Email: nshorter@cfl.rr.comnshorter@cfl.rr.com Website: http://www.nshorter.comhttp://www.nshorter.com

2. 2 Presentation Layout 1.LIDAR Overview 2.Problem Statement 3.Present Art 4.Proposed Solution 5.Completed Work 6.Future Work 7.Potential Implementation Obstacles 8.Milestones

3. http://www.nshorter.com3 LIDAR Overview Data Collection –Plane Equipped with GPS, INS & LIDAR –LIDAR – Light Detection and Ranging (active sensor) Collection of 3D points Laser sent out from Emitter, reflects off of Terrain, Returns to Receiver –Receiver measures back scattered electromagnetic radiation (laser intensity) Time Difference Determines Range to Target http://www.toposys.com/

4. http://www.nshorter.com4 LIDAR Captured Characteristics Range to Target (elevation from INS & LiDAR) Longitude and Latitude (GPS) First and Last Return Pulses –First – shrubbery, vegetation, power lines, birds and buildings –Last – buildings (unless vegetation is really dense, then vegetation too) Returned Laser Intensity

5. http://www.nshorter.com5 LiDAR Noise Geolocation results from LiDAR, GPS and INS sensor systems –Accuracy Limitations –Offset and Drift in both GPS and INS –Misalignment between INS and LiDAR Atmosphere – Intensity and Path Distortion Shadowing Effect from Tall buildings Artifacts from non uniform sampling from multiple strips

6. http://www.nshorter.com6 Problem Statement Input –LIDAR (Light Detection and Ranging) Data Collection of Irregularly Distributed 3-Dimensional Points –Aerial Photograph Output –Semi to complete Automatic (minimal user intervention) development of 3-Dimensional Virtual Model

7. http://www.nshorter.com7 Applications for 3D Reconstruction Military Applications –Automatic Target Recognition –Reconstructed Models of Opponent Terrain (UAV?) Tourism/Entertainment –Virtual Walkthrough of Theme Park Commercial –Change Detection (Natural Disasters) –Network Planning for Mobile Communication –Noise Nuisance (Universal Studios, 408 Expressway - walls) –Urban Planning

8. http://www.nshorter.com8 Ideal Algorithm Features Ideal Attributes –Handle Complex (non planar) buildings –Irregular Points –Multiple Source as Available –Reconstruction exists as membered simple entities –Building Isolation –Complete Automation –Aerial Image Projection Dissertation Implementation –Yes –Only LiDAR and Aerial Imagery –Yes

9. http://www.nshorter.com9 Existing 3D Reconstruction Research Data Sources –LIDAR –Aerial Imagery –GIS Ground Plans Model Based Reconstruction –Pre-defined models with parameters –Minimize error between models and data Data Driven –Group Coplanar Pts –Identify Break Lines –Derive Model to Minimize Error

10. http://www.nshorter.com10 Triangulation Based Methods Triangular Irregular Network (TIN) –Series of Non-Overlapping Triangles Modeling given Surface TIN 3D Reconstruction Methods –Clustering approach Spherical Normal Vectors of Triangles –TIN region growing approach Merge Triangles to Same Region if Normal Vectors within Threshold

11. http://www.nshorter.com11 Existing 3D Reconstruction Methods Most still under development Most Methods Use ‘Grided’ (Interpolated) LIDAR Data –Advantages Less Computationally Complex DTM & DSM Thresholding to distinguish Building from Non Building Use of additional conventional methods –Disadvantages Decrease in Accuracy Uncertainty from Building and Ground Interpolation

12. http://www.nshorter.com12 Masters Research Greedy Insertion Triangulation –Implemented Noise Filtering Technique Proposed FSART normal vector clustering Proposed Planar Regression to combat Category Proliferation Realized simple planar reconstruction algorithm (MSEE Thesis)

13. http://www.nshorter.com13 Post Masters, Pre-Candidacy Research Implemented FA, GA, and FSART Clustering for reconstruction performance comparison Developed Category Proliferation and Clustering Performance measures Ran comparison on 4 buildings

14. http://www.nshorter.com14 Proposed Algorithm – System Level Anticipated Challenges: –Automatic Image and LiDAR Registration –Matlab Rendering Reconstructed City Blocks –Matlab mapping Images to Models

15. http://www.nshorter.com15 Building Detection Anticipated Challenges: –Morphological Filtering needs a priori window size

16. http://www.nshorter.com16 Wall Tri. And Gnd Pt. Identification Wall Triangle = (Max diff. in elev. > 2m) & (pitch > 60 degrees) Gnd Pt. = Wall Tri. Lowest elevation pt

17. http://www.nshorter.com17 Building Extraction Clustering will automatically identify and separate individual buildings

18. http://www.nshorter.com18 Building Features Vegetation Points –Significant first and last return elevation diff. –Corresponding green aerial image color –Nearest points have diff. elev. or adjacent triangles have significantly diff. norm vectors Common Building Points –Spatially close in terms of long. and lat. –Bounded by aerial img. edges and exterior wall tri. –Bounded building does not contain terrain pts – or - –Triangulation of all points, building points connected

19. http://www.nshorter.com19 Building Reconstruction

20. http://www.nshorter.com20 Building Reconstruct Cluster Features Surface change - normal vector orientation difference between adjacent triangles (X,Y,Z) - Longitude, Latitude, Elevation Edge - Aerial imagery edge detection Color - Aerial imagery corresponding color (building surface differs from clutter) Triangle planar coef, pt. height diff., same normal vector, or planar equation Feedback – difference between reconstruction and raw LiDAR and Aerial Image Edges

21. http://www.nshorter.com21 Algorithm Novelty Novel Building Detection Method– Irregular LiDAR –Uses raw pts and triangulation Novel Automated Building Extraction –Clustering for automated extraction Novel Clustering for Reconstruction –Capable of handling complex building structure –Proposing Clustering instead of Existing (ART, K- Means, etc.) Automated LiDAR and Image Registration

22. http://www.nshorter.com22 Implementation Obstacles Matlab Obstacles –Matlab Rendering of Large City Sets –Matlab displaying Large City Models with Mapped Images Will have to program Open GL or other rendering solution (learning curve) Additional reader development for different data sources (Fairfield mostly simple buildings)

23. http://www.nshorter.com23 Future Tasks and Milestones – Su 07 Investigate systematic noise removal Continue Debugging and Testing of GIT for 150k+ pts –Currently works for 10k pts in > 10 seconds Development of Building Detection –Investigate no a-priori max build size for morphological filtering

24. http://www.nshorter.com24 Future Tasks and Milestones – Fa 07 Develop Automatic LiDAR & Image Registration Development of Building Extraction –Investigate which clustering approach Publish Detection and Extraction Procedure

25. http://www.nshorter.com25 Future Tasks and Milestones – Spr 08 Develop Reconstruction Algorithm –Propose Clustering Method Automatically map images to constructed models Develop OpenGL to render multiple building subsets (neighborhood block sizes)

26. http://www.nshorter.com26 Future Tasks and Milestones – Su 08 Publish Reconstruction Algorithm Write Dissertation (~July 21 deadline) Defend Dissertation (~July 10 deadline)

27. http://www.nshorter.com27 Acknowledgements Data Contributors –Dr. Simone Clode, Dr. Franz Rottensteiner, AAMHatch –Mr. John Ellis, AeroMap –Mr. Steffen Firchau, TopoSys –Mr. Paul Mrstik, Terra Point Advisor (Committee Chair) –Dr. Takis Kasparis Committee Members –Dr. Michael Georgiopoulos, Dr. Georgios Anagnostopoulos, Dr. Andy Lee, and Dr. Wasfy Mikhael

28. http://www.nshorter.com28 Accomplishments WSEAS International Conference on Systems Theory and Computation – August 2006 WSEAS Transactions on Signal Processing – August 2006 MSEE – August 2006 Invited Harris Talk UCF Graduate Research Forum Poster Board Presentation (Spring 2006) UCF Graduate Research Forum Oral Presentation (Spring 2007)

29. http://www.nshorter.com29 Thank You for your Attendance Additional Questions?