1. The Ohio State University Center for Mapping Charles K. Toth 1 & Dorota A. Grejner-Brzezinska 2 Center for Mapping 1 and Department of Civil and Environmental Engineering and Geodetic Science 2 The Ohio State University e-mail: toth@cfm.ohio-state.edu http://www.cfm.ohio-state.edu/ CLEM 2001 Specialist Meeting on Centerline Extraction and Maintenance August 6-7, 2001 Santa Barbara OSU Integrated GPS/INS/CCD System for High-Precision Centerline Extraction

2. The Ohio State University Center for Mapping National Consortium for Remote Sensing in Transportation - Flows Develop, demonstrate, and disseminate cost-effective remote sensing techniques for application to transportation flows NCRST - F

3. The Ohio State University Center for Mapping Traffic Monitoring Traffic Management Freight and Intermodal Analysis From Satellite or Airborne Platforms NCRST - F Application Areas

4. The Ohio State University Center for Mapping

5. Multi-sensor and multi-platform fusion for flow acquisition and interpretation Findings to date –Positioning quality at flight altitude: 2-5cm per coordinate –Positioning quality on the ground: 20-30cm from 300m altitude –Strong dependence on system calibration Progress to Date

6. The Ohio State University Center for Mapping Tower Images

7. The Ohio State University Center for Mapping

8. Presentation Outline  Problem description – background  Solution/System design  Hardware configuration  Prototype imaging software suite  Performance experiences  Imaging component  (GPS/IMU positioning system)  Summary

9. The Ohio State University Center for Mapping cable location Precision Centerline Mapping

10. The Ohio State University Center for Mapping  Mapping road lane markers at decimeter-level accuracy while maintaining traffic flow and ensuring safety of survey personnel  Finding the connection between the high- accuracy vehicle navigation and the road surface  Mechanical solutions (early system concept)  Accident hazard  Strong dependence on driver  Non-contact measurements Problem Description

11. The Ohio State University Center for Mapping  Multisensor data fusion  High speed image acquisition  2D/3D image processing (photogrammetry)  Computer vision  Real-time processing  Tightly-coupled GPS/INS system  Post-processing  Real-time relative motion support for the real-time image processing system Proposed Solution

12. The Ohio State University Center for Mapping Digital Camera Y H  Z INS -Y INS GPS Antenna INS Mapping Concept

13. The Ohio State University Center for Mapping System Design Concept Image Acquisition Control and Storage Image Acquisition Control and Storage Strapdown Navigation Solution Tightly Coupled GPS/INS Kalman Filter Strapdown Navigation Solution Tightly Coupled GPS/INS Kalman Filter User Interface, Control & Display Unit User Interface, Control & Display Unit Pulnix TMC 6700 Digital Color Camera Pulnix TMC 6700 Digital Color Camera Rover GPS Station Rover GPS Station LN-100 Base GPS Station Base GPS Station L1 and L2 phase observable Delta V Delta  L1 and L2 Image Data Exposure Control Time Tag/Sync EO Data Host/Slave Communication Control Signal Optimal Position, Velocity, Attitude Estimates Real-time Image Processing Block Real-time Image Processing Block Image Post- processing Block Image Post- processing Block

14. The Ohio State University Center for Mapping  Digital frame camera used in tests presented here  based on Lockheed Martin Failrchild 4K  4K CCD sensor  60mm by 60mm imaging area (15-micron pixel size)  Hasselblad 553 ELX camera body with 50 mm Zeiss lens  6 second image acquisition rate (0.16 Hz)  Target sensor  Pulnix color digital camera (TMC 6700)  9 mm by 6 mm imaging area  30 images per second max acquisition rate (30 Hz) Imaging Component

15. The Ohio State University Center for Mapping RGB to S Transformation Median Filter Binary Conversion Boundary Points Centerline Extraction RGB to S Transformation Median Filter Binary Conversion Boundary Points Centerline Extraction Feature Point Extraction Feature Point Matching Affine Model Formation Centerline Strip Formation Image n Centerline Position Refinement Export to GIS/CAD Post-processing Stereo Image Processing Data Acquisition Image Preprocessing Feature Point Extraction Image n-1 Final Navigation Data Relative Motion Estimates Image Processing Concept

16. The Ohio State University Center for Mapping Typical Road/Line Patterns

17. The Ohio State University Center for Mapping B/W Image of Low-quality Road Surface

18. The Ohio State University Center for Mapping Sobel Edge Operator

19. The Ohio State University Center for Mapping Color Separation RGB to IHS transformation

20. The Ohio State University Center for Mapping Color Transformation

21. The Ohio State University Center for Mapping Scanline Centerline Intersection noise Centerline Extraction (1) SL RL CL P

22. The Ohio State University Center for Mapping Centerline Extraction (2) Monoscopic solution moderate accuracy High accuracy solution 3D processing

23. The Ohio State University Center for Mapping Feature Point Extraction R(x, y) = det[C] – k trace 2 [C] Corner detector:

24. The Ohio State University Center for Mapping Feature Point Matching s1 s2 s3 m2m2 m11m11 m21m21 m13m13 m23m23

25. The Ohio State University Center for Mapping Strip Formed Pairwise affine transformation (6-parameter model)

26. The Ohio State University Center for Mapping StatusStatus  Prototype system implemented in C++  Experiences  Two sets of images (~15-20) with various road/centerline conditions  Computation-intensive algorithms with convincing performance (post-processing)  Remaining tasks  Real-time data acquisition (camera interface and navigation data from GPS/IMU system)  Software reengineering (multithreading)  Performance tuning (dual P-IV @ 1.7 GHz)

27. The Ohio State University Center for Mapping System Calibration Components GPS Camera Calibration Lever Arm Calibration INS OTF Calibration GPS Base INS Sensor Mount CCD Boresight Misalignment

28. The Ohio State University Center for Mapping Digital camera GPS antenna LN 100 Test Hardware Configuration

29. The Ohio State University Center for Mapping Performance Analysis Tests  Single side-looking camera, tilted downwards by 5°, 4K by 4K B/W imagery  50-mm focal length  Imagery collected along the surveyed road (edge/center line location)  Stereo-pairs formed by subsequent images  7-8 m object distance for boresighting  8-20 m object distance for ground control points (check points)

30. The Ohio State University Center for Mapping Vehicle Trajectory Start -83.052-83.05-83.048-83.046-83.044-83.042-83.04 39.997 39.998 39.999 40 40.001 40.002 40.003 longitude [deg] latitude [deg] Start/end Calibration range Centerline test range

31. The Ohio State University Center for Mapping Positioning Accuracy 4.1654.174.1754.184.1854.194.1954.24.205 x 10 5 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 GPS time [sec] RMS [m]

32. The Ohio State University Center for Mapping Attitude Accuracy 4.1654.174.1754.184.1854.194.1954.24.205 x 10 5 0 5 10 15 20 25 30 35 GPS time [sec] RMS [arcsec]

33. The Ohio State University Center for Mapping Boresight Calibration Test Range

34. The Ohio State University Center for Mapping Boresight Misalignment Performance Boresight performed by comparison of GPS/INS results with AT solution  Accuracy of the boresight components  1-2 cm for offsets  17-22 arcsec for attitude

35. The Ohio State University Center for Mapping Centerline Test Range

36. The Ohio State University Center for Mapping Ground coordinate difference for check points measured from different stereo-pairs Ground coordinate difference for check points measured on stereo-pairs from different passes Testing Positioning Performance

37. The Ohio State University Center for Mapping GPS Losses of Lock

38. The Ohio State University Center for Mapping Positional Error Growth during GPS Outage Positional Error Growth during GPS Outage

39. The Ohio State University Center for Mapping SummarySummary  Multisensor all-digital mapping system  Real-time image processing needed  Automatic extraction of centerlines is a reality  Real-time implementation is feasible  Direct orientation can be achieved with high accuracy  Strong GPS-dependency in urban areas  INS support needed (attitude less critical)  Proper system calibration is crucial

40. The Ohio State University Center for Mapping LIDAR/Digital Image Data Courtesy of The EarthData Group Baltimore, MD

41. The Ohio State University Center for Mapping Panchromatic Image and LIDAR Elevations Plotted as Intensity

42. The Ohio State University Center for Mapping ++++++ ++ ++++ +++ ++++++++ ++++++++

44. Image/LIDAR Sampling Pattern 0.4-0.8  1.064 

45. The Ohio State University Center for Mapping LIDAR Spots Overlaid in the Nadir Area

46. The Ohio State University Center for Mapping Misalignment of LIDAR Spots and Imagery

47. The Ohio State University Center for Mapping What Could Be the Problem?  Sensor modeling of the camera (interior orientation)  LIDAR sensor modeling (scan angle error)  Digital camera boresight misalignment  LIDAR boresight misalignment  Varying navigation performance  Mechanical flex of the sensor mount  Etc.