Presentation on theme: "Stabilization and Georegistration of Aerial Video Over Mountain Terrain by Means of LIDAR Mark Pritt, PhD Lockheed Martin Gaithersburg, Maryland"— Presentation transcript:
Stabilization and Georegistration of Aerial Video Over Mountain Terrain by Means of LIDAR Mark Pritt, PhD Lockheed Martin Gaithersburg, Maryland Kevin LaTourette Lockheed Martin Goodyear, Arizona IGARSS 2011, Vancouver, Canada July 24-29, 2011
2 Problem: Georegistration Georegistration is the assignment of 3-D geographic coordinates to the pixels of an image. It is required for many geospatial applications: Fusion of imagery with other sensor data Alignment of imagery with GIS and map graphics Accurate 3-D geolocation Inaccurate georegistration can be a major problem: Misaligned GIS Correctly aligned
3 Solution Our solution is image registration to a high-resolution digital elevation model (DEM): A DEM post spacing of 1 or 2 meters yields good results. It also works with 10-meter post spacing. Works with terrain data derived from many sources: LIDAR: BuckEye, ALIRT, Commercial Stereo Photogrammetry: Socet Set® DSM SAR: Stereo and Interferometry USGS DEMs
4 Create predicted images from the DEM, illumination conditions, sensor model estimates and actual images. Register the images while refining the sensor model. Iterate. Methods
5 Methods (cont) Predicted Image from DEM Predicted Image from Aerial Image Registration Tie Point Detections The algorithm identifies tie points between the predicted and the actual images by means of NCC (normalized cross correlation) with RANSAC outlier removal.
6 The algorithm uses the refined sensor model as the initial guess for the next video frame: The refined sensor model enables georegistration. Exterior orientation: Platform position and rotation angles Interior orientation: Focal length, pixel aspect ratio, principal point and radial distortion Methods (cont) Initial Camera Estimate camera model Use camera focal length & platform GPS if avail. Estimate camera model Use camera focal length & platform GPS if avail. Register Predict images from DEM and camera Register images with NCC Predict images from DEM and camera Register images with NCC Refine Compose registration fcn & camera LS fit for better cam estimate Iterate Compose registration fcn & camera LS fit for better cam estimate Iterate Next Frame Register to previous frame Compose with cam of prev. frame for init. cam estimate Register to previous frame Compose with cam of prev. frame for init. cam estimate Iterate Iterate for each video frame Finish Trajectory Propagate geo data from DEM Resample images for orthomosaic Trajectory Propagate geo data from DEM Resample images for orthomosaic
7 Example 1: Aerial Motion Imagery Inputs: Aerial Motion Imagery over Arizona, U.S. 16 Mpix, 3.3 fps, panchromatic
8 Example 1 (cont) Problem: Too shaky to find moving objects Zoomed to full resolution (1 m)
9 Outputs: Sensor camera models Images georegistered to DEM Platform trajectory Example 1: Results
10 Example 1 Results (cont) ATV Vehicle HumanHuman Pickup Truck Video is now stabilized, and as a result, moving objects are easily detected.
11 Example 2: Oblique Motion Imagery Inputs: Oblique Motion Imagery Over Arizona, U.S. 16 Mpix, 3.4 fps, pan LIDAR DEM Area: 24 km 2 Post Spacing: 1 m
12 Example 2: Results Map coordinates Stabilized Video Inset Orthorectified Video Background LIDAR DEM Aligned Map Graphics Target Tracking Aligned Map Graphics
13 Example 2 Results (cont) IMAGE 1Camera Iteration 123 Num tie points: RMSE: Mean Δx: Mean Δy: Sigma Δx: Sigma Δy: IMAGE 591Camera Iteration 123 Num tie points RMSE Mean Δx100 Mean Δy0.900 Sigma Δx Sigma Δy How fast does the algorithm converge? The initial error is high, but it decreases after only several iterations. Subsequent frames have better initial sensor model estimates and require only 2 iterations.
14 Example 3: Aerial Video Inputs: Aerial Video Over Arizona, U.S. 720 x 480 Color 30 fps LIDAR DEM Area: 24 km 2 Post Spacing: 1 m
15 Example 3: Results Map coordinates Orthorectified Video Background Image Draped Over DEM Aligned Map Graphics
16 Example 3 Results (cont) Map Graphics Stay Aligned with Features in Video
17 Example 4: Thermal Infrared Video Inputs: MWIR Video Over White Tank Mountains in Arizona 1 Mpix, 3.3 fps Commercial LIDAR DEM Post Spacing: 2 m
18 Example 4: Results Video Mosaic Georegistered and Draped Over Mountains in Google Earth
19 Demo Click picture to play video
20 Conclusion We have introduced a new method for aerial video georegistration and stabilization. It registers images to high-resolution DEMs by: Generating predicted images from the DEM and sensor model; Registering these predicted images to the actual images; Correcting the sensor model estimates with the registration results. Processing speed is 1 sec per 16-Mpix image on a PC. Absolute geospatial accuracy is about 1-2 meters. We are developing a rigorous error propagation model to quantify the accuracy. Applications: Video stabilization and mosacs Cross-sensor registration Alignment with GIS map graphics