Automated Registration of Synthetic Aperture Radar Imagery to LIDAR IGARSS 2011, Vancouver, Canada July 24-29, 2011 Mark Pritt, PhD Lockheed Martin Gaithersburg, Maryland mark.pritt@lmco.com Kevin LaTourette Lockheed Martin Goodyear, Arizona kevin.j.latourette@lmco.com

Problem: SAR Image Registration Registration of SAR and optical imagery is difficult. Features appear different. Different viewpoints and illumination conditions cause difficulties: SAR layover does not match optical foreshortening. Shadows do not match. Conventional techniques rely on linear features. But these features can be rare and noisy in SAR imagery. A noted hard problem in a flat area, it is extremely difficult if not impossible over mountainous or urban terrain Many conventional techniques attempt to perform a 2D-2D registration between SAR/EO images, and while this may be sufficient in flat/planar scenes, the techniques will fail in mountainous or urban terrain. Since each image is a 2D representation of a 3D scene, the perspective distortions induced by the terrain must be accounted for. SAR image MSI image

Solution Our solution is image registration to a high-resolution digital elevation model (DEM): A DEM post spacing of 1 or 2 meters yields good results. It also works with coarser post spacing. Works with terrain data derived from many sources: LIDAR: BuckEye, ALIRT, Commercial Stereo Photogrammetry: Socet Set® DSM SAR: Stereo and Interferometry USGS DEMs Solves problems ranging from Cross sensor registration, Radar/Optical/Infrared, including over rugged and urban terrain, and true orthorectification of SAR images.

Methods Create a predicted image from the DEM, illumination conditions and sensor model estimate. Register the predicted and the actual images. Refine the sensor model. Required input: (1) SAR Image to be registered, (2) Estimate of the image collection geometry and Image Formation Processing and (3) A high resolution DEM. From the SAR image metadata, we can create a mapping from the 3D world space of the DEM into the 2D pixel space of the SAR image. To simulate Radar backscatter, we use a weighted average of Lambertian and Specular shading, accounting for Layover and Shadow. Use the Sensormodel to render the SAR-shaded DEM, and register the simulated and actual images together. Since we have accounted for perspective distortions, layover, etc., there is no need for a scale/rotation invariant registration such as SIFT/SURF…instead a Normalized Cross-Correlation based method is ideally suited. The resulting registration function is then used to properly update the Sensor-camera model. Predicted SAR Image SAR Image

Methods (cont) The same approach works for SAR and optical sensors. Projection into the imaging plane is similar. Layover in SAR images is similar to occlusion in optical images. Radar shadow is similar to optical shadow. SAR Sensor Image Plane Scene Layover Shadow Scene Occlusion Optical Sensor Image Plane Shadow

Methods (cont) To register SAR and optical images, use the DEM as the “bridge”. Generate a predicted “DEM” image for each SAR and optical image. Register the predicted images to the actual images. This neatly bypasses the problem of direct SAR-optical registration. SAR Image DEM MSI Image

Example 1: SAR-LIDAR Registration COSMO-SkyMed SAR Image of Mosul, Iraq BuckEye LIDAR DEM Area: 100 km2 21,000 x 20,000 pixels Post Spacing: 1 meter Absolute Accuracy: 1.5 m (CE90)

COSMO-SkyMed SAR Image Results COSMO-SkyMed SAR Image 0.75 cm COSMO-SkyMed Spotlight-mode SAR image, processed in RMA-INCA (Range Migration Algorithm – Imaging Near Closest Approach)

Predicted SAR Image from DEM and Estimated SAR Camera Model Results (cont) Predicted SAR Image from DEM and Estimated SAR Camera Model Flicker with previous slide

Normalized Cross-Correlation Image Between Predicted and Actual Images Results (cont) Normalized Cross-Correlation Image Between Predicted and Actual Images Flicker with previous slide

COSMO-SkyMed SAR Image Results: Zoom Note the SAR layover and shadow COSMO-SkyMed SAR Image

Predicted SAR Image from DEM Zoom (cont) Note the SAR layover and shadow Predicted SAR Image from DEM Flicker with previous slide

Flicker with previous slide Zoom (cont) Cross Correlation Flicker with previous slide

Registration Accuracy NCC Registration Tie Points After least-squares fit to shift-only registration function with RANSAC outlier removal, 4572 tie points remained. Best shift: Δx = 16.76m Δy = 4.27m After fitting the tie points to a registration function, we then perform an error propagation analysis by applying the registration function to our tie points, computing the residuals and various other statistics.

Registration Accuracy (cont) Error Propagation Statistic x y Mean Residual 0 pixels Sigma Residual 0.948 pixels 0.981 pixels RMSE 1.364 pixels Circular Error Propagated to DEM 1.48 m (CE90) Propagated to Ground 2.1 m (CE90) We first note that the average residuals in x- and y- are both zero, indicating that no bias is present, similarly the fact that our standard deviations in x- and y- are so close indicates that our shift only approach was appropriate. Had an affine or higher order function been necessary, we would expect to see skewed or larger values. The geographic location of each pixel in the image is computed to within 2.1 meters with 90% confidence. This includes the geospatial errors in the DEM and the registration. CE90 = circular error 90%

Results: SAR-MSI Registration SAR Image: COSMO-SkyMed, Date: Oct 2008, GSD: 1 m

SAR-MSI Registration (cont) MSI Image: IKONOS, Date: Oct 2010, GSD: 2.2 m Flicker with previous slide

SAR-MSI Registration (cont) SAR Image: COSMO-SkyMed, Date: Oct 2008, GSD: 1 m

SAR-MSI Registration (cont) MSI Image: IKONOS, Date: Oct 2010, GSD: 2.2 m Flicker with previous slide

SAR-MSI Registration (cont) SAR Image: COSMO-SkyMed, Date: Oct 2008, GSD: 1 m

SAR-MSI Registration (cont) MSI Image: IKONOS, Date: Oct 2010, GSD: 2.2 m Flicker with previous slide

Example 2: SAR-MSI-LIDAR Fusion COSMO- SkyMed SAR Waterton, Colorado Ikonos MSI BuckEye LIDAR DEM BuckEye Lidar: March 2003 (4.1 x 5.2 km, 0.75-m post spacing) Ikonos: July 9, 2001 (1-m GSD). COSMO SkyMed SAR: Oct 31, 2008 (0.5-m GSD)

Results: EO Image Draped Over DEM Note alignment of features

Results: SAR Image Draped Over DEM Note alignment of features Flicker with previous slide

Results: MSI Image Draped Over DEM Note alignment of features Flicker with previous slide

Results: Fly-Through Click picture above to play movie

Conclusion We have introduced a new method for registering SAR images with other sensor data: LIDAR, Digital Elevation Models, Optical Images, MSI It works by image registration to a high-resolution DEM. It does this by generating a predicted image from the DEM and sensor model estimate. It then registers the predicted and actual images and refines the sensor model estimate. Accuracy: 1-2 m CE90 Our approach also extends to the case where no DEM is available: DEM can be generated from stereo EO or interferometric SAR.

Conclusion (cont.) For an extension to Video Geo-registration: Pritt, M & LaTourette, K., Stabilization and Georegistration of Aerial Video Over Mountain Terrain by Means of LIDAR. FR1.T08.4