1. Automated Registration of Synthetic Aperture Radar Imagery to LIDAR
IGARSS 2011, Vancouver, Canada
July 24-29, 2011
Mark Pritt, PhD
Lockheed Martin
Gaithersburg, Maryland
Kevin LaTourette
Lockheed Martin
Goodyear, Arizona

2. Problem: SAR Image Registration
Registration of SAR and optical imagery is difficult.
Features appear different.
Different viewpoints and illumination conditions cause difficulties:
SAR layover does not match optical foreshortening.
Shadows do not match.
Conventional techniques rely on linear features.
But these features can be rare and noisy in SAR imagery.
A noted hard problem in a flat area, it is extremely difficult if not impossible over mountainous or urban terrain
Many conventional techniques attempt to perform a 2D-2D registration between SAR/EO images, and while this may be sufficient in flat/planar scenes, the techniques will fail in mountainous or urban terrain. Since each image is a 2D representation of a 3D scene, the perspective distortions induced by the terrain must be accounted for.
SAR image
MSI image

3. Solution
Our solution is image registration to a high-resolution digital elevation model (DEM):
A DEM post spacing of 1 or 2 meters yields good results.
It also works with coarser post spacing.
Works with terrain data derived from many sources:
LIDAR: BuckEye, ALIRT, Commercial
Stereo Photogrammetry: Socet Set® DSM
SAR: Stereo and Interferometry
USGS DEMs
Solves problems ranging from Cross sensor registration, Radar/Optical/Infrared, including over rugged and urban terrain, and true orthorectification of SAR images.

4. Methods
Create a predicted image from the DEM, illumination conditions and sensor model estimate.
Register the predicted and the actual images.
Refine the sensor model.
Required input: (1) SAR Image to be registered, (2) Estimate of the image collection geometry and Image Formation Processing and (3) A high resolution DEM.
From the SAR image metadata, we can create a mapping from the 3D world space of the DEM into the 2D pixel space of the SAR image. To simulate Radar backscatter, we use a weighted average of Lambertian and Specular shading, accounting for Layover and Shadow.
Use the Sensormodel to render the SAR-shaded DEM, and register the simulated and actual images together. Since we have accounted for perspective distortions, layover, etc., there is no need for a scale/rotation invariant registration such as SIFT/SURF…instead a Normalized Cross-Correlation based method is ideally suited.
The resulting registration function is then used to properly update the Sensor-camera model.
Predicted SAR Image
SAR Image

5. Methods (cont) The same approach works for SAR and optical sensors.
Projection into the imaging plane is similar.
Layover in SAR images is similar to occlusion in optical images.
Radar shadow is similar to optical shadow.
SAR Sensor
Image Plane
Scene
Layover
Shadow
Scene
Occlusion
Optical Sensor
Image Plane
Shadow

6. Methods (cont)
To register SAR and optical images, use the DEM as the “bridge”.
Generate a predicted “DEM” image for each SAR and optical image.
Register the predicted images to the actual images.
This neatly bypasses the problem of direct SAR-optical registration.
SAR Image
DEM
MSI Image

7. Example 1: SAR-LIDAR Registration
COSMO-SkyMed SAR Image of Mosul, Iraq
BuckEye LIDAR DEM
Area: 100 km2
21,000 x 20,000 pixels
Post Spacing: 1 meter
Absolute Accuracy: 1.5 m (CE90)

8. COSMO-SkyMed SAR Image
Results
COSMO-SkyMed SAR Image
0.75 cm COSMO-SkyMed Spotlight-mode SAR image, processed in RMA-INCA (Range Migration Algorithm – Imaging Near Closest Approach)

9. Predicted SAR Image from DEM and Estimated SAR Camera Model
Results (cont)
Predicted SAR Image from DEM and Estimated SAR Camera Model
Flicker with previous slide

10. Normalized Cross-Correlation Image Between Predicted and Actual Images
Results (cont)
Normalized Cross-Correlation Image Between Predicted and Actual Images
Flicker with previous slide

11. COSMO-SkyMed SAR Image
Results: Zoom
Note the SAR layover and shadow
COSMO-SkyMed SAR Image

12. Predicted SAR Image from DEM
Zoom (cont)
Note the SAR layover and shadow
Predicted SAR Image from DEM
Flicker with previous slide

13. Flicker with previous slide
Zoom (cont)
Cross Correlation
Flicker with previous slide

14. Registration Accuracy
NCC Registration Tie Points
After least-squares fit to shift-only registration function with RANSAC outlier removal, 4572 tie points remained.
Best shift:
Δx = 16.76m
Δy = 4.27m
After fitting the tie points to a registration function, we then perform an error propagation analysis by applying the registration function to our tie points, computing the residuals and various other statistics.

15. Registration Accuracy (cont)
Error Propagation
Statistic x y
Mean Residual
0 pixels
Sigma Residual
0.948 pixels
0.981 pixels
RMSE
1.364 pixels
Circular Error
Propagated to DEM
1.48 m (CE90)
Propagated to Ground
2.1 m (CE90)
We first note that the average residuals in x- and y- are both zero, indicating that no bias is present, similarly the fact that our standard deviations in x- and y- are so close indicates that our shift only approach was appropriate. Had an affine or higher order function been necessary, we would expect to see skewed or larger values.
The geographic location of each pixel in the image is computed to within 2.1 meters with 90% confidence.
This includes the geospatial errors in the DEM and the registration.
CE90 = circular error 90%

16. Results: SAR-MSI Registration
SAR Image: COSMO-SkyMed, Date: Oct 2008, GSD: 1 m

17. SAR-MSI Registration (cont)
MSI Image: IKONOS, Date: Oct 2010, GSD: 2.2 m
Flicker with previous slide

18. SAR-MSI Registration (cont)
SAR Image: COSMO-SkyMed, Date: Oct 2008, GSD: 1 m

19. SAR-MSI Registration (cont)
MSI Image: IKONOS, Date: Oct 2010, GSD: 2.2 m
Flicker with previous slide

20. SAR-MSI Registration (cont)
SAR Image: COSMO-SkyMed, Date: Oct 2008, GSD: 1 m

21. SAR-MSI Registration (cont)
MSI Image: IKONOS, Date: Oct 2010, GSD: 2.2 m
Flicker with previous slide

22. Example 2: SAR-MSI-LIDAR Fusion
COSMO- SkyMed
SAR
Waterton, Colorado
Ikonos
MSI
BuckEye
LIDAR DEM
BuckEye Lidar: March 2003 (4.1 x 5.2 km, 0.75-m post spacing)
Ikonos: July 9, 2001 (1-m GSD) COSMO SkyMed SAR: Oct 31, 2008 (0.5-m GSD)

23. Results: EO Image Draped Over DEM
Note alignment
of features

24. Results: SAR Image Draped Over DEM
Note alignment
of features
Flicker with previous slide

25. Results: MSI Image Draped Over DEM
Note alignment
of features
Flicker with previous slide

26. Results: Fly-Through
Click picture above to play movie

27. Conclusion
We have introduced a new method for registering SAR images with other sensor data:
LIDAR, Digital Elevation Models, Optical Images, MSI
It works by image registration to a high-resolution DEM.
It does this by generating a predicted image from the DEM and sensor model estimate.
It then registers the predicted and actual images and refines the sensor model estimate.
Accuracy: 1-2 m CE90
Our approach also extends to the case where no DEM is available:
DEM can be generated from stereo EO or interferometric SAR.

28. Conclusion (cont.) For an extension to Video Geo-registration:
Pritt, M & LaTourette, K., Stabilization and Georegistration of Aerial Video Over Mountain Terrain by Means of LIDAR.
FR1.T08.4