Automatically Collect Ground Control Points from Online Aerial Maps The 36th Asian Conference on Remote Sensing Quezon City, Metro Manila Philippines Automatically Collect Ground Control Points from Online Aerial Maps Tengfei Long longtf@radi.ac.cn Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences October 22, 2015

Outline Background Technical frame Online matching Results Conclusions

Background Ground Control Points (GCPs) for remotely sensed (RS) images Necessity Satellite 90% CE (meters) GeoEye-1 2.5 WorldView1 7.6 WorldView2 12.2 Quickbird 23 Variable errors in exterior-orientation parameters, and most of remotely sensed images are not ready to use Geometric bias (Ozcanli et al. 2014) Difficulties Scene source Size (Pixel×Pixel) GF-1 WFV 12000 ×13400 GF-1 PAN 18192 ×18000 GF-2 PAN 29200 ×27619 Large data volume Multi-source: different time, different sensor, different spectral Automatic approach High resolution Large data volume Expensive Reference images

Better than 5 meters in many study areas. Background Reference images  Online resources Google satellite images (Google Earth) Bing aerial images MapQuest satellite maps Mapbox satellite images … Better than 5 meters in many study areas. Free or cheap, and geometric accuracy is improving But… No automatic solution

Area-based matching (ABM) Feature-based matching (FBM) Background Area-based matching (ABM) Feature-based matching (FBM) Automatic registration Normalized cross-correlation Least squares matching Phase correlation Mutual Information SIFT, SURF, FAST, … Iterative Closest Point Soft assignment Probabilistic methods SIFT Most robust and widely used Computational expensive Sensitive to noise, illumination and geometric distortion Accurate Efficient Robust to geometric distortion, illumination and occlusion Less accurate

Problems Challenges when SIFT directly used in RS images： Large image size: time consuming Large scene: similar features, less distinct Accuracy: extracted independently Distribution: not well distributed Outliers: complicate distortion

Outline Background Technical frame Online matching Results Conclusions

Analysis Considering the specific task of geometric rectification of remotely sensed image Initial imaging models are available RPCs, rigorous sensor model, affine transformation in Geo-referenced images, etc. Only tens of GCPs are required Too many GCPs do not necessarily benefit accuracy, and tens of GCPs are enough. Distribution of GCPs is important

Improvement strategies Image tiling Small image tile (256 × 256) and simple distortion Small scene Well distributed Using prior geometric information Predict master image tile Using attributes of SIFT feature Scale, contrast, dominant orientation Refining the results of SIFT Least squares matching, sub-pixel accuracy

Least squares matching Technical frame Technical frame based on image tiling and improved SIFT Initial imaging model Slave image Tile Master image Tile Master image(s) Cross matching Distance ratio SIFT matching Scale constraints Angle constraints Similarity transformation Affine transformation Outlier removing Least squares matching Slave image tiling Largest contrast

Outline Background Technical frame Online matching Results Conclusions

Online Aerial Maps Many Providers Unified Projection Multiple Scales Google, Bing, MapBox, MapQuest, ESRI, Yandex… Unified Projection Web Mercator projection Multiple Scales Zoom 1~Zoom 23 Global reference 250m ~ 0.3m

Static Maps API Service Download online reference image tile via Static Maps API Service

Online image tile parameters Calculate zoom level from ground sample distance (GSD) i.e. spatial resolution Calculate online image coordinates from longitude (λ) and latitude (φ) width and height Calculate longitude (λ) and latitude (φ) from online image coordinates

Online Matching A slave image tile matched with different online aerial tiles

Outline Background Technical frame Online matching Results Conclusions

Robustness Compare with ordinary SIFT (Lowe 2004) and SR-SIFT (Yi et al. 2008) 12 test tiles from 6 image pairs including Landsat-5 vs Landsat-5 , HJ-1 vs Landsat-8, GF-1 vs Bing aerial map, ZY-3 vs RapidEye, GF-1 MSS vs GF-1 PAN, Kompsat-2 vs Worldview-1 Tiles Proposed method Ordinary SIFT SR-SIFT 1 17 14 (11) 2 5 6 (4) 7 (6) 3 9 8 (6) 4 11 13 (9) 8 10 (5) 6 28 49 (26) 50 (25) 7 16 (5) 17 (6) 34 (3) 105 104 (102) 103 (101) 10 101 102 (98) 0 (0) 12 4 (2) 5 (3)

Least Squares Matching Accuracy Refining position using Least Squares Matching (LSM) 42 scenes of GF-1 MSS images, and 25 check points for each scene Check the RMSEs of matched points before and after Least Squares Matching (LSM). Slave tile Master tile Least Squares Matching

Online Matching Test data (reference: Bing aerial maps) No. Data source Band Image size GSD (m) Elevation Initial Model 1 Landsat-5 Band 4 6850×5733 30 566-4733 Rigorous 2 Cbers-4 Band 3 6000×6000 20 1010-2442 3 GF-1 4548×4595 8 41-825 RPC 4 ZY-3 Band 1 8819×9279 5.8 769-2549 5 Theos Pan 14083×14115 1285-1736 Affine 6 Band3 6000×88154 657-1634 Test result (20 check points) Test No. Required GCPs Correct Matches Run-time (s) Rectification Model RMSE (pixel) 1 30 32 7.11 Rigorous 1.24 2 100 97 36.83 TD-RPC 1.63 3 36 9.67 RPC-Refine 1.38 4 35 23.48 1.82 5 77 67.41 1.46 6 36.93 2.01 Test 4: Before After Test 5: Before After

Outline Background Technical frame Online matching Results Conclusions

Conclusions Geometric rectification for RS images Efficient, robust, accurate and well distributed Automatically collect from online aerial maps Free or cheap Not suitable for dense matching ---- time consuming Accuracy of online aerial maps is limited, but it is improving

Questions & Suggestions?