1. Robust Feature Extraction for 3D Point Clouds from Vehicle Based Laser Scanner Abdul Nurunnabi, David Belton, Geoff West Department of Spatial Sciences Curtin University, Perth, Australia CRC for Spatial Information (CRCSI)

2. Objectives  Investigating problems of outlier in 3D point cloud data.  Introduction to diagnostics and robust statistics.  Robust plane fitting.  Outlier detection in point cloud data and robust saliency features estimation.  Point cloud (planar and complex objects) segmentation and merging.  Ground surface extraction.

3. Outliers in point cloud: the physical limitations of sensors, boundaries between 3D features, occlusions, multiple reflectance and noise can produce off-surface points that appear to be outliers or gross errors (Sotoodeh, 2007). An outlier is an observation that deviates so much from the bulk (centre) of the observations and/or pattern of the majority observations and/or violates the usual assumptions as it seems that it was generated by a different mechanism or from a different population (Hawkins, 1980; Hubert et al., 2008; Rousseeuw and Leroy, 2003). Problems: the presence of outliers in a dataset may cause the parameter estimation to be erroneous, misclassifying the outcomes and consequently creating problems when making inferences with the wrong model. 3 Outlier/Gross error/Noise (?)

4. Robust Statistics Robust statistics is for robust fit which is similar to the fit without outliers. The basic philosophy of robust statistics is to produce statistical procedures which are stable with respect to small changes in the data or model and even large changes should not cause a complete breakdown of the procedures (Davies and Gather, 2004). Diagnostic Statistics Find all the outliers that matter. Rather than modifying the fitting method, diagnostics condition on the fit using standard methods to attempt to diagnose incorrect assumptions, allowing the analyst to modify them and refit under the new set of assumptions (Stahel and Weisberg, 1991). 4

5. Covariance Statistics and PCA The covariance matrix for a point p i = (x i, y i, z i ) (in a 3D point cloud P ) with k neighbourhood N pi is defined as: SVD is a way to solve the eigenvalue equation. Eigenvalues is the i th diagonal element of and ; Eigenvectors (PCs), approximates the normal, is a measure of the variation along the normal, and curvature is: where PCA is very sensitive to outliers. Sample mean and covariance matrix used here have an unbounded influence function and zero breakdown point (Hampel et al. 1986). Local neighbourhood Variations in different directions 5

6. Outlier Effects on Plane and Normal Point cloud with noise PCA normals without noise PCA normals with noise 6

7. Robust PCA (RPCA) Each i th direction is scored by its corresponding value of outlyingness: (Stahel and Donoho, 1981; 1982): RPCA (Hubert and Rousseeuw, 2005) 1. Use PP to make the data m < n, 2. Calculate fast-MCD based univariate centre and scale, and use them to find outlying cases, 3. Use h – set of inliers for the covariance matrix to get robust PCs. 7 Diagnostic plot Fitted plane, green points are distant in terms of score and red points are orthogonal outliers

8. 8 PCA robust method (a) Road scene data; planar surface (b) front view (c) side view. Robust Plane Extraction Extracted plane Fitted plane

9. Outlier Detection Mobile mapping laser scanner points Front view Back view RANSAC Robust method 9

10. Outlier Detection and Robust Saliency Features Estimation  Finding outliers locally for every point in a cloud.  Outliers are determined using robust statistical methods: robust z-score and robust Mahalanobis distance.  Combination of the ideas of point to plane orthogonal distance and surface points variation along the normal.  Probabilistic iterative process is used to get outlier free subset.  Estimation of local saliency features based on most consistent subset in a local neighbourhood.

11. n=100, Op=80 Outlier Effects on Different Methods n=20, Op=20 n=100, Op=70 11 Results from 1000 samples of 100 points Results from 1000 samples of 100 points with 70% outliers Box plots of θ Average θ versus outlier percentage

12. Methods Sample size 2550100100010000 PCA0.001270.001250.001150.001560.00540 RD0.865240.863310.883451.064031.23031 RANSAC0.215010.215150.229190.445402.36701 MSAC0.221480.225070.242750.479922.51789 MCMD_Z0.00566 (38)0.005750.005690.007320.02551 (93) MCMD_MD0.00497 (43)0.004750.004720.006400.02124 (111) Different sample sizes (Op=20%) Methods Outlier percentage 5102050 PCA0.001240.001250.001150.00112 RD0.860820.861190.883450.87385 RANSAC0.219150.220130.229190.60328 MSAC0.226560.228300.242750.70448 MCMD_Z0.00339 (65)0.003780.005690.02122 (28) MCMD_MD0.00257 (85)0.003300.004720.02032 (30) Processing Time (in sec.) Different outlier percentages (n=100) 12

13. Robust Segmentation 13 Workflow

14. DataMethodsTimeTS PSOSUS Real dataset 2 PCA82.2715727 RD30333.14191811 RANSAC1206.78261560 MSAC1703.67241362 MCMD_Z249.6720 00 MCMD_MD234.4520 00 Performance evaluation in segmentation for planar and non-planar surfaces Segmentation: Complex Surfaces 14 TS: Total Segments PS: Perfect Segments OS: Over Segments US: Under Segments

15. DataMethodsTimeTS PSOSUS Real dataset 2 + 20% noise PCA118.453131810 RD36727.88201811 RANSAC7657.053013111 MSAC8556.87221442 MCMD_Z328.97191801 MCMD_MD311.27191801 Performance evaluation in segmentation for planar and non-planar surfaces Complex Surfaces with Artificial Noise Real points, n = 25,585 + 20% outliers/noise 15

16. Segmentation 16 1,030,398 points 7

17. Segments and Merging 17

18. 18 Road Features Extraction

19. 19 Road Features Extraction

20. Robust Ground Extraction Tricube and bisquare weights Steps 1. Fitting Robust Locally Weighted Regression (RLWR) to get robust polynomial fit for 2D stripe X-Z or Y-Z. Step 2. Down-weighting (i) Calculates residuals for each point. (ii) Use bisquare weight function to down-weight the z-values which are above the fitted line, while rest of the points will be unchanged. (iii) The new set of z-values is used to get the next RLWR polynomial. (iv) Fitting and down-weighting continues until the difference between two consecutive fitted polynomial is very small. (v) The final fit is considered as the ground level, and the points within lowest level and lowest level added with a predefined threshold are identified as ground points.

21. Robust Ground Surface Extraction Dataset 1 Dataset 2 21 Proposed methodRegion growing methodProposed methodRegion growing method

22. Robust Ground Surface Extraction 22 1,703,315 points

23. Conclusions This presentation proposes statistically robust algorithms for (i) planar surface fitting (ii) outlier detection and robust saliency features estimation (iii) segmentation and (iv) ground surfaces extraction. Advantages 23 Diagnostic-robust PCA outperforms non-robust (LS, PCA) methods for local surface fitting. Better results than RANSAC and MSAC in case of thick and small (e.g. n=20) plane. Faster than RANSAC for large datasets. MCMD diagnostics successfully identify high percentage of uniform and cluster outliers. More breakdown point than RANSAC, MSAC Significantly faster than RANSAC and existing statistical robust methods. Give more accurate and robust saliency features than LS, PCA, RANSAC and MSAC. Robust segmentation algorithm is efficient both for planar and non-planar complex surfaces. Outperforms non-robust methods. Reduces over segmentation and under segmentation. The results are more consistent over changes in neighbourhood size and angle threshold. Ground surface extraction method efficiently classify ground and non-ground surface points. Takes less time than region growing based segmentation method. Save time for the remaining point cloud processing tasks.

24. Future Works Limitations:  Similar to other robust techniques, proposed methods (excluding MCMD_MD) breakdown for more than 50% outliers.  MCMD can identify outliers locally not globally. Future works: We are investigating to use robust techniques for different feature extraction, free-form surface reconstruction, different geometric features fitting and object detection and recognition. 24

25. Questions ?