Segmentation and classification of man-made maritime objects in TerraSAR-X images IEEE International Geoscience and Remote Sensing Symposium Vancouver, Canada July 27th 2011 Michael Teutsch, email: michael.teutsch@iosb.fraunhofer.de Günter Saur, email: guenter.saur@iosb.fraunhofer.de

Outline Motivation Concept Segmentation Classification Examples Conclusions and future work

Motivation I Applications: Tracking of cargo ship traffic Surveillance of fishery zones, harbours, shipping lanes Detection of abnormal ship behaviour, criminal activities Search for lost containers or hijacked ships Aims / Challenges: Detection of man-made objects (not here) Precise orientation and size estimation Separation of clutter, non-ships, different ship types Robustness against various SAR-specific noise effects Fast processing time Here: Analyze object appearance, avoid models and prior knowledge

Motivation II: Difficult examples

Concept

Pre-processing 3x3 median filter Ground Sampling Distance (GSD) normalization to 2.0 meters/pixel

Segmentation I: Structure-emphasizing LBP filter Local Binary Pattern: Rotation invariant uniform LBPs: Texture primitives: Timo Ojala et al., „Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 7, July 2002.

Segmentation II: Structure-emphasizing LBP filter Rotation invariant uniform LBPs (texture primitives): Rotation invariant variance measure: For each pixel position (x,y), fixed P, and varying R:

Segmentation III: Rotation compensation with HOG A. Korn, „Toward a Symbolic Representation of Intensity Changes in Images“, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 10, no. 5, 1988.

Segmentation IV: Rotation compensation with HOG+PCA FUSION

Segmentation V: Size estimation with row/col. histograms

Segmentation VI: Experimental data set 17 different TerraSAR-X StripMap images 756 manually labeled detections including orientation and length No ground truth, manual labeling is sensed truth Labeling inspired by CFAR-detection including potential clutter Scale normalization to 2.0 meters / pixel

Segmentation VII: Orientation and size estimation results method rotation estimation error median mean LBP & HOG & PCA with median filter 5.24° 11.65° without median filter 5.99° 12.16° LBP & HOG 6.71° 12.99° LBP & PCA 12.09° 24.38° HOG only 10.68° 23.36°

Segmentation VIII: Examples

Classification I: Classes clutter (ambiguity) non-ship ship structure 1 clutter unstructured ship ship structure 2

Classification II: Concept G. Saur, M. Teutsch, „SAR signature analysis for TerraSAR-X based ship monitoring“, Proceedings of SPIE Vol. 7830, 2010. M. Teutsch, W. Krüger, „Classification of small Boats in Infrared Images for maritime Surveillance“, 2nd International Conference on WaterSide Security (WSS), Marina di Carrara, Italy, Nov. 3-5, 2010.

Classification III: Experiments and results 5 classes: clutter, non-ship, unstr. ship, structure 1, structure 2 543 samples with good segmentation and possible manual labeling: 53 clutter, 110 non-ship, 322 unstr. ship, 17 structure 1, 41 stucture 2 362 training samples and 181 test samples Runtime for segmentation and classification: ~ 2 sec per detection Classification results: classifier SVM 1 SVM 2 3-NN cascade correct rate 96.68 % 93.29 % 91.45 % 80.66 %

Classification IV: Examples clutter unstructured ship unstructured ship non-ship unstructured ship ship structure 1

Classification V: Examples for whole processing chain ship structure 2 unstructured ship ship structure 2

Conclusions Future work Aim: Segmentation and classification of man-made objects in satellite SAR Challenge: Robustness against various object appearances, noise effects Segmentation: Pre-processing, structure-emphasizing filter with LBPs, orientation estimation with HOGs and PCA, size estimation with row/column histograms, median orientation estimation error: 5.2° Classification: Extensive feature calculation, feature evaluation and selection, classification with cascaded SVM and 3-NN, 81% correct classification Future work Improve size estimation (LBPs instead of row/column histograms?) More data for classification (esp. structure classes) Other approaches for 3rd classification-stage (local features?) Is object structuredness and classifiability based on appearance measurable?

Thanks a lot for your attention! Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB Thanks a lot for your attention! Karlsruhe Ettlingen Ilmenau

Segmentation: Orientation estimation error distrib.

Segmentation: Examples – The bad guys