1. Segmentation and classification of man-made maritime objects in TerraSAR-X images
IEEE International Geoscience and Remote Sensing Symposium Vancouver, Canada
July 27th 2011
Michael Teutsch,
Günter Saur,

2. Outline Motivation Concept Segmentation Classification Examples
Conclusions and future work

3. Motivation I Applications: Tracking of cargo ship traffic
Surveillance of fishery zones, harbours, shipping lanes
Detection of abnormal ship behaviour, criminal activities
Search for lost containers or hijacked ships
Aims / Challenges:
Detection of man-made objects (not here)
Precise orientation and size estimation
Separation of clutter, non-ships, different ship types
Robustness against various SAR-specific noise effects
Fast processing time
Here: Analyze object appearance, avoid models and prior knowledge

4. Motivation II: Difficult examples

5. Concept

6. Pre-processing 3x3 median filter
Ground Sampling Distance (GSD) normalization to 2.0 meters/pixel

7. Segmentation I: Structure-emphasizing LBP filter
Local Binary Pattern:
Rotation invariant uniform LBPs:
Texture primitives:
Timo Ojala et al., „Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 7, July 2002.

8. Segmentation II: Structure-emphasizing LBP filter
Rotation invariant uniform LBPs (texture primitives):
Rotation invariant variance measure:
For each pixel position (x,y), fixed P, and varying R:

9. Segmentation III: Rotation compensation with HOG
A. Korn, „Toward a Symbolic Representation of Intensity Changes in Images“,
IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 10, no. 5, 1988.

10. Segmentation IV: Rotation compensation with HOG+PCA
FUSION

11. Segmentation V: Size estimation with row/col. histograms

12. Segmentation VI: Experimental data set
17 different TerraSAR-X StripMap images
756 manually labeled detections including orientation and length
No ground truth, manual labeling is sensed truth
Labeling inspired by CFAR-detection including potential clutter
Scale normalization to 2.0 meters / pixel

13. Segmentation VII: Orientation and size estimation results
method
rotation estimation error
median
mean
LBP & HOG & PCA
with median filter
5.24°
11.65°
without median filter
5.99°
12.16°
LBP & HOG
6.71°
12.99°
LBP & PCA
12.09°
24.38°
HOG only
10.68°
23.36°

14. Segmentation VIII: Examples

15. Classification I: Classes
clutter (ambiguity)
non-ship
ship structure 1
clutter
unstructured ship
ship structure 2

16. Classification II: Concept
G. Saur, M. Teutsch, „SAR signature analysis for TerraSAR-X based ship monitoring“, Proceedings of SPIE Vol. 7830, 2010.
M. Teutsch, W. Krüger, „Classification of small Boats in Infrared Images for maritime Surveillance“, 2nd International Conference on WaterSide Security (WSS), Marina di Carrara, Italy, Nov. 3-5, 2010.

17. Classification III: Experiments and results
5 classes: clutter, non-ship, unstr. ship, structure 1, structure 2
543 samples with good segmentation and possible manual labeling:
53 clutter, 110 non-ship, 322 unstr. ship, 17 structure 1, 41 stucture 2
362 training samples and 181 test samples
Runtime for segmentation and classification: ~ 2 sec per detection
Classification results:
classifier
SVM 1
SVM 2
3-NN
cascade
correct rate
96.68 %
93.29 %
91.45 %
80.66 %

18. Classification IV: Examples
clutter
unstructured ship
unstructured ship
non-ship
unstructured ship
ship structure 1

19. Classification V: Examples for whole processing chain
ship structure 2
unstructured ship
ship structure 2

20. Conclusions Future work
Aim: Segmentation and classification of man-made objects in satellite SAR
Challenge: Robustness against various object appearances, noise effects
Segmentation: Pre-processing, structure-emphasizing filter with LBPs, orientation estimation with HOGs and PCA, size estimation with row/column histograms, median orientation estimation error: 5.2°
Classification: Extensive feature calculation, feature evaluation and selection, classification with cascaded SVM and 3-NN, 81% correct classification
Future work
Improve size estimation (LBPs instead of row/column histograms?)
More data for classification (esp. structure classes)
Other approaches for 3rd classification-stage (local features?)
Is object structuredness and classifiability based on appearance measurable?

21. Thanks a lot for your attention!
Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB
Thanks a lot for your attention!
Karlsruhe
Ettlingen
Ilmenau

22. Segmentation: Orientation estimation error distrib.

23. Segmentation: Examples – The bad guys