EFFICIENT ROAD MAPPING VIA INTERACTIVE IMAGE SEGMENTATION Presenter: Alexander Velizhev CMRT’09 ISPRS Workshop O. Barinova, R. Shapovalov, S. Sudakov, A. Velizhev, A. Konushin

Introduction Roadway monitoring systems are widely-used for supervising road pavement surface and repair planning

Problem statement Analysis road pavement only by video sequences

Problem statement (2) Object types: – Lane marking – Road patches and defects Solution requirements: – High object detection rate – Maximum automation

Problem statement (3) Source imageExpected result

Problem details Some real examples

Our algorithm outline 1. Video rectification 2. Image preprocessing 3. Image segmentation 4. Features calculation 5. Interactive classification Automatic offline stage Interactive online stage

Video rectification Image preprocessing Image segmentation Features calculation Interactive classification Video rectification Using of raw video has severe drawbacks: – Objects are represented with different spatial resolution on the same frame – Projective distortions – Elongated objects exceed the bounds of single frame

Video rectification (2) Image preprocessing Image segmentation Features calculation Interactive classification Video frames are converted to orthogonal projection and stitched to each other Video rectification

Image preprocessing Image segmentation Features calculation Interactive classification Video rectification Retinex transform Contrast adjustment Bilateral filter Image preprocessing Source image

Image segmentation Image preprocessing Image segmentation Features calculation Interactive classification Video rectification Main goal is representing all objects of interest as different segments We use the hierarchical version of mean shift algorithm

Features calculation Image preprocessing Image segmentation Features calculation Interactive classification Video rectification More than 100 various features are used for classification of segments Feature types: – Colour statistics (colour variance, Lab components’ percentiles,... ) – Shape statistics (elongation, orientation, area, …) – Difference with neighborhood of the segments

Interactive classification Image preprocessing Image segmentation Features calculation Interactive classification Video rectification

Interactive classification (2) Image preprocessing Image segmentation Features calculation Interactive classification Video rectification User manually marks object segments Learning of cascade of classifiers Automatic classification of the next road part User corrects classification results End Start

Cascade of classifiers Cascade of classifiers corresponds image segmentation levels We descend a hierarchy from large to small segments and reject segments that do not contain pixels of objects of interest Classifier training uses the data passed to a corresponding cascade layer by preceding version of cascade

Why do we use the cascade? To solve a problem of unbalanced classes To speed-up classification

Online learning We introduce an online version of the random forest algorithm Special class costing The algorithm’s code is a part of our open source “GML Balanced On-line Learning Toolkit ” – balanced-on-line-learning-toolkit-2.html balanced-on-line-learning-toolkit-2.html

Why do we use online learning? We don’t need to store all training database in memory Short learning time User actions immediately impact on the classification results

How to measure system efficiency ? We are modeling “ideal” user actions to measure the efficiency of the interactive classification Efficiency criterion: – a minimal number of mouse’s clicks for making correct classification

Results Source image Segmented image Analysis result

Results (2) Image part Clicks Manual classification Interactive classification

Results (3) Image part Error, %

Summary We present a tool for efficient interactive mapping of road defects and lane marking Intensive use of computer vision methods on different stages of our data processing workflow increases usability of the tool

Weak points Image segmentation errors can degrade classifier and true object bounds cannot be extracted Algorithm is not robust to user mistakes

Future work Ultimate goal: Development of the universal semantic segmentation system which can be used for object extraction from large class of images Nearest plan: Improving the quality of image segmentation by integration colour and range data

CMRT’09 ISPRS Workshop Efficient road mapping via interactive image segmentation O. Barinova R. Shapovalov S. Sudakov A. Konushin A. Velizhev