Visual SLAM Visual SLAM SPL Seminar (Fri) Young Ki Baik Computer Vision Lab.

Outline  What is SLAM?  What is Visual SLAM?  Overall process and Problem  Advances and challenges  Conclusion

What is SLAM?   SLAM : Simultaneous Localization and Mapping is a technique used by robots and autonomous vehicles to build up a map within an unknown environment while at the same time keeping track of their current position. Where am I ? Map building Observation

What is SLAM?   SLAM : Simultaneous Localization and Mapping basically uses some statistical techniques based on recursive Bayesian estimation such as Kalman filters and particle filters (aka. Monte Carlo methods).

What is Visual SLAM?   SLAM : Simultaneous Localization and Mapping can use many different types of sensor to acquire observation data used in building the map such as laser rangefinders, sonar sensors and cameras. Visual SLAM - is to use cameras as a sensor.

Why Visual SLAM?  Vision data can inform us more meaningful information (such as color, texture, shape…) relative to other sensors.

Overall process of Visual SLAM Map management Map management Measurement Initialization Prediction Update

Visual SLAM DEMO Mono-slam

Problems   Proposal   Data association   Filter   Map management   Real-time

Proposal   Odometry Most case of localization system, odometry information is provided as a prior knowledge to predict poses. Use Encoder to Distance & Angle Estimation in Forward movement case of vehicles. Continuously Update Distance & Angle Change + Odometry info. Left Encoder Distance Right Encoder Distance Angle Change Distance Change

Proposal   But… Hand held camera system (monocular camera)does not provide odometry. t t+1 ?

Tricks   Fixed Position, Velocity assumption   Localization can be failed when abrupt motion and/or sudden change are occurred.   The Solution still have to be found!!! P t+1 =P t +N P t+1 =P t + ∇ t( V t +N )

Data association   What is data association? To find matches between observed data and map. ?

Data association   For data association, at the beginning… Small (e.g. 11x11) image patches around salient points to represent features. Normalized Cross Correlation(NCC) to match features. Small patches + accurate search regions lead to fast pose estimation.

Data association   However Simple patches are insufficient for large view point or scale variantions. Small patches help speed but prone to mismatch. Search regions can’t always be trusted. (camera occlusion, motion blur)

Data association   A solution Use a richer feature descriptor such as SIFT-like descriptors. SIFT descriptor

Data association DEMO SLAM - SIFT vs NCC

Data association DEMO SLAM - SIFT

Data association   A solution Use other information. such as edge, line, contour, etc.

Data association   A good solution Is not yet found satisfying richer information and fast processing speed simultaneously. Richer descriptor Speed down

Which filter is appropriate?   Extended Kalman Filter (EKF) Optimal Minimum Mean Square Error (MMSE) estimator of the state by approximating 1 st order Talyor expansion. Uncented Kalman Filter (UKF) : 2 nd order Talyor approximation   Particle Filter (PF) Based on Sequential Monte Carlo.

EKF vs PF EKF Estimate 1 st order Talyor approximation. Scale of map Square of number of features Particle Filter Does not limit order of TA. Scale of map The increase is linear in relation to number of particles. N features x N features N particles x N features PF based approach is regarded more robust and faster than EKF based approach.

What filter appropriate to use? DEMO PF-based mono-SLAM

What filter appropriate to use? DEMO PF-based mono-SLAM

Map management   Scale of map Linearly increased in relation to number of features. Partitioned map (or grid map) is proposed as a solution.

Map management   A solution Log-likely increasing method in relation to number of features or particles (still not be found). scale N of features or particles

Is process able to work on-line?   SLAM problem have to be solved in real time. Fast Algorithm Parallel processing (SIMD, GPGPU, etc.) Speed!!! What’s needed for right now???

Some Challenges   Deal with large maps   Use different feature kinds on an informed way.   Benefit from other approaches such as SFM but keep efficiency.   Incorporate semantics and beyond- geometric scene.

Conclusion   SLAM problem is mathematically solved.   Computer vision implementation still be improving: Feature description Kind of features Parameterization and robustness

Q&AQ&A