Presentation is loading. Please wait.

Presentation is loading. Please wait.

Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa

Similar presentations


Presentation on theme: "Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa"— Presentation transcript:

1 Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa Laboratório de Técnicas Inteligentes Escola Politécnica da Universidade de São Paulo Carlos Henrique Costa Ribeiro Divisão de Ciência da Computação Instituto Tecnológico de Aeronáutica

2 MultiBot - Meeting #1, Lisboa part I2 Preface This is a two-part talk about the research on Intelligent Mobile Robotics and related topics at: LTI – USP (Laboratório de Técnicas Inteligentes – Universidade de São Paulo, Brazil) NCROMA-ITA (Laboratório de Navegação e Controle de Robôs Móveis Autônomos – Instituto Tecnológico de Aeronáutica, Brazil). These research groups are involved in the MultiBot cooperation project CAPES/GRICES with ISLab-IST.

3 MultiBot - Meeting #1, Lisboa part I3 LTI - EPUSP Prof. Anna Reali 5 PhD Students Alexandre Simões *, Reinaldo Bianchi, Valdinei Silva *, Valguima Odakura, Waldemar Bonventi. 3 Master Students Alexandre Cunha *, Antônio Selvatici, Luiz Carlos Maia 3 Undergraduate Students Rafael Pacheco, Márcio Seixas, Júlio Kawai 2 Final Course Projects

4 MultiBot - Meeting #1, Lisboa part I4 NCROMA – ITA Prof. Carlos Ribeiro 1 PhD Student Letícia Friske 5 Master Students Luís Almeida, Ricardo Maia, Juliano Pereira, Esther Colombini *, Celeny Alves * 2 Undergraduate Students Lucas Gabrielli, Fábio Miranda

5 MultiBot - Meeting #1, Lisboa part I5 Intelligent Mobile Robotics Part 1: Navigation Map building Localization Perception Computer Vision Part II: Learning Contents

6 MultiBot - Meeting #1, Lisboa part I6 Map Building Our goal: Test map building algorithms in real robots fast enough? precise enough? ok for learning applications (e.g. path learning)?

7 MultiBot - Meeting #1, Lisboa part I7 Map Building Sildomar Takahashi, Carlos Ribeiro Roberto Barra, Ricardo Domenecci, Anna Reali Efficient Learning of Variable-Resolution Cognitive Maps for Autonomous Indoor Navigation. Arleo, Millan e Floreano, IEEE-SMC, Advantages: Complete algorithmic description Simple structure Limitations: Assumes structure (orthogonal obstacles / walls) Reliance on dead-reckoning (but can be adapted to more sophisticated localization)

8 MultiBot - Meeting #1, Lisboa part I8 The Basic Algorithm 1. Explores environment; 2. Once an obstacle is detected: 1. Determines obstacle frontiers either via: An a priori sensor model A pre-trained neural net 2. Includes obstacle in the global map; 3. Defines new partition to explore. If there is none, END. Else finds route to new partition. 4. Executes route and explores new partition. Once an obstacle is detected, Step 2. Else, Step 1.

9 MultiBot - Meeting #1, Lisboa part I9 Detection of obstacle frontiers Robô Células Ocupadas Reta Calculada Either a priori model or neural net model Integration over time Straight-line adjustment and correction (according to a priori actuator model) 0 y x My Mx

10 MultiBot - Meeting #1, Lisboa part I10 Very Scientific Set-up Walls Obstacles Robot 3 x 3,5 m

11 MultiBot - Meeting #1, Lisboa part I11 Global Maps: Magellan, Neural net model Map 1 Map 2 Map 3

12 MultiBot - Meeting #1, Lisboa part I12 Global Map: Pioneer, a priori model + straight-line model-based correction

13 MultiBot - Meeting #1, Lisboa part I13 Conclusions Tested algorithm (possibly with some modifications) is a good compromise efficiency/precision for realistic applications: fast yet fairly accurate. Next steps: Studies on simultaneous localization and mapping (SLAM algorithms). Valguima Odakura (Anna Reali): SLAM based on visual landmarks. Fabio Miranda (Carlos Ribeiro): Bayesian landmark learning. Techniques for map building acceleration.

14 MultiBot - Meeting #1, Lisboa part I14 Markov Localization Luís Almeida, Carlos Ribeiro Júlio Kawai, Anna Reali Position estimation based on Bayesian update: Belief update based on sensor info Belief update based on action info Sensor/actuator models and initial belief distribution: arbitrary. Simple to implement. Computationally costly (Monte Carlo implementation – particle filters – is a possible fix).

15 MultiBot - Meeting #1, Lisboa part I15 Markov Localization Probabilistic position grid Action Model Sensor Model Markov State Estimator ptpt stst atat p t+1 odometers sonars

16 MultiBot - Meeting #1, Lisboa part I16 Monte Carlo Localization + GA Optimization Luís Almeida, Carlos Ribeiro Standard Markov update (over set of particles) MC GA GA on population of particles (fitness as combination of belief / particle cluster distribution) Basic idea: use GA to create a better set of particles for next MC update. Initial results: ok (in need of statistical validation).

17 MultiBot - Meeting #1, Lisboa part I17 Next Steps Validation of GA approach. Better sensor and actuator models. Implementation in a real robot. Literature on Monte Carlo methods (applications on signal detection and tracking): many variations to be tried...

18 MultiBot - Meeting #1, Lisboa part I18 Computational Vision Image Segmentation Using Color Classification Using Background Model Using Optical Flow Based on Binocular Stereo Vision

19 MultiBot - Meeting #1, Lisboa part I19 Color Classification - I Using threshold values: In the color representation space Neural Network – MLP + backpropagation alg.: Alexandre Simões, Anna Reali

20 MultiBot - Meeting #1, Lisboa part I20 Derived Application: Alexandre Simões, Anna Reali C1 C2 C3 C4 C5 Orange Classifier Branco... Verde Claro Verde Escuro Amarelo Laranja Claro Laranja Escuro R G B Danificado Orange Classifier - CEAGESP, SP

21 MultiBot - Meeting #1, Lisboa part I21 For number of clusters = 2 to Cmax, do: Apply FCM-GK (non-supervised fuzzy classifier) to RGB image; Calculate the ratio c/s for each cluster set: c = Cluster dimension/number of members s = Separation among clusters Choose the cluster set, based on c/s. Show color classification result for the best cluster set. Non-supervised iterative fuzzy color classification Waldemar Bonventi, Anna Reali

22 MultiBot - Meeting #1, Lisboa part I22 An example: soccer The best cluster set 6 clusters

23 MultiBot - Meeting #1, Lisboa part I23 Another example: Rio de Janeiro The best cluster set 3 clusters

24 MultiBot - Meeting #1, Lisboa part I24 Computational Vision Image Segmentation Using Color Classification Using Background Model Using Optical Flow Based on Binocular Stereo Vision

25 MultiBot - Meeting #1, Lisboa part I25 Background Model - I Background subtraction Thresholding the error between an estimate of the image without moving objects – M(C) – and the current image: Model can not adapt to environment changes! M(C) Current Image

26 MultiBot - Meeting #1, Lisboa part I26 Background Model – II Márcio Seixas, Anna Reali Time-Adaptive, Per-Pixel Mixture-of-Gaussians: Time series of observations at a given pixel (its color) is modeled by a mixture-of-gaussians. Based on the persistence and the variance of each of the gaussians of the mixture, it is determined which gaussians may correspond to background colors. Hypothesis: gaussian distributions with low variance and high persistence correspond to background model. Per-pixel models are updated as new observations are obtained (according to a learning rate). It is capable of dealing with long-term scene changes (e.g. lighting changes)!

27 MultiBot - Meeting #1, Lisboa part I27 Derived Application: platform occupancy Fixed model M(C): Original: Original – M(C): Adaptive Model: Terminal Rodoviário de Santo Amaro TRENDS & Prefeitura de São Paulo Márcio Seixas, Anna Reali

28 MultiBot - Meeting #1, Lisboa part I28 Computational Vision Image Segmentation Using Color Classification Using Background Model Using Optical Flow Based on Binocular Stereo Vision

29 MultiBot - Meeting #1, Lisboa part I29 Optical Flow - idea

30 MultiBot - Meeting #1, Lisboa part I30 Robot (with camera) navigating in a stationary scenario. Calculation of the optical-flow divergent to estimate the time- to-crash value in order to avoid collisions with obstacles. We are now investigating a robust method to directly calculate the per-pixel time-to-crash value: Vision-based robotic behavior: Antonio Selvatici, Anna Reali Gray levels: near bright; far dark Black: unknown distance Original sequencePixel time-to-crashFiltered values

31 MultiBot - Meeting #1, Lisboa part I31 Derived Application: monitoring of underground rail tracks Luiz Maia, Anna Reali ALSTOM & Metrô de São Paulo

32 MultiBot - Meeting #1, Lisboa part I32 Computational Vision Image Segmentation Using Color Classification Using Background Model Using Optical Flow Based on Binocular Stereo Vision

33 MultiBot - Meeting #1, Lisboa part I33 Distance-Map Calculation 1. Calibration [Zhang, ICCV 99] 2. Matching – blob coloring+centroid+correlation 3. Triangulation Segmentation: based on color + distance-map. Binocular Stereo Vision Rafael Pacheco, Anna Reali

34 MultiBot - Meeting #1, Lisboa part I34 Derived Application: Outdoors Measurement TRENDS & Prefeitura de São Paulo Rafael Pacheco, A. Reali

35 MultiBot - Meeting #1, Lisboa part I35 Conclusions In CV, we are now investigating: Automatic learning of fuzzy color classifiers – Waldemar Bonventi, LTI ; A framework for high-level feedback to adaptive, per- pixel, mixture-of-gaussian background models – Márcio Seixas, LTI ; Mathematical formulation for direct and robustly calculate the per-pixel, time-to-crash values, considering a moving observer in a stationary scenario – Antonio Selvatici, LTI ; Distributed, real-time approach to calculate the optical flow, considering a stationary observer in a dynamic scenario – Luiz Maia, LTI.


Download ppt "Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa"

Similar presentations


Ads by Google