1. Robot Vision with CNNs: a Practical Example P. Vitullo P. Campolucci G. Apicella L. Pompeo D. Bellachioma S. Graziani M. Balsi Dep. of Electronic Engineering “La Sapienza” Univ. of Rome, Italy X. Vilasís–Cardona S. Luengo J. Solsona R. Funosas A. Maraschini A. Aznar V. Giovenale P. Giangrossi Barcelona, 19/2/03

2. Framework of this work completely autonomous robot simple (cheap) hardware vision-based guidance –short term: line following –longer term: navigation in a real environment

3. Architecture Cellular Neural Networks to handle all the image processing Fuzzy-rule-based navigation

4. Cellular Neural Networks Fully parallel analog vision chips Capable of real-time nonlinear image processing and feature detection Algorithmically programmable to implement complex operations On-board image acquisition (focal-plane processing)

5. Cellular Neural Networks Recurrent Neural (?) Network Locally connected  VLSI-friendly Space-invariant synapses (cloning templates) –small number of parameters: explicit design Continuous variables – analog computing (discrete-time model for digital)

6. Topology Locally connected  VLSI Space-invariant synapses

7. Discrete–time model Binary state variable Analog or binary input depending on implementation

8. Application Input ports: analog arrays u, x(0) Output port: binary array x(  ) “Analog instruction”: {A,B,I} (cloning template) Feature detection (nonlinear image filtering)

9. CNN “Universal” Machine Local memory Global control (broadcasting cloning templates and memory transfer commands) “Analogic” computing: stored-program analog/logic algorithms

10. Task: line following The robot is to follow a maze of straight lines crossing at approximately right angles Functions required by vision module: Acquiring image, cleaning, thinning lines Measuring orientation/displacement of lines

11. Image processing algorithm Image acquisition Binarization Line thinning

12. Image processing algorithm (ctd.) Directional line filtering Projection

13. Fuzzy control

14. Simulation

15. el cochecito (Barcelona) control (386) CNN emul. (DSP)

16. Visibilia (Rome) PAL B/W CAMERA FPGA-based CNN emulator Celoxica RC-100 board Xilinx Spartan II 200Kgates microcontroller Jackrabbit BL1810 PIC 16F84 SERVO MOTOR (steering) LCD PS/2 mouse port Rabbit2000 microcontroller Parallel port E Parallel port ASerial port D STEPPER MOTOR (advancing) STEPPER MOTOR CONTROLLER

18. Celoxica RC-100 VGA

19. Jackrabbit BL1810

20. driving start vert hor follow vert hor Y Y N N Y N normal driving crossing timer:=0 timer>10s N Y store left avail. turn left if avail. else right diag (L/R) Y follow diag Y N

21. Continuation of the work more realistic tasks: obstacle avoidance navigation in a real-life environment

22. Obstacle avoidance using other sensors together with vision, e.g. ultrasound monocular range evaluation local path-finding strategies

23. Hybrid (topological/metric) navigation

24. door recognition

25. Robot Vision with CNNs: a Practical Example M. Balsi Dep. of Electronic Engineering “La Sapienza” Univ. of Rome, Italy balsi@uniroma1.it Barcelona, 19/2/03