Robot Vision with CNNs: a Practical Example P. Vitullo P. Campolucci G. Apicella L. Pompeo D. Bellachioma S. Graziani M. Balsi Dep. of Electronic Engineering “La Sapienza” Univ. of Rome, Italy X. Vilasís–Cardona S. Luengo J. Solsona R. Funosas A. Maraschini A. Aznar V. Giovenale P. Giangrossi Barcelona, 19/2/03

Framework of this work completely autonomous robot simple (cheap) hardware vision-based guidance –short term: line following –longer term: navigation in a real environment

Architecture Cellular Neural Networks to handle all the image processing Fuzzy-rule-based navigation

Cellular Neural Networks Fully parallel analog vision chips Capable of real-time nonlinear image processing and feature detection Algorithmically programmable to implement complex operations On-board image acquisition (focal-plane processing)

Cellular Neural Networks Recurrent Neural (?) Network Locally connected  VLSI-friendly Space-invariant synapses (cloning templates) –small number of parameters: explicit design Continuous variables – analog computing (discrete-time model for digital)

Topology Locally connected  VLSI Space-invariant synapses

Discrete–time model Binary state variable Analog or binary input depending on implementation

Application Input ports: analog arrays u, x(0) Output port: binary array x(  ) “Analog instruction”: {A,B,I} (cloning template) Feature detection (nonlinear image filtering)

CNN “Universal” Machine Local memory Global control (broadcasting cloning templates and memory transfer commands) “Analogic” computing: stored-program analog/logic algorithms

Task: line following The robot is to follow a maze of straight lines crossing at approximately right angles Functions required by vision module: Acquiring image, cleaning, thinning lines Measuring orientation/displacement of lines

Image processing algorithm Image acquisition Binarization Line thinning

Image processing algorithm (ctd.) Directional line filtering Projection

Fuzzy control

Simulation

el cochecito (Barcelona) control (386) CNN emul. (DSP)

Visibilia (Rome) PAL B/W CAMERA FPGA-based CNN emulator Celoxica RC-100 board Xilinx Spartan II 200Kgates microcontroller Jackrabbit BL1810 PIC 16F84 SERVO MOTOR (steering) LCD PS/2 mouse port Rabbit2000 microcontroller Parallel port E Parallel port ASerial port D STEPPER MOTOR (advancing) STEPPER MOTOR CONTROLLER

Celoxica RC-100 VGA

Jackrabbit BL1810

driving start vert hor follow vert hor Y Y N N Y N normal driving crossing timer:=0 timer>10s N Y store left avail. turn left if avail. else right diag (L/R) Y follow diag Y N

Continuation of the work more realistic tasks: obstacle avoidance navigation in a real-life environment

Obstacle avoidance using other sensors together with vision, e.g. ultrasound monocular range evaluation local path-finding strategies

Hybrid (topological/metric) navigation

door recognition

Robot Vision with CNNs: a Practical Example M. Balsi Dep. of Electronic Engineering “La Sapienza” Univ. of Rome, Italy Barcelona, 19/2/03