1. MTA-SZTAKI Roboteyepair an ultra-high speed stereo vision system based on CNN technology

2. MTA-SZTAKI Motivation The mechanical properties of robots enable faster operation than their current vision systems; A high-speed, medium resolution, low cost vision system has many potential application fields in the industry; Both stereo and mono versions have potential applications;

3. MTA-SZTAKI Goals Main characteristics of the Roboteyepair: –Standalone vision system; –Capable to capture and process image pairs at frame rates up to 5000 per second; –Medium resolution; –High performance focal plane CNN processors; –Equipped with serial and Ethernet communication ports; –Developed for industrial use.

4. MTA-SZTAKI The CNN visual processor 128x128 sensor-processor array; Optimized for performing array operations on analog and logic values; Time to perform a typical operation: –Logic: 300ns (morphology, binary logic operations, skeletonizing, etc.) –Analog: 3-5ms (diffusion, edge detection, Sobel, etc..) 8 internal image memories; 128Mpixel/sec I/O speed (128 internal AD/DA) 0.35mm technology, 12x12mm silicon surface; Operates as a coprocessor, a master processor is needed

5. MTA-SZTAKI Block diagram of the system

6. MTA-SZTAKI Realization of the system

7. MTA-SZTAKI The finished boards

8. MTA-SZTAKI The complete system

9. MTA-SZTAKI Programming the DSP Image processing –Native languages (AMC, Alpha, C) Handling two CNN chips –The target chip has to be specified for every command –Synchronization Adapting CNN image processing library Communication using the ETRAX chip

10. MTA-SZTAKI Programming the communication processor ETRAX 100 2.4 LINUX kernel –Driver (ETRAX  XBUS) TCP/IP communication module (INET Daemon) –Assigns the communication driver to the given TCP port –standard input  standard output redirecting A special daemon has also been implemented –Boot device

11. MTA-SZTAKI Monitoring software on the PC side Why is it needed: –Supporting system setup –Application development and testing –Parameter adjustment –Testing camera parameters Capabilities: –Multi-window graphical interface –Editor, compiler and software downloading devices

12. MTA-SZTAKI The project will be finished in May, 2003

13. MTA-SZTAKI Stereo machine vision system for robots; Visual navigation for NASA Mars Mission 2007 Non-invasive speed and displacement measurement, quality control of narrow textile bands and tapes (textile industry); Classification of large number of tiny objects (pills, grains, pebbles, etc.); Monitoring fast industrial processes (sparkplug); Applications

14. MTA-SZTAKI Replace the current DSP with a TMSC3206115 DSP running @600 MHz Equip the system with an optional 1.3 MPixel CMOS sensor Support of communication via USB port These developments are expected to be finished by September, 2003 Future developments

15. MTA-SZTAKI Thank you for your attention!