Vision-Guided Robot Position Control SKYNET Tony BaumgartnerBrock Shepard Jeff Clements Norm Pond Nicholas Vidovich Advisors: Dr. Juliet Hurtig & Dr. J.D. Yoder November 12, 2003

Problem Identification Goal: Develop a vision guided robot positioning control system Current systems are limited to Teach- Repeat style On completion the robot will automatically perform tasks specified by the software

Budget/Purchasing A budget of approximately $20,000 has been provided by Ohio Northern University –Robot –Gripper Desktop computer Two CCD (Charge-Coupled Device ) cameras

Background Information Articulated SCARA Selectively Compliant Assembly Robot Arm

Vision Systems Eye-On-Hand Configuration External Stereo Camera Configuration

Prototype Final prototype will include: –CCD camera(s) –Desktop computer –Robot Control Unit –Robot –Gripper

Specifications Robots are measured according to two specifications: –Repeatability –Absolute Accuracy

Object Recognition Algorithms Normalized Cross-Correlation –Slower –Less Sensitive To Light Shape-Based Matching –Faster –Similarity Measure The Sum of Absolute Differences

Difference Algorithm Visuals

Design Deliverables PurchasingTarget Date –Robot09/12/ /21/03 –Gripper09/12/ /21/03 Image Processing –Camera pan/tilt/zoom09/30/ /20/04 –CAD11/03/ /09/04 –Difference Algorithm11/04/ /19/03 –Object Recognition01/09/ /02/04 –User Interface02/02/ /20/04 General –Controller Integration12/01/ /02/04 –Gripper Implementation12/01/ /02/04 –Testing03/08/ /01/04

Overall Block Diagram ControllerCPURobot Gripper Cameras - Pan - Tilt - Zoom Image Processing C++/C# UI 3D Model - Laser - Virtual CAD Algorithm

Software Block Diagram Capture Initial Capture Current Compare Using Difference Read y? Identify Object / Find Coordinates Send Instructions Controller Commands Robot Check Actions Done ?

Robot Decision Matrix 0 – Unacceptable 1 – Acceptable 2 – Average 3 – Good 4 – Excellent 5 – Best Case

QUESTIONS ?

References ABB Product Specification Sheet (2003) Lin, C.T., Tsai D.M. (2002) Fast normalized cross correlation for defect detection. Machine Vision. Yuan-Ze University,1-5. Phil Baratti “Robot Precision” (personal communication, November ) Stegar, C.., Similarity measures for occlusion, clutter, and illumination invariant object recognition. In: B. Radig and S. Florczyk(eds), Mustererkennung 2001, Springer Munchen, pp Stegar, C., Ulrich, M Performance Evaluation of 2D Object Recognition Techniques. Technical Report: Technische Universitat Munchen, Robots and Manufacturing Automation, pg

Software Flow Description Take Initial Picture: –Initialize cameras –Capture image –Store image to hard disk or in RAM (will depend on speed) Take Current Picture: –Start current image capture loop –Capture image and store in memory

Software Flow Description Compare Current Picture With Initial … : –Use difference algorithm to compare initial image and current image – this yields a bitmap of the pixels that have changed based on a specified error factor –Store difference image in memory

Software Flow Description Ready to Manipulate Object? : –Check difference image to see that an object has entered the area – using Boolean to keep track –If object is already present check to see if it has stopped moving –If done moving, act on object, else return to ‘Take Current Picture’ Block

Software Flow Description Identify and Find Object Coordinates … : –Take pictures of scene using both cameras focused on object –Compare the images received from the cameras using the Stereo Vision Algorithm to determine object type and relevant points relative to the position of the robot arm Send Coordinates & Instructions to Robot Controller : –Communicate with robot controller sending incremental instructions based on object type and coordinates

Software Flow Description Use Stereo Vision Algorithm to make sure desired actions … : –Take pictures of scene using both cameras focused on object –Compare the images received from the cameras using the Stereo Vision Algorithm to determine whether the robot arm has moved correctly. After adjusting to mistakes by the robot, send more coordinates and instructions to robot controller until it has performed the correct task