Manipulation in Human Environments Aaron Edsinger & Charlie Kemp Humanoid Robotics Group MIT CSAIL

Domo

Manipulation in Human Environments Work with everyday objects Collaborate with people

Applications Aging in place Cooperative manufacturing Household chores

Three Themes Use Your Body Social Manipulation Task relevant features

Use Your Body Simplify perception (tool tip, hand) Test assumptions (flat surface) Compliance simplifies contact (Placing, Grasping and Transferring)

Structure In Human Environments Sense from above Flat surfaces Objects for human hands Objects for use by humans Look the user in the eye Interpretable body Tall and narrow

Social Complementary action Person can simplify perception and action for the robot Robot can cue the human intuitively (body language) Lot's of examples of tasks where a robot can be helpful without doing everything (robot doesn't have to solve everything to be helpful)

Task Relevant Features What is important? What is irrelevant? *Distinct from object detection/recognition.

Other Examples Donald Norman Circular openings Tips Handles Contact Surfaces

Why are tool tips common? Single, localized interface to the world Physical isolation helps avoid irrelevant contact Helps perception Helps control

Distinct Perceptual Problem Not object recognition How should it be used Distinct methods and features

Generalize What You've Learned Across objects Perceptually map tasks across objects key features -> key features Across manipulators Motor equivalence Manipulator details may be irrelevant

Tool Tip Detection Visual + motor detection method Kinematic Estimate Visual Model

Acquire a Visual Model

Use The Hand's Frame Combine weak evidence Rigidly grasped

RSS 2006 Workshop Manipulation for Human Environments Much to be done!

Summary Importance of Task Relevant Features Example of the tool tip Large set of hand tools Robust detection (visual + motor) Kinematic estimate Visual model

In Progress Perform a variety of tasks Insertion Pouring Brushing

Mean Pixel Error for Automatic and Hand Labeled Tip Detection

Mean Pixel Error for Hand Labeled, Multi-Scale Detector, and Point Detector

Learning from Demonstration

The Detector Responds To Fast Motion Convex

Video from Eye Camera Motion Weighted Edge Map Multi-scale Histogram (Medial-Axis, Hough Transform for Circles) Local Maxima

Defining Characteristics Geometric Isolated Distal Localized Convex Cultural/Design Far from natural grasp location Long distance relative to hand size

Other Task Relevant Features?

Detecting the Tip

Include Scale and Convexity