1. Manipulation in Human Environments
Aaron Edsinger & Charlie Kemp
Humanoid Robotics Group
MIT CSAIL

2. Domo

3. Manipulation in Human Environments
Work with everyday objects
Collaborate with people

4. Applications
Aging in place
Cooperative manufacturing
Household chores

5. Three Themes
Use Your Body
Social Manipulation
Task relevant features

6. Use Your Body Simplify perception (tool tip, hand)
Test assumptions (flat surface)
Compliance simplifies contact (Placing, Grasping and Transferring)

7. Structure In Human Environments
Sense from above
Flat surfaces
Objects for human hands
Objects for use by humans
Look the user in the eye
Interpretable body
Tall and narrow

8. Social Complementary action
Person can simplify perception and action for the robot
Robot can cue the human intuitively (body language)
Lot's of examples of tasks where a robot can be helpful without doing everything (robot doesn't have to solve everything to be helpful)

9. Task Relevant Features
What is important?
What is irrelevant?
*Distinct from object detection/recognition.

10. Other Examples Donald Norman Circular openings Tips Handles
Contact Surfaces

11. Why are tool tips common?
Single, localized interface to the world
Physical isolation helps avoid irrelevant contact
Helps perception
Helps control

12. Distinct Perceptual Problem
Not object recognition
How should it be used
Distinct methods and features

13. Generalize What You've Learned
Across objects
Perceptually map tasks across objects
key features -> key features
Across manipulators
Motor equivalence
Manipulator details may be irrelevant

15. Tool Tip Detection Visual + motor detection method Kinematic Estimate
Visual Model

20. Acquire a Visual Model

21. Use The Hand's Frame
Combine weak evidence
Rigidly grasped

24. RSS 2006 Workshop
Manipulation for Human Environments
Much to be done!

25. Summary Importance of Task Relevant Features Example of the tool tip
Large set of hand tools
Robust detection (visual + motor)
Kinematic estimate
Visual model

26. In Progress
Perform a variety of tasks
Insertion
Pouring
Brushing

27. Mean Pixel Error for Automatic and Hand Labeled Tip Detection

28. Mean Pixel Error for Hand Labeled, Multi-Scale Detector, and Point Detector

29. Learning from Demonstration

30. The Detector Responds To
Fast Motion
Convex

31. Video from Eye Camera
Motion Weighted Edge Map
Multi-scale Histogram (Medial-Axis, Hough Transform for Circles)
Local Maxima

32. Defining Characteristics
Geometric
Isolated
Distal
Localized
Convex
Cultural/Design
Far from natural grasp location
Long distance relative to hand size

33. Other Task Relevant Features?

34. Detecting the Tip

35. Include Scale and Convexity