1. Hybrid Manipulation: Force-Vision CMPUT 610 Martin Jagersand

2. Today: How to incorporate other sensory feedback. Focus on tasks where the number of contact constraints increase during manipulations. Acquire necessary constraint geometry on- line.

3. Preliminaries: Visual Servoing Applications

4. Limitations of Visual Control Accuracy is limited by: Visual tracking and Visual goal specification (Jagersand et. al. ICRA 97) Specifying well defined visual encodings can be difficult (Hager, Hespana, Dodds, Jagersand) Limited to non-occluded settings Not all tasks lend themselves to visual specification.

5. Observation: Force constraints natural in tasks Example: Inserting a box Sequence of motions Increasing number of contact constraints

6. Preliminaries: Uncalibrated Visual Servoing Let y = visual observation; x = motor control Linear system model: y= Linear p-controller Estimate the Visual-Motor Jacobian

7. What do we need to add to do things like this?

8. Learning Constraint Geometry Impact force along surface normal: Sliding motion: 3 rd vector:

9. Constraint Frame With force frame = tool frame we get: Assume frictionless => Can update each time step P1P1 P2P2 P3P3

10. Hybrid Control Law Let Q Joint -> Tool Jacobian Let S be a switching matrix, e.g. diag([0,1,1]) Velocity control u: Visual partForce part

11. Accounting for Friction Friction force is along motion direction! Subtract out to recover surface normal:

12. Final Hybrid Controller Structure

13. Motion Sequence

14. Switching of Constraints in Hybrid Force-Vision Feedback

15. Sliding along a Curved Surface In principle any smooth surface can be followed Detail of measured force Estimated surface normals

16. Estimation Accuracy The constraint coordinate frame was acquired within a few degrees of the true surface frame

17. Conclusions Hybrid Force-Vision control allow more accurate positioning than vision or force alone Many tasks naturally involve an increasing number of contact constraints We showed that the contact geometry can be successfully estimated Combined Uncalibrated Vision & Force control