1. RSS 2011 Workshop on RGB-D Cameras
A Constraint-Based Method for 3-DOF Haptic Rendering of Arbitrary Point Cloud Data
Adam Leeper Sonny Chan Kenneth Salisbury 1

2. Overview
Motivation
Part One: Haptic Algorithm
Part Two: Real-Time Strategies
Results

3. Force feedback for teleoperation is costly to measure.
Motivation
Force feedback for teleoperation is costly to measure.
We can use a model to estimate interaction forces.
Remote sensors produce 3D points. 3

4. Potential Fields and Penalty Methods
Haptic force is computed from current HIP position.
Force is proportional to penetration depth.
Geometric Shapes
Infinite Wall
F = -k*x x

5. Potential Fields and Penalty Methods
Haptic force is computed from current HIP position.
Force is proportional to penetration depth.
“pop-through” when objects are thin

6. Constraint-Based Methods
A proxy / god-object is constrained to the surface.
A virtual spring connects proxy to HIP.

7. El-Far, Georganas, El Saddick. 2008.
Some Previous Methods
Cha, Eid, Saddik. EuroHaptics 2008.
Depth-image tessellation.
Proxy mesh algorithm.
El-Far, Georganas, El Saddick
Per-point AABB collision detection.
Proxy constrained to discrete point locations.

8. Constraint-Based Methods
Constraint method works well for implicit functions
Salisbury & Tarr, 1997
f < 0
f > 0

9. Constraint-Based Methods
A constraint-plane is given by the surface point and normal.

10. From Points to an Implicit Surface
Great. So how do we make an implicit surface from points?
First, we’ll give each point a compact weighting function.
Then we have two options:
Metaballs: constructive geometry
Surfels: surface estimation

11. From Points to an Implicit Surface
Metaballs
Each point produces a 3D scalar field f(x,y,z).
The net scalar field is simply the sum of all points.
A threshold value, T, is chosen to define an isosurface on this field.
T = 0.6
T = 0.2

12. From Points to an Implicit Surface
Metaballs
Each point produces a 3D scalar field f (x,y,z).
The net scalar field is simply the sum of all points.
A threshold value, T, is chosen to define an isosurface on this field.
Don’t need point normals!
T = 0.2
T = 0.6

13. From Points to an Implicit Surface
Metaballs
Each point produces a 3D scalar field f (x,y,z).
The net scalar field is simply the sum of all points.
A threshold value, T, is chosen to define an isosurface on this field.

14. From Points to an Implicit Surface
Surfels
Local surface estimation (Adamson and Alexa 2003)
weighted-average point position
weighted-average point normal

15. Auto-generated parameters adapt easily to any input cloud!
Selecting Parameters
Auto-generated parameters adapt easily to any input cloud!
For each point:
Compute the average distance, d, to the nearest N=3 neighbors.
Set R to some multiple m of d. Generally m ~= 2.
For metaball rendering, T = 0.5 – 0.8 works for most data.

16. Part Two: Real-Time Strategies
Fast Collision Detection
Spatial Issues
Temporal Issues

17. Fast Collision Detection
Haptic servo loop is typically 1kHz, can’t use all points!
Points have only compact support of radius R.
A kd-tree or octree provides fast radius and kNN searches.

18. Use a voxel-grid filter to down-sample cloud.
Spatial Issues
640x480 depth image = 300,000 points.
Some are outside the workspace of the haptic device.
Sensor quantization noise should be filtered.
Our kinesthetic sense just isn’t that good.
(Most) haptic devices just aren’t that good.
Use a voxel-grid filter to down-sample cloud.

19. Temporal Issues
Cloud pre-processing must not interfere with servo loop.
Sensor noise feels like vibration, especially at edges.
New Cloud
Processing Cloud
Update Thread
Servo Thread
. . .
Cloud 0
Cloud 1
Cloud N
Discarded
We used N = 4.

20. Bonus: Multiple Point Sources
This algorithm inherently handles multiple sensor clouds.
The union of nearby points in each cloud is used for rendering.
New Cloud
Processing Cloud
Update Thread
New Cloud
Servo Thread
. . .
Cloud 0
Cloud 1
Cloud N
Discarded

21. Real-Time Performance:
Results
Metaballs:
Only option for sparse or non-planar regions.
Feels more wavy/knobbly in high noise regions.
Surfels:
Better spatial noise reduction for planar regions.
Requires normal estimation.
Real-Time Performance:
Kinect updates at about 10Hz.
Haptic loop time < 200us.

22. Conclusions
Point clouds can be used to generate an implicit surface suitable for stable haptic rendering with no pop-through.
Remote environments can be explored haptically with real- time updates from a 3D sensor.
This strategy could be used to generate feedback constraint forces for robot teleoperation in a remote environment.

23. ? Acknowledgments Thanks to colleagues Reuben Brewer, Gunter Niemeyer.
National Science Foundation
NSERC of Canada
 Come try the demo this afternoon! ?