1. Tracking Overview and Mathematics

2. Christoph Krautz 2 Motivation Technologies – Advantages and Disadvantages –Common Problems and Errors –Acoustic Tracking –Mechanical Tracking –Inertial Tracking –Magnetic Tracking –Optical Tracking –Inside-out versus Outside-in Mathematics –Transformations in the 2D-space –Transformations in the 3D-space Discussion Motivation Technologies Tracking Mathematics Content

3. Christoph Krautz 3 What is tracking? The repeated localization of the position and orientation (pose) of one or several real physical objects Why is tracking needed in AR? Integration of virtual objects into real world (images) Motivation Technologies Tracking Mathematics Motivation

4. Christoph Krautz 4 Motivation Technologies – Advantages and Disadvantages –Common Problems and Errors –Acoustic Tracking –Mechanical Tracking –Inertial Tracking –Magnetic Tracking –Optical Tracking –Inside-out versus Outside-in Mathematics –Transformations in the 2D-space –Transformations in the 3D-space Discussion Motivation Technologies Tracking Mathematics Content

5. Christoph Krautz 5 Motivation Technologies Tracking Mathematics Common Problems and Errors High update rate required (usually in real-time systems) Dynamic tracker error, e.g. sensor‘s motion Distortion due to environmental influences, e.g. noise Long-term variations –Cause readings to change from one day to the next day

6. Christoph Krautz 6 Motivation Technologies Tracking Mathematics Acoustic Tracking The Geometry –The intersection of two spheres is a circle. –The intersection of three spheres is two points. One of the two points can easily be eliminated. Ultrasonic –40 [kHz] typical (Slide taken from SIGGRAPH 2001 Course 11 – Slides by Allen, Bishop, Welch) From [1]

7. Christoph Krautz 7 Motivation Technologies Tracking Mathematics Acoustic Tracking - Methods Time of Flight –Measures the time required for a sonic pulse to travel from a transmitter to a receiver. –d [m] = v [m/s] * t [s], v = speed of sound –Absolute range measurement Phase Coherence –Measures phase difference between transmitted and received sound waves –Relative to previous measurement still absolute!! (Slide taken from SIGGRAPH 2001 Course 11 – Slides by Allen, Bishop, Welch)

8. Christoph Krautz 8 Motivation Technologies Tracking Mathematics Acoustic Tracking – Discussion Advantages –Small and lightweight (miniaturization of transmitters and receivers) –Only sensitive to influences by noise in the ultrasonic range Disadvantages –Speed of Sound (~331 [m/s] in air at 0°C) Varies with temperature, pressure and humidity  Slow  Low update rate

9. Christoph Krautz 9 Motivation Technologies Tracking Mathematics Mechanical Tracking Ground-based or Body-based Used primarily for motion capture Provide angle and range measurements –Gears –Bend sensors Elegant addition of force feedback (Slide taken from SIGGRAPH 2001 Course 11 – Slides by Allen, Bishop, Welch) From [1]

10. Christoph Krautz 10 Motivation Technologies Tracking Mathematics Mechanical Tracking – Discussion Advantages –Good accuracy –High update rate –No suffering from environmental linked errors Disadvantages –Small working volume due to mechanical linkage with the reference

11. Christoph Krautz 11 Motivation Technologies Tracking Mathematics Inertial Tracking Inertia –Rigidity in space Newton’s Second Law of Motion –F = ma(linear) –M = I(rotational) Accelerometers and Gyroscopes –Provide derivative measurements

12. Christoph Krautz 12 Motivation Technologies Tracking Mathematics Inertial Tracking - Accelerometers Measure force exerted on a mass since we cannot measure acceleration directly. Proof-mass and damped spring –Displacement proportional to acceleration Potentiometric and Piezoelectric Transducers (Slide taken from SIGGRAPH 2001 Course 11 – Slides by Allen, Bishop, Welch) From [1]

13. Christoph Krautz 13 Motivation Technologies Tracking Mathematics Inertial Tracking - Gyroscopes Conservation of angular momentum Precession –If torque is exerted on a spinning mass, its axis of rotation will precess at right angles to both itself and the axis of the exerted torque

14. Christoph Krautz 14 Motivation Technologies Tracking Mathematics Inertial Tracking - Gyroscopes From [1]

15. Christoph Krautz 15 Motivation Technologies Tracking Mathematics Inertial Tracking - Gyroscopes From [1]

16. Christoph Krautz 16 Motivation Technologies Tracking Mathematics Inertial Tracking - Gyroscopes

17. Christoph Krautz 17 Motivation Technologies Tracking Mathematics Inertial Tracking - Gyroscopes

18. Christoph Krautz 18 Motivation Technologies Tracking Mathematics Inertial Tracking – Discussion Advantages –Lightweight –No physical limits on the working volume Disadvantages –Error accumulation due to integration (numerical) Periodic recalibration –Hybrid systems typical –Drift in the axis of rotation of a gyroscope due to the remaining friction between the axis of the wheel and the bearings

19. Christoph Krautz 19 Motivation Technologies Tracking Mathematics Magnetic Tracking Three mutually-orthogonal coils –Each transmitter coil activated serially Induced current in the receiver coils is measured –Varies with »the distance (cubically) from the transmitter and »their orientation relative to the transmitter (cosine of the angle between the axis and the local magnetic field direction) Three measurements apiece (three receiver coils) Nine-element measurement for 6D pose AC at low frequency DC-pulses (Parts of the slide taken from SIGGRAPH 2001 Course 11 – Slides by Allen, Bishop, Welch)

20. Christoph Krautz 20 Motivation Technologies Tracking Mathematics Magnetic Tracking – Discussion Advantages –Small –Good update rate Disadvantages –Small working volume –Ferromagnetic interference –Eddy currents induced in conducting materials  Distortions  Inaccurate pose estimates –Use of DC transmitters overcomes that problem –Sensitive to electromagnetic noise

21. Christoph Krautz 21 Motivation Technologies Tracking Mathematics Optical Tracking Provides angle measurements –One 2D point defines a ray –Two 2D points define a point for 3D position –Additional points required for orientation Speed of Light –2.998 * 10 8 [m/s] (Slide taken from SIGGRAPH 2001 Course 11 – Slides by Allen, Bishop, Welch) From [1]

22. Christoph Krautz 22 Motivation Technologies Tracking Mathematics Optical Tracking – Active Targets Typical detectors –Lateral Effect PhotoDiodes (LEPDs) –Quad Cells Active targets –LEDs From [1]

23. Christoph Krautz 23 Motivation Technologies Tracking Mathematics Optical Tracking – Passive Targets Typical detectors –Video and CCD cameras Computer vision techniques Passive targets –Reflective materials, high contrast patterns From [1]

24. Christoph Krautz 24 Motivation Technologies Tracking Mathematics Optical Tracking – Passive Targets From [A.R.T. GmbH]

25. Christoph Krautz 25 Motivation Technologies Tracking Mathematics Optical Tracking – Discussion Advantages –Good update rate (due to the speed of light) Well suited for real-time systems Disadvantages –Accuracy tends to worsen with increased distance –Sensitive to optical noise and spurious light Can be minimized by using infrared light –Ambiguity of surface and occlusion

26. Christoph Krautz 26 Motivation Technologies Tracking Mathematics Inside-out versus Outside-in Inside-out From [3]

27. Christoph Krautz 27 Motivation Technologies Tracking Mathematics Inside-out versus Outside-in Outside-in From [3]

28. Christoph Krautz 28 Motivation Technologies – Advantages and Disadvantages –Common Problems and Errors –Acoustic Tracking –Mechanical Tracking –Inertial Tracking –Magnetic Tracking –Optical Tracking –Inside-out versus Outside-in Mathematics –Transformations in the 2D-space –Transformations in the 3D-space Discussion MotivationTechnologies Tracking Mathematics Content

29. Christoph Krautz 29 Representation –x, y, z (position) and , ,  (orientation) –with respect to a given reference coordinate system MotivationTechnologies Tracking Mathematics Position and Orientation (Pose) From [1]

30. Christoph Krautz 30 Translation MotivationTechnologies Tracking Mathematics Transformations in the 2D-space 1 23 1 2 X Y

31. Christoph Krautz 31 Scale MotivationTechnologies Tracking Mathematics Transformations in the 2D-space 1 23 1 2 X Y

32. Christoph Krautz 32 Rotation MotivationTechnologies Tracking Mathematics Transformations in the 2D-space 1 23 1 2 X Y X Y 

33. Christoph Krautz 33 Scale and Rotation can be combined by multiplication of their matrices Translation cannot be combined with them by multiplication Introduction of Homogeneous Coordinates MotivationTechnologies Tracking Mathematics Transformations in the 2D-space From [1]

34. Christoph Krautz 34 MotivationTechnologies Tracking Mathematics Transformations in the 2D-space

35. Christoph Krautz 35 Translation MotivationTechnologies Tracking Mathematics Transformations in the 3D-space

36. Christoph Krautz 36 Scale MotivationTechnologies Tracking Mathematics Transformations in the 3D-space

37. Christoph Krautz 37 Rotation MotivationTechnologies Tracking Mathematics Transformations in the 3D-space

38. Christoph Krautz 38 e.g. Rotation through  about the z axis MotivationTechnologies Tracking Mathematics Transformations in the 3D-space

39. Christoph Krautz 39 Rotation-Sequences –Concatenation of several rotations –Can be performed by using Rotation matrices (matrix multiplication) Euler-angles Quaternions MotivationTechnologies Tracking Mathematics Transformations in the 3D-space

40. Christoph Krautz 40 Euler-angles –Three angles ,  and  Each represents a rotation about one of the coordinate axes (X, Y and Z). –Gimbal Lock –Ambiguities R(, 0, 0) = R(0, , ) MotivationTechnologies Tracking Mathematics Transformations in the 3D-space

41. Christoph Krautz 41 Quaternions MotivationTechnologies Tracking Mathematics Transformations in the 3D-space Unit Quaternions A unit quaternion represents a rotation about the axis through the angle

42. Christoph Krautz 42 Multiplication-operator for quaternions: MotivationTechnologies Tracking Mathematics Transformations in the 3D-space The result is a rotation p composed by the rotations q and r.

43. Christoph Krautz 43 MotivationTechnologies Tracking Mathematics Transformations in the 3D-space Advantages of quaternions: –No gimbal lock –Unique representation of a rotation –Interpolation can be properly carried out (spherical interpolation on the 4-sphere; Shoemake, 1985) –Rotation-sequences can be easily performed

44. Christoph Krautz 44 MotivationTechnologies Tracking Mathematics Conclusion Each tracking technology has advantages and disadvantages Multi-Sensor-Fusion for minimizing the measurement errors Transformations in the 3D-space have to be handled with care

45. Christoph Krautz 45 MotivationTechnologies Tracking Mathematics Thank you for your attention! Any questions?

46. Christoph Krautz 46 MotivationTechnologies Tracking Mathematics References: [1] G. Bishop, G. Welch and B. D. Allen, „Tracking: Beyond 15 Minutes of Thought”, SIGGRAPH 2001 Course Notes, University of North Carolina at Chapel Hill [2] G. Bishop, G. Welch and B. D. Allen, „Tracking: Beyond 15 Minutes of Thought”, SIGGRAPH 2001 Course Slides, University of North Carolina at Chapel Hill [3] Ribo, Miguel, “State of the Art Report on Optical Tracking”, 2001