Virtual Me

Motion Capture (mocap) Motion capture is the process of simulating actual movement in a computer generated environment The capture subject (such as a person) would wear a special suit or materials that send their location data to a computer. The data can be used to study motion or to give an illusion of life to 3D computer models

Human Body Parts Tracking Eye Tracking Finger Tracking Head Tracking Hand Tracking Arm Tracking Leg Tracking

1- Mechanical Motion Capture – Using an exoskeleton – Each joint is connected to an angular encoder – The value of movement of each encoder is recorded by a computer – Pros: Real time No occlusion Large capture range No magnetic or electrical interference – Cons: Freedom of movement is limited Fixed configuration of sensors Breakable The joints are only simple hinge joints which leads to Restricting the capture subject’s movement

2- Magnetic Motion Capture tracking sensors are placed on the capture subject to measure spatial relationship to a magnetic transmitter The tracking sensors output their translations and orientations – Pros: No post-processing is required to compute rotations Can be used for real-time applications Tracking sensors are not occluded by non-metallic objects – Cons: Wiring and batteries limiting capture subject’s movement Capture volume is normally smaller than optical systems Tracking sensors are prone to magnetic and electrical interference

3- Optical Motion Capture Capture subjects wear markers (reflective or emitting) Cameras in a reflective marker system are equipped with LEDs which their lights are reflected by markers Markers in an emitting marker system are LEDs illuminated one at a time or all at once and identifying markers with frequency amplitude modulation – Pros: Free movement Capture volume can be larger than other systems – Cons: Extensive post-processing is required Markers can be occluded by capture subjects or props, causing loss of data Rotational data needs to be computed from positional data in the post- processing Lighting needs to be controlled, especially for passive systems

4- Marker less Tracking Using computer vision Detect Humans in the picture Uses techniques to extract motion information from the image – Silhouette detection – Joint detection

Market Survey

PhaseSpace 4:24 Impulse cams 20:100 impulse active LED Price:$29, to $123, feature: full body motion tracing (including figures using gloves)???

Nintendo Wii Accelerometer Optical sensors Price:$199 Limitations: – Can only track the motion of the hand

Sony PS3 Eye Web Cam price: cam:$34.99, playstation 3:$ feature: recognizes hand and body movements Limitations: – Officially Works only with Sony PS3 – Low resolution 640x480

The Virtual Me Approach A frame work for Human motion capture Color based markers Arbitrary number of cams Detect major limbs motion and orientation Hopefully real time

Sample application A simple game will be used as a demonstration Virtual Goal The player will defend his goal from multiple footballs

System Design

Image Processing Pipelines IPP(1)IPP(2)IPP(K) Marker Detector Merging Module Constraint module History Body Constructor Camera Interface

Time Plan

Time Plan 2

Development Language – C++ Tools – Visual Studio 2008 – Open CV 2.0 – Matlab (rapid proto typing)

Questions