1. SM1205 Interactivity Topic 10: Motion Tracking Part II Spring 2010SCM-CityU1

2. Motion Tracking Library In cityu.scm package (developed by Hongbo Fu) – Webcam class An easy-to-use wrapper for webcam access Spring 2010SCM-CityU2 import cityu.scm.Webcam; // import Webcam class // create a webcam with size 400 x 300 var webcam: Webcam = new Webcam (400, 300); // webcam will dispatch READY event when the webcam is ready to use webcam.addEventListener( Webcam.READY, onCameraReady); function onCameraReady(e:Event): void { addChild(webcam); } import cityu.scm.Webcam; // import Webcam class // create a webcam with size 400 x 300 var webcam: Webcam = new Webcam (400, 300); // webcam will dispatch READY event when the webcam is ready to use webcam.addEventListener( Webcam.READY, onCameraReady); function onCameraReady(e:Event): void { addChild(webcam); }

3. Motion Tracking Library In cityu.scm package (developed by Hongbo Fu) – MotionTracker class Spring 2010SCM-CityU3 var tracker:MotionTracker=new MotionTracker(webcam);

4. MotionTracker Class Properties – trackX: Number; trackY: Number Center of tracked region With respect to top-left corner of webcam (instead of stage) Methods – hitTestObject – hitTestPoint – hideTrackedArea – hideTrackedCenter Spring 2010SCM-CityU4 Spring 2010 4 (0, 0) (trackX, trackY)

5. Demos Spring 2010SCM-CityU5

6. Clipping Regions By default, motion tracking is done over the entire video domain – Check MotionTracker-Skeleton.fla Spring 2010SCM-CityU6

7. Clipping Regions What if motion only within specific regions (called clipping regions) are needed? – Motion outside those clipping regions will be ignored Demo Spring 2010SCM-CityU7 clipping regions

8. Clipping Regions Specify a clipping region when creating a motion tracker – cl, ct, cr, cb are with respect to top-left corner of webcam – trackX, trackY are also with respect to top-left corner of webcam – You can have only ONE clipping region for one tracker Spring 2010SCM-CityU8 tracker = new MotionTracker(webcam, cl, ct, cr, cb ); tracker = new MotionTracker(webcam, cl, ct, cr, cb ); cl cr ct cb Clipping Region

9. Multiple Clipping Regions To have multiple clipping regions, you can create multiple trackers – All trackers share the same Webcam instance – Each tracker has its own associated clipping region Spring 2010SCM-CityU9 // initialize 1st motion tracker tracker = new MotionTracker(webcam, 0, 0, 200, 200); // initialize 2nd motion tracker tracker2 = new MotionTracker(webcam, 300, 300, 500, 500); // initialize 1st motion tracker tracker = new MotionTracker(webcam, 0, 0, 200, 200); // initialize 2nd motion tracker tracker2 = new MotionTracker(webcam, 300, 300, 500, 500);

10. Class Exercise Extend the previous example to have two clipping regions as follows – Hint: create two trackers with their own clipping regions – How trackers work at the overlapped areas of clipping regions? Spring 2010SCM-CityU10

11. Example: Ping-Pong Game Spring 2010SCM-CityU11 box barL ball barR box2

12. Example: Ping-Pong Game Use tracker and tracker2 to control barL and barR, respectively Spring 2010SCM-CityU12

13. Class Exercise Make ball bounce against stage borders – Change ball’s x and y properties continuously in onTimer event listener – Check if ball hits stage borders Spring 2010SCM-CityU13

14. Hit Test between Ball and Bars Let’s just use hitTestObject method Spring 2010SCM-CityU14 if (ball.hitTestObject(barR)) { trace("barR got hit"); } if (ball.hitTestObject(barL)) { trace("barL got hit"); } if (ball.hitTestObject(barR)) { trace("barR got hit"); } if (ball.hitTestObject(barL)) { trace("barL got hit"); }

15. Hit Test between Ball and Bars Collision response Spring 2010SCM-CityU15 Case 1: Ball is on left side of barR Case 2: Ball is on right side of barR

16. Hit Test between Ball and Bars But how to determine which case to handle? Solution: use signs of ball’s speedX – speedX > 0  Case 1 Response: put ball exactly at the left side of barR – speedX < 0  Case 2 Response: put ball exactly at the right side of barR Spring 2010SCM-CityU16

17. Class Exercise Add sound effects when ball hits stage borders/bars – You can use Flash built-in sounds Hints – Export sounds for ActionScript Spring 2010SCM-CityU17 var ws:WoodSound = new WoodSound(); ws.play();

18. Moving Speed Use MotionTracker’s speedX and speedY to get the moving speed of tracked object speedX (in pixels) – Positive value if object moves along positive x-axis – Negative value if objects moves along negative x-axis speedY (in pixels) – Positive value if object moves along positive y-axis – Negative value if objects moves along negative y-axis Spring 2010SCM-CityU18 trace(tracker.speedX, tracker.speedY);

19. Example Transfer moving speed of tracked object to ball – E.g., hand Demo Spring 2010SCM-CityU19

20. Example Let’s modify the previous ping-pong example – Keep one tracker only, without any clipping region – No vertical bars – Add friction Spring 2010SCM-CityU20

21. Example Change ball’s speeds only if there is some intersection between ball and moving object Spring 2010SCM-CityU21 if (tracker.isTracked() && tracker.hitTestPoint(ball.x, ball.y, true)) { speedX += tracker.speedX; speedY += tracker.speedY; } if (tracker.isTracked() && tracker.hitTestPoint(ball.x, ball.y, true)) { speedX += tracker.speedX; speedY += tracker.speedY; }

22. Example Interact with more balls (demo) Spring 2010SCM-CityU22

23. Assignment 3 Topic: Webcam Interaction with Motion Requirements – Use motion tracker – Interact with motion – Have music/sound elements – Meaningful system – Documentation (in Word format) Title, motivation, how-to-use, screenshots, references Spring 2010SCM-CityU23

24. Assignment 3 Submission – Via ACS Max total file size: 50M.fla,.swf,.doc (documentation) files and other files – Deadline: 23:59:59, 12 May (Wed), 2010 No late submission is allowed In-class presentation, 12 May, 2010 – Each student has roughly 5 minutes for presentation Spring 2010SCM-CityU24

25. Assignment 3 Assessment – Based on Implementation difficulty + UI design + creativity (80%) Presentation performance (10%) Documentation (10%) – Note: if you use any technique which has not been taught in this course, you must state its source or reference in the documentation clearly Otherwise, some marks will be deducted Spring 2010SCM-CityU25

26. Summary of MotionTracker Make sure your Flash application is under the same folder as the tracking library Spring 2010SCM-CityU26

27. Summary of MotionTracker Without clipping region With clipping region – cl, ct, cr, cb are with respect to top-left corner of webcam Spring 2010SCM-CityU27 tracker = new MotionTracker(webcam); tracker = new MotionTracker(webcam, cl, ct, cr, cb ); tracker = new MotionTracker(webcam, cl, ct, cr, cb );

28. Summary of MotionTracker Properties – trackX, trackY: center of tracked region With respect to top-left corner of webcam – speedX, speedY Moving speed of tracked object Spring 2010SCM-CityU 28 Spring 2010 28 (0, 0) (trackX, trackY)

29. Summary of MotionTracker Methods – hitTestObject To test if bounding boxes of display object and tracked region intersect with each other – hitTestPoint To test if a given point is within the tracked region – isTracked To check if any moving object is tracked – hideTrackedArea – hideTrackedCenter Spring 2010SCM-CityU29