1. Electronic Conducting System Kenzo Abrahams Supervisor: Mehrdad Ghaziasgar Co-supervisor: James Connon Mentored by: Diego Mushfieldt

2. Overview Introduction User Interface Specification High Level Design Low level Design Demo

3. Introduction Interactive Conducting System Tracks hands using webcam Real-time alterations depending on hand gestures ◦ Change volume ◦ Change tempo

4. User Interface Specification Graphic User Interface (GUI) User interacts using webcam and mouse

5. User Interface Specification

8. High Level Design The solution can be broken up into 3 parts ◦ Input ◦ Image processing ◦ Adjust music

9. Low Level Design

10. Input ◦ The frames need to be acquired from the webcam ◦ cvQueryFrame(capture)

11. Low Level Design Image Processing ◦ Convert copies of frames to HSV colour space ◦ cvCvtColor(frame, img_hsv, CV_BGR2HSV)

12. Low Level Design Image Processing ◦ Skin segmentation ◦ Use of a predefined method ◦ Detect if pixels in the rectangle fall within a certain range

13. Low Level Design Image Processing ◦ Set hand as region of interest ◦ cvSetImageROI(pHueImg, pHandRect )

14. Low Level Design Image Processing ◦ Perform Camshift cvCamShift( pProbImg, prevHandRect, cvTermCriteria( CV_TERMCRIT_EPS | CV_TERMCRIT_ITER, 10, 1 ), &components, &HandBox )

15. Low Level Design Adjust Music ◦ Library RTcmix is used to produce the music ◦ Can be imbedded into C++ code ◦ load("WAVETABLE") ◦ Wave = maketable("wave", 1000, "tri") ◦ WAVETABLE(start time, duration, amp, frequency, pan, wave)

16. Demo Overview of demonstration ◦ Track the right hand ◦ Perform the gestures that are going to be used ◦ Track the left hand ◦ Perform simple gestures such as raising and lowering hand ◦ Move each hand separately

17. Project Plan GoalDue Date Learn how to use OpenCV and its tools. Elicit the requirements and define a designers interpretation of the problem. Completed From the users requirements design a prototype for the system Completed Construct the system with all its functionality present Term 3 Test the system and deployment Term 4

18. References O'Niel, D L. (2008). Music Theater Jobs. Available: http://www.ehow.com/list_6547349_music-theater-jobs.html. Last accessed 29th March 2012. Bradski, G Kaehler, A. (2008). Getting to Know OpenCV. In: Loukides, M Learning OpenCV. United States of America: O’Reilly. pp31-87. Ivanciuc, O. (2005). SVM - Support Vector Machines. Available: http://www.support-vector-machines.org/. Last accessed 29th March Nakra, T., Ivanov, Y., Smaragdis, P., Ault, C. (2009). The USB Virtual Maestro: an Interactive Conducting System, pp.250- 255, NIME2009 Borchers, J., Hadjakos, A., M¨uhlh¨auser, M. (2006), MICON a music stand for interactive conducting. Proceedings of the 2006 conference on New interfaces for musical expression, pp254–259.

19. Questions and Answers