1. Project #12: Feature Extraction from Multimedia Databases SURF-IT By:Isaac Mahgrefteh Mentor: Prof. Sharad Mehrotra

2. Introduction The rapid deployment of cameras and a variety of other sensors in smart buildings opens the doors to a new age of application programming. Fraud detection, crime prevention, power utilization, occupancy monitoring, and social network analysis are just a small fraction of such application. At UCI, as part of "Responsphere", many buildings on campus are fitted with variety of sensors including video, audio, RFID, people- counters and environmental sensors. My grand goal: To automatically detect key identities in a in multimedia database created from the Responsphere video cameras.

3. How Will This Extracted Information be Used? A fundamental challenge is being able to search and visualize the information collected. Once the algorithms are in place we will use them to automatically, annotate, structure and present meaningful information about the data collected. This will be used as part of StreamViewer - A tool for the visualization and synchronization of multimedia streams.

4. First Step at Addressing the Problem In order to process a multimedia database, the first step is to extract meaningful information from it: Is there an object in the foreground? Which direction is the object traveling? Attributes specific to that object such as: height, size, color.

5. TOOLKIT OPENCV: a C library developed by Intel which contains over 500 functions for computer vision. Eclipse: Integrated Design Environment. Small data set of images of people walking in Bren Hall

6. A Sample Image

7. How To Detect If There Is an Object In The Frame? For static cameras the process is relatively simple. 1) Feed images from the same camera to the program, so that the computer can learn a background model. 2) Once we have accumulated enough images, we can subtract the current frame from the background model created. 3) If the current frame contains information that is significantly different from the background - trigger: "motion detected".

8. The math. Looking Deeper Into Image Subtraction. S(x,y) = sum of the pixel values in a location N=number of frames collected M(x,y)= S(x,y) /N Sq(x,y)= (S(x,y))^2 Standard deviation α(x,y)= sqrt((Sq(x,y)/N)-( S(x,y)/N)2) After that the pixel in a certain pixel location in certain frame is regarded as belonging to a moving object if the following condition is met

9. The math. Looking Deeper Into Image Subtraction. Abs(m(x,y)-p(x,y))> C α(x,y) why should C = 3? due to the “three sigmas” rule. That states states that for a normal distribution, almost all values lie within 3 standard deviations of the mean.

10. The math. Looking Deeper Into Image Subtraction.

11. The Future And Beyond I am really enjoying the Feature Extraction project, and I hope to continue with this project throughout the upcoming school year. The features that I am planning to add through the summer, and continue through the year, are the following: Detecting if a person has backpack. Which way is the person walking. Color of clothing. Hair color.

12. If You Like This then you will Love These opencv reference manual: http://worldlibrary.net/eBooks/Give- Away/Technical_eBooks/OpenCVReferenceManual.pdf Learning Opencv Vision With The Opencv Library, which is a free. ebook for ANTPAC users :http://search.safaribooksonline.com/9780596516130 For more infomation on Stream Viewer and Responsphere please vist Ronen Vaisenberg website www.ics.uci.edu/~ronen/

13. Thank You For Everything!!! I would like to thank SURF-IT and everyone for talking time out of their day to learn about feature extraction. I would also like to thank my mentor Proffsor Sharad Mehrotra. for this amazing opportunity. Ronen Vaisenberg, who got me interested in the project, and the idea of higher education. Jay Lickfett, helps me with debugging, and anything i can think of relating to computers. SATViewer team, I have learned more about computer science in a couple of weeks then I could have imagin