1. Sensing Meets Mobile Social Networks: The Design, Implementation and Evaluation of the CenceMe Application Emiliano Miluzzo†, Nicholas D. Lane†, Kristóf Fodor†, Ronald Peterson†, Hong Lu†, Mirco Musolesi†, Shane B. Eisenman§, Xiao Zheng†, Andrew T. Campbell† †Computer Science, Dartmouth College §Electrical Engineering, Columbia University

2. Outline Introduction of CenceMe Introduction of CenceMe Design Consideration Design Consideration CenceMe Implementation CenceMe Implementation CenceMe Classifier CenceMe Classifier System Performance System Performance User Study User Study Conclusion Conclusion Pros&Cons Pros&Cons

3. Motivation Text messaging: “Where are you?” “What are you doing?” Text messaging: “Where are you?” “What are you doing?” Sensors in mobile phone: GPS, accelerometers, microphone, camera … etc Sensors in mobile phone: GPS, accelerometers, microphone, camera … etc Data collection through sensors Data collection through sensors

4. Introduction of CenceMe People-centric sensing application People-centric sensing application Implementation on Nokia N95; Symbian/JME VM platform Implementation on Nokia N95; Symbian/JME VM platform Share user presence information (Facebook) Share user presence information (Facebook)

5. Contributions Design, implementation and evaluation Design, implementation and evaluation Lightweight classifier Lightweight classifier Trade-off: time fidelity v.s. latency Trade-off: time fidelity v.s. latency Complete User study Complete User study

6. Mobile Phone limitations OS Limitations OS Limitations API and Operational Limitations API and Operational Limitations Security Limitations Security Limitations Energy Management Limitations Energy Management Limitations

7. Architecture Design Issues Split-Level Classification (primitives, facts) Split-Level Classification (primitives, facts) –Customized tag –Resiliency –Minimize bandwidth usage/energy –Privacy/data integrity Power Aware Duty-Cycle Power Aware Duty-Cycle

8. CenceMe Implementation Operations (Phone): Sensing Sensing Classification to produce primitives Classification to produce primitives Presentation of people's presence on the phone Presentation of people's presence on the phone Upload of primitives to backend servers Upload of primitives to backend servers Classifications (Backend Server): classifying the nature of the sound collected from the microphone classifying the nature of the sound collected from the microphone classifying the accelerometer data to determine activity (sitting, standing, walking, running) classifying the accelerometer data to determine activity (sitting, standing, walking, running) scanned Bluetooth/MAC addresses in range scanned Bluetooth/MAC addresses in range GPS readings GPS readings random photos random photos

9. Phone Software

10. ClickStatus

11. Backend Software

12. Phone classifiers (1/2) Audio Audio –Feature extraction –Classification

14. Phone classifiers (2/2) Activity Activity

15. Backend Classifier Conversation Conversation Social Context Social Context –Neighborhood conditions –Social Status Mobility Mode Detector Mobility Mode Detector Location Location “Am I Hot” “Am I Hot” –Nerdy, party animal, cultured, healthy, greeny

16. System Performance Classifier accuracy Classifier accuracy Impact of mobile phone placement on body Impact of mobile phone placement on body –8 users –Annotations as ground truth for comparison with classifier outputs Environmental conditions Environmental conditions Sensing duty cycles Sensing duty cycles

17. General Result

18. Phone placement on body Pocket, lanyard, clipped to belt Pocket, lanyard, clipped to belt Insignificant impact conversation v.s. Non-conversation Insignificant impact conversation v.s. Non-conversation

19. Environmental impact Independent of activity classification Independent of activity classification More important: transition between locations More important: transition between locations

20. Duty Cycle (1/2) Problem detecting short term event Problem detecting short term event Experiment: 8 people. Reprogram different duty cycles. Experiment: 8 people. Reprogram different duty cycles.

21. Duty Cycle (2/2) N=5, Conversation classification delay: 1.5 mins N=5, Conversation classification delay: 1.5 mins N=30, Conversation classification delay: 9 mins N=30, Conversation classification delay: 9 mins

22. Power Benchmarks Measuring battery voltage, current, temperature Measuring battery voltage, current, temperature Battery lifetime: 6.22+/- 0.59 hours Battery lifetime: 6.22+/- 0.59 hours

23. Memory and CPU Benchmarks

24. User Study Survey user experience Survey user experience Feedback: Feedback: –Positive from all users –Willing to share detail status and presence information on Facebook –Privacy not an issue –Stimulate curiosity among users –Self-learning on activity patterns and social status

25. User Study A new way to connect people A new way to connect people What is the potential CenceMe demographic? What is the potential CenceMe demographic? Learn about yourself and your friends Learn about yourself and your friends My friends always with me My friends always with me

26. Rooms for improvement Battery life to up to 48 hours Battery life to up to 48 hours Finer grained privacy policy settings. Finer grained privacy policy settings. Shorter classification time Shorter classification time

27. Conclusion A complete design, implementation and evaluation A complete design, implementation and evaluation First application to retrieve and publish sensing presence First application to retrieve and publish sensing presence A complete user study and feedback for future improvement A complete user study and feedback for future improvement

28. Pros&Cons Pros Pros –A complete study –Use off-the-shelf devices Cons Cons –Non-pragmatic –Entertainment oriented –Energy consumption problem