1. Accelerometer based motion gestures for mobile devices Presented by – Neel Parikh Advisor Committee members Dr. Chris Pollett Dr. Robert Chun Dr. Mark Stamp

2. AGENDA: Android Platform Development Tools Sensor Simulator Design Methodology Shake feature Image rotation feature Zoom feature Scroll feature Challenges involved Conclusion

3. Accelerometer based motion gestures for mobile devices Project Summary: Goal of Project - Develop Accelerometer based motion feature like shake, scroll, zoom, image rotation to extend Webkit browser interface supported by the Android platform. Implementation and development - Google’s Android SDK with Sensor Simulator for simulating accelerometer inputs and movements. Programming and Framework – Java programming with Eclipse plug in for Android

4. Android Architecture sfsf OS Layer Linux Kernel Middleware System libraries. Android runtime. Dalvik Virtual Machine  executed files in ‘.dex format’  Faster and efficient as compared to traditional JVM.  Register based. Application framework Applications Accelerometer based features like scroll, zoom, shake, image rotation.

5. Android Development tools (ADT) ADT - A plug in for Eclipse framework.  Supports debugging and monitoring from within eclipse.  Efficient means to create, test and debug applications. Device Emulator  emulates software and hardware functioning of the android platform. DDMS (Dalvik Debug and Monitoring Service)  Logcat – helps in checking flow of control and recording log messages. MESSAGE FILTER LOGCAT

6. Android Development tools (ADB) ADB – Android Debug Bridge  Command based.  Lists currently running Virtual emulators  ID – unique identifier  Serial no – string identifier  State – state of emulator Installing an application CLI showing installation procedure

7. Sensor Simulator Open source simulator by OpenIntents  Used to simulate accelerometer based movements to extend the webkit browser interface for features like scroll, shake, zoom, image rotation.  Limitation: required manual changes to [x,y,z] settings to simulate movement and change accelerometer values. Automation -  Continuous set of inputs from simulator to emulator.

8. Simulation to test Automated inputs Scenario – record changes in accelerometer values by simulating ‘throwing a phone’ Initially [x,y,z] = [0,0,-1] During fall [x,y,z] changes rapidly and alert message generated When phone strikes ground another alert message generated. [x,y,z] comes to rest. Log file records cahnges in [x,y,z]

9. Design Methodology Accelerometer based features like scroll, zoom, image rotation, shake employ this design strategy.  ‘Test.java’ – launches main application activity  ‘AccelerometerReader.java’ – manages sensors Flow for Test.java  onCreate – manages initialization and instantiation  Run – manages communication for Sensors and handlers  Feature implementation and display on content view. Activity starts In Test.java onCreate() { - Initialization - A Thread instantiated - set the content view - Thread Started } run() { - Connection to sensor simulator - Instantiate Accelerometer Reader object - capture values read by Accelerometer Reader - call the handler } Back in Test.java - Message handled by the handler - feature functionality executed, - Current view invalidated - New view created and drawn - new view set to the content view in onCreate()

10. Design Methodology Flow for AccelerometerReader.java Accelerometer reader constructor – manages initialization of sensors Handles reading of sensor values and communication with the thread of main activity. Returns [x,y,z] back to the main activity. Activity starts In AccerometerReader.java AccelerometerReader { - Initialization of sensors - get all supported sensors - Check for accelerometer - enable the accelerometer - read the current accelerometer value [x,y,z] - return [x,y,z]to the main run() of Test.java } Back in Test.java - the [x,yz] are read continuously - Proceed with the handler call

11. Shake Feature The test scenario : the user enters some text as input. On shaking the Sensor Simulator the Text Box entry would be cleared. It eliminates the need for dedicated function keys like ‘clear’ and ‘back’ on the phone. Log file showing the shake detection

12. Rotation feature Principle :  if the orientation of the phone is changed by a certain degrees, then the orientation of the application running on the phone must also get oriented and adjust itself accordingly.  Fig 1: Initial position when application launched  Fig 2: Image rotated by 90 degrees clockwise  Fig 3: Image rotated by 90 degrees anti - clockwise  Log files detecting the rotation

13. Zoom feature Extend the Webkit browser interface to support the zooming of Web page Tilt in forward direction (towards) corresponds to zoom out Tilt in backward direction (away from user) corresponds to zoom in

14. Scroll feature Extend the Webkit browser interface to support the scrolling of Web page Support scrolling in up, down, left, right directions Also responds to scrolling at specific angles i.e. top left of screen or bottom right corner of the page.

15. Media capabilities Independent application to test Media API’s of Android Functionality of play, pause, stop and restart for testing an audio file

16. Challenges involved and Learning's Android Platform launched in November 07  Learning Curve, limited resources & reference material. Exhaustive event handling & test applications  Journey from ‘Hello World’ to ‘Accelerometer based features’ Automation functionality in Sensor Simulator  Skillfully modify and enhance the code.  Time consuming Compatibility  New releases of SDK, JDK, Eclipse. Patience, persistence and dedication

17. Conclusion Implementation of Accelerometer based features like Scroll, Zoom, Shake, etc to extend the Webkit browser interface. Reduce need for dedicated function keys. Simplicity of platform, flexibility of application and development.

18. Future Work Explore capabilities of sensors like Compass, Orientation etc. Develop games based on motion of Accelerometer. In future, Android could power variety of electronic and embedded systems.

19. Acknowledgements Dr Pollett and Committee members Q/A