1. Shoogle Multimodal Excitatory Interfaces on Mobile Devices John Williamson Roderick Murray-Smith Stephen Hughes Shoogle, v., n., (c. 1724) Also shogle, schogle, shougle 1. v. intr. To sway, move unsteadily, to rock, wobble, swing 2. v. tr. To shake, joggle, to cause to totter or rock, to swing backwards and forwards Dictionary of the Scots Language

2. An Example Scenario Would like to know if any messages have arrived Standard alerts can be intrusive and annoying Visual attention is limited Solution: Simply pick up the device and shake it, hear and feel contents rattling about For example, get a quick overview of the quantity and type of messages

3. Model-based Interaction Excite – listen – feel Reveal contents of a mobile device through shaking Active perception: output in context of input motion User driven feedback Intuitive display of abstract data Exploit human understanding of natural physical processes Easy to extend interaction in natural ways Physically based but not physically limited

4. Features Mobile and enclosed : no moving parts Eyes-free : vibration and audio only Rich feedback : communicate as densely as possible Non-disruptive : do not interrupt Respect privacy : do not reveal private information

5. Concept Shake device to summarise contents Device becomes physical container Holding content balls Model-based audio and haptic display Natural sounding audio Inertial sensing for movement Can be passive or active change in a pocket

6. Sonification Takes advantage of impact perception Dense communication in extremely short bursts Many attributes can be perceived from a single impact. Mass, material, velocity...

7. A Video (1:58)

8. Sensing Hardware Use inertial sensing 3 DoF accelerometers Cheap Small and internal Rich signals Prototypes: iPaq 5550 / MESH inertial sensor pack Could also run on std. phones Bluetooth SHAKE sensor

9. Physics: Springs, Friction and Collisions Physics Virtual balls anchored via Hooke-law springs Subject to nonlinear frictional forces Collide with walls, losing some energy and rebounding Simple Euler integration model Measured accelerations incorporated directly

10. Physics: Springs, Friction and Collisions Springs Randomly allocated positions Only term in which ball mass appears Friction Nonlinear stick/slip model Removes lots of annoying low-level activity Collisions Elastic collisions Random direction jitter to remove repetitive bounces No interball collisions Friction Velocity

11. Audio Synthesis Granular approach Impact sounds triggered on wall collisions Samples of real-world impacts Many impacts recorded for each type to maintain variations Wood, metal, glass, water, ice, gravel, ping-pong balls...

12. Vibrotactile Feedback Produced on impact But fixed pattern Presence, not identity Greatly increases solidness of feedback Mass decides frequency division Large balls -> loudspeaker + motor >M max

13. Tap Summary Tap device and balls bounce up Sensed with capacitive sensors Time between impacts linked to relative arrival times

14. Summary Totally eyes-free interface User driven display – works on demand Physically compact Intuitive and compelling to use Come try it if you don't believe me!

15. Summary Applicable in many interaction contexts: Messages, files, memory space remaining, battery life, IM contacts online... Several possible extensions of the interaction style Tapping, sieving More sophisticated linking of content to physics

16. Shoogle Multimodal Excitatory Interfaces on Mobile Devices John Williamson Roderick Murray-Smith Stephen Hughes Shoogle, v., n., Also shogle, schogle, shougle 1. v. intr. To sway, move unsteadily, to rock, wobble, swing 2. v. tr. To shake, joggle, to cause to totter or rock, to swing backwards and forwards Dictionary of the Scots Language