Just Say Britney Spears: Multi-Modal Search in On Device Portals Gail Rahn Frederick Medio Systems, Inc. January 2009 M3DD 2009

Agenda Introduction On Device Portals (ODPs) Multimodal ODP Medio Universal Search ODP Solving Development Challenges Live Demonstration –Medio Universal Search ODP on J2ME Featurephone 2 Copyright 2009 Medio Systems, Inc.

Introduction Gail Rahn Frederick or Mobile Software Architect at Medio Systems, Inc. –Mobile Search, Advertising and Analytics Universal Handset Support –J2ME, BREW, Windows Mobile, Android, iPhone, RIM, etc… –WAP, Mobile AJAX, HTML4 on Mobile Devices –1600+ handset models –Live on 9 mobile operators in 6 countries t-zones J2ME ODP for T-Mobile USA –22% Market Share in Mobile Search Microsoft has 6% share –Site Search Provider for Leading Mobile Publishers Copyright 2009 Medio Systems, Inc.

On Device Portals Handset-resident mobile application –Preloaded or free download Enhances discovery and consumption of endorsed mobile content and services Search, browse and purchase Provides a compelling user experience –Overcomes / circumvents browser flaws –Targets mass-market featurephones Personalized experience Sometimes monetized with advertising Copyright 2009 Medio Systems, Inc.

Multimodal ODP Many Ways to Use an ODP –Search, Browse, Discover, Purchase Many Search Modes in ODP –Text, Voice, Bar Code, Image Navigate an ODP using familiar actions on a mobile device –saying a phrase –entering text –snapping a photo Copyright 2009 Medio Systems, Inc.

Medio Universal Search ODP Consistent UI/UX across All Mobile Platforms –Native App, WAP and Mobile Web Multi-modal, search-driven J2ME ODP –Drives repeat customer usage –Increases usage of browsable and consumable mobile content –Consistent UI/UX despite J2ME fragmentation –Search by text, voice, image and barcode –Also available on other mobile platforms and mobile web Integrated Advertising –Leaderboards –Interstitials –Sponsored Links Copyright 2009 Medio Systems, Inc.

Solving Challenges with Voice Search Sound Capture J2ME Platforms require JSR-135 with sound record. Requires AMR or other low-bitrate codec. Environment Mobile environments are typically noisy: utterances come from cars, bars, airports... rarely from quiet home or office environments. Complex Grammars Must be efficient at handling unstructured/natural utterances –Past: Walled-Garden ODP grammars includes song artists and titles, wallpaper titles, game titles and studios, etc. –Present and Future: Web-Enabled ODPs support millions of music tracks, mobile site names, URLs, free-text queries – requires flexible voice inputs –Consider grammar-less voice to text translation using phonemes.phonemes Copyright 2009 Medio Systems, Inc.

Solving ODP Development Challenges Operator Integration –Handoffs to operator services to preview and purchase content HTTP 302 redirects, browser launches, J2ME app launches Network Efficiency –Compact binary protocol (See M3DD 2008 Poster by Medio) –Piggyback status updates with search requests –Asynchronous network communication –Dynamic UI updates to refresh links, pages, chrome and advertisements J2ME Implementation Bugs –Share MIDlet with failed test case with operator and OEM Prove KVM bug with simple, commented code Three-way communication (ODP vendor, operator, OEM) Copyright 2009 Medio Systems, Inc.

Live Demonstration Demo Devices Nokia 6263 –T-Mobile USA 3G –Series 40 5 th Edition –MIDP 2.1, CLDC 1.1 –JSR 135 for Voice Recording –Released in Dec 2007 –Price: FREEFREE Blackberry 8120 (Pearl)Blackberry 8120 –See me for demo Britney Spears Ringtones Barack Obama Inauguration Nordstrom in Santa Clara, California Facebook Kate Winslet and Leonardo DiCaprio Santa Clara Weather Sample queries Copyright 2009 Medio Systems, Inc.

M3DD 2009 Just Say Britney Spears: Multi-Modal Search in On Device Portals Gail Rahn Frederick Medio Systems, Inc. January 2009