1. MITA 1 MITA : Seminar 12 Applications Daniela Grudinchi, Petri Heinilä, Paula Hiltunen Pekka Jäppinen, Jarno Laitinen, Saku Vaittinen Lappeenranta University of Technology

2. MITA 2 Overview of Presentation Rich call Streaming Browsing Coffee break & free discussion Messaging DRM

3. MITA 3 Mobile environment [5,6] –Network limitations Connectivity Bandwidth Latency Cost of use –Terminal limitations Display size Input methods CPU Compatibility

4. MITA 4 What is rich call Information Text Graphics Images Animations Sounds Multimedia Physical effects Consist of calls combining different media and services into a single call session

5. MITA 5 Mobility in rich call and streaming Rich call and streaming need: –Reliable communication medium with enough bandwidth –Terminals that have needed media presentation and management properties –Modern devices with enough memory for applications and data

6. MITA 6 Rich Call Example Form a standard plain voice call During the call transmit piece of video from last vacation Before ending the call play an interactive game with the mobile device

7. MITA 7 Value to the Consumer Connectivity – Being together Personality – Self-Expression and Seamless Continuous Mobile Awareness Content Richness – Rich Multimodal Mobile Communication Experience

8. MITA 8 Consumer Service Concepts Seamless Interaction –Usage of different services simultaneously Smart Service Routing: the calls can be –Let through –Redirected –Barred Incoming call may have different attributes: –Subject and priority –Media type and characteristics –User’s location –Calendar, time, and presence information

9. MITA 9 Consumer Service Concepts End-to-End Communication:- – Communication will move from ears or eyes to both ears and eyes –“Hear what I say” will be supplemented with “see what I mean”

10. MITA 10 Rich Call context diagram Rich call service Phone book Initiating applications Supporting application Rich call service Presence service Capability service Phone book/ Directory service Rich call service Initiating applications Supporting application Phone book A -subscriberB -subscriberNetwork

11. MITA 11 Rich call in MITA MITA Interaction modes –Messaging –Browsing –Rich Call Rich Call Interaction mode deals with the characteristics related to communication session and end-to-end content transport and processing The rich call interaction mode defines communication capabilities to be provided for applications

12. MITA 12 Rich call in MITA Rich Call applications deliver services to the consumer Rich Call support uses the services of the Mobile Internet layer for communication and local processing

13. MITA 13 Rich call in MITA Layered Element Model Rich Call Support Other and 3 rd party applications Browser Application Email Application Phone Application Streaming Application Instant Messaging Application AApplication Programming Interfaces Browsing Support Messaging Support Application Layer Application Development Interfaces

14. MITA 14 Streaming Oneway communication to transfer media type information from sender (server) to client Sequential, data frames should be ordered (but not have to) Session based, immediate (no storage) Streaming has soft realtime constraints (for human)

15. MITA 15 Communication model of streaming Participants: client(s), sender (server) and router After discovery client asks media (stream) from sender Sender determines QoS parameters for the requested media and configures the routers along the path to the client. => Streaming session initiated Sender pushes data blocks / frames to client. At end of media sender closes the streaming session.

16. MITA 16 Streaming applications Audio –News, Radio,.. Video (usually with audio) –Movies, Live-broadcasts,.. Mass software delivery Requirements –Buffering and caching –Coding: AAC (Advanced Audio Coding) MPEG4

17. MITA 17 Simple streaming Client device gets the stream location URL Client initiates the “session” with HTTP GET Server starts to push media resource over HTTP channel. At end of the resource server closes channel. Client may buffer and cache the stream Client uses codec (eg. MPEG2) to show the stream

18. MITA 18 Simple streaming consequences Point-to-point model –Each packet is individually routed from server to client => huge network level overhead No real-time quarantees Existing infrastructure, HTTP –Extremely large number of existing servers and client HTTP components –Media and data type independence

19. MITA 19 Advanced Streaming Using multicast and router QoS configuration IPv4 or IPv6 multicast – channel group based sending of the stream RTP (Realtime Protocol) – Transport of the media stream, timing, UDP RTSP (Realtime Streaming Protocol) – Control of the media flow SIP (Session Initiation Protocol) – Transferring session initialization

20. MITA 20 Advanced Streaming consequencies Multicast – copy on routing branch semantics, rational network level overhead If used RSVP (Resource reservation protocol) QoS settings can be applied to communication channel Control of timing with RTP Infrastructure is complex and not (yet) supported –Routers does not have or have incompatible QoS configuration capabilities (easier to add bandwith to the network than control capabilities).

21. MITA 21 Streaming in mobile devices For streaming, mobile device represents just another access technology –Wireless bandwith, GPRS (56Kib/s avg), EDGE (116Kib/s avg) – sufficient –Handover resolving – buffering –Disconnected communication - Synchronization capabilities Memory issues – buffering and caching –Need modern mobile devices

22. MITA 22 Browsing Asymmetric system Client-server communication model –Browser (client) connects to the server and requests document –Server replies with the requested document Sessionless communication –Requests are not linked to each other. Markup-based description languages are the backbone of the services

23. MITA 23 Browser Browser consists of two main parts: –Content Access (based on Mobile Internet protocols supporting HTTP requests and URI addressing) –Content Handling (based on plug-in architecture) Markup language describes the content of document Cascading Style Sheets (CSS) describes how browser presents the content

24. MITA 24 Browsing effects to mobile world Mobile devices are designed primarily to support two main interaction modes, calling and messaging Browsing is seen as a complementary near real- time interaction –The browser is an application in a mobile device used to access mobile services in the form of pages The use of mobile device is changing –Accessing data services in addition to the regular calling and messaging services

25. MITA 25 Limitations from mobility [1,7] Small screen size (many shapes available) Typically one hand operation (pointing methods) –harder text input and harder object activation (e.g. keyboard and mouse not common) –Some devices support only vertical scrolling Slower data transfer between the terminal and the server The amount of stored cookie data is limited Expenses (bandwidth rate compared to wire line)

26. MITA 26 Mobile browsers [1] Basically mobile browsing basic structure and behavior do not differ from web browsing –e.g. mobile billing provides mechanisms for receiving, viewing and paying bills Some extensions, features not very useful in web browsing –Location information for e.g. guiding services –Mobile commerce aspect Interfaces for many mobile commerce related services, e.g. mobile wallet

27. MITA 27 Mobile browsers [1,8] There are and will be a wide variety of different kinds of mobile devices –Information about characteristics and user preferences is needed -XHTML MP supports CSS as Wireless CSS -All web sites can be shown in any class of device and screen size -only the style sheet is amended for different needs Different mobile browsers interpret the standards and protocols in different ways

28. MITA 28 Mobility and browsing services Service has to be designed for mobile users Personalisation and content adaption becomes more important The purpose of use is different in web browsing and mobile browsing –More infotainment than entertainment –Services have to be quick to find, easy to use, carrying no extra baggages [8]

29. MITA 29 Possibilities and challenges Location information can provide new possibilities –Local browsing Advertisement on services become problematic –Should user pay for getting advertisements? Distributed content fetch –Some content coming over GPRS service while other come from local proxy over Bluetooth

30. MITA 30 The latest (Nokia) news of mobile browsing Mobile terminals will use Wireless Profiled TCP/IP (wTCP/IP) – Part of OMA WAP 2.0 specification – Includes optimized settings that improve performance over wireless links

31. MITA 31 The latest (Nokia) news of mobile browsing XHTML Mobile Profile (XHTML MP) has replaced WAP’s content language WML –A subset of XHTML –Drops things that are not useful for small screens, e.g. frames –Content can be shown both in web and mobile browsers Phones with WAP 2.0 supporting XHTML MP have been already released

32. MITA 32 Some trends in development [1] Service discovery and service installation more user friendly Optimize the engines of the mobile devices for handling various kinds of XML-based data Java also common in low-end devices Ability to access services offline True device independent environment (Semantic web) Capabilities to use voice based services (VoiceXML)

33. MITA 33 VoiceXML [11] Standardisation is required to enable effective voice- activated retrieval of Internet content Internet applications with –Spoken selection dialogues –Voice commands –Interactive replies Well-suited for two tasks –Delivering Internet content in speech form (e.g. enables access via mobile phones) –Developing new interactive and voice-controlled phone services

34. MITA 34 Voice Browsing [9,11] Voice browsing technology is a rapidly-growing field –Prediction: by the end of 2005 there will be 56 million mobile voice portal users with 250,000 voice sites in North America The evolution of voice browsing –Speech recognition software gives the caller series of options, e.g. booking airline tickets –Voice browsing websites offering voice portals, e.g. search engines over wireless devices –The voice web: entirely voice-based network of sites

35. MITA 35 Voice Browsing [10,12] Multimodality is coming –Web access by more than one channel at a time, e.g. using voice and keypad at the same time –In October this year one demonstration of multimodal web browsing for wireless users was announced –Multimodal user interface enables users to switch between applications in the same session

36. MITA 36 And now something completely different… The Coffee Break!!!

37. MITA 37 Messaging Non real-time, client-server based communication Store-and-forward messaging –SMS, MMS Store-and-retrieve messaging –e-mail, Instant Messaging (IM) Store-and-push –e.g. stock information, kyykkä headlines

38. MITA 38 Instant Messaging Short and direct peer-to-peer text conversations Definition has evolved as user interfaces and networking capabilities have improved –Message can be any kind of data (text, voice, video etc.)

39. MITA 39 Roots of IM Early days of multi-user (UNIX) computers Communication: talk, write Presence: finger Limited user community –university, corporation IRC, MUD first multi-user “chat rooms”

40. MITA 40 Towards Modern IM Graphical user interfaces and Internet have lead to rapid increase in IM in fixed networks AIM, Jabber, ICQ, Yahoo, MSN Messenger etc. Combined communication and presence information File sharing, white boards, meeting features

41. MITA 41 Going Mobile The same features are now in mobile devices Extended reachability: “any time, anywhere” Ubiquitous access –Start conversation in your desktop computer and continue with your mobile terminal Location-based functions and services –Example: Send IM to city's taxi service to pick me up “right here”

42. MITA 42 Presence Consumer's ability and willingness to communicate Dynamic profile of the consumer containing customer information –Availability, mood, intentions, contact preferences etc. Buddy list

43. MITA 43 Presence Current sessions –Phone call, browsing, retrieving streaming content etc. Customer context –In a meeting, on a holiday etc. Preferred communication type –IM, pager, phone etc.

44. MITA 44 Location Location Services are (going to be) integral part of the Mobile Internet New dimension to mobile applications by adding value to services Can be adapted straight to MIM applications –Buddy list showing user's location –Machine-to-person messaging (advertisements, tickets etc.)

45. MITA 45 Applications SMS based solutions –Every message charged as single SMS WAP based solutions –Extends use of first GPRS capable phones Smart phone applications –All-IP –“Always on”

46. MITA 46 Applications Familiar user interface Lots of MIM applications in market already Combine messaging and presence, location support is still tomorrow's technology

47. MITA 47 Future Messaging services will evolve into more versatile and flexible communication means New paradigms, technologies and content types –Location-based messaging and services –UMTS, 4G, WLAN? –Video and gaming services Local presence? Obviously great market potential

48. MITA 48 Introduction to DRM …was done two weeks ago (team Hautamäki & Kangas) next week mobile devices & DRM this time application oriented Digital Rights Management (DRM) technology protects content owner rights when selling and distributing the content online in a digital form. [15]15

49. MITA 49 Players of the value chain of mobile services DRM device personalisation, entertainm. Client (Virtual) Telecom operator Content providers Phone manufacturer Payment Services, Connection, Invoice Marketing artists coders.. Payment Data Content aggeregators, portals Payment Client Superdistribution: - give rights & content - copy content - new rights created -> new payment (Details p. 445-449 [1])1 Local usage rights See [19,20] for more (better charts)1920

50. MITA 50 Nokia’s requirements for a DRM system (2000) [18]18 ”A key component of this vision [MITA] is protection of the: –intellectual property of businesses -the privacy of individuals and –the rights of content providers These are Nokia's general needs of a DRM system.” Efficiency. Makes efficient use of limited resources of mobile device Support for multiple delivery channels (streaming, superdistribution..) Support for a variety of devices Interoperability between various content provider's DRM systems Ease of use. Cost effectiveness. Support for relative, emerging standards. Support for flexible rights management (metered, pay per view..)

51. MITA 51 Example of challenges Standardization: Fragmentation e.g. 14 payment schemes [16], 25 DRM standards initiative [22]. Does OMA help?1622 Example MPEG-21 multimedia delivery standard [15] :15 –Open Mobile Alliance (Nokia et al.): Open Digital Rights Language –XML-based, open, free standard –Motion Picture Experts Group (MPEG): eXtensive rights Markup Language (XrML) - more complicated than OMA (More papers on [15,16,17,22])15161722

52. MITA 52 OMA & mobile DRM roadmap OMA DRM 1.0: Forward-lock: phone won’t send Combined: content and rights: e.g. use for a day Separate: encrypted content and rights delivery: enables superdistribution to other users - v.2 PKI-infrastructure? [ 15 ] 15

53. MITA 53 Nokia content delivery platform [15]15 1: Client initiates purchase 2: Discussion about content & rights 3: Announcement to user 4: Client initiates download 5: Authentication 6: Content from storage 7: Content & rights to user 8: Charging OMA v1.0 compliant system product data from Nokiadata

54. MITA 54 Another DRM solution (NDS) [24]24 Various formats end-to-end encrypted User need certain player for handset Operator of course needs different components (encryption, license server, ticket server, delivery..) Own extensions, but OMA compliant Other NDS DRM methods (not too cheap and easy?): –Use OS and SIM integrated security –Hardware encryption

55. MITA 55 MITA book reference implementation -Requirements for MITA’s DRM demonstration [1] :1 -Only compliant (so called TransSec nodes) devices may get rights -Possible to track rights (on-line) -Creating rights possible off-line -Any payment method will do -Modularity e.g. separate protocol engine, User Rights Management API and Access Control API

56. MITA 56 Music, music, music… -Estimates from ringing tones: in Europe in 2001: $0,6..1,7 Mrd worldwide 2002: $1,5 Mrd -Two ways to get music: -download -streaming -Good idea for mobile users - Hot topic also in Internet [25] :25 P2P & RIAA, iTunes, Napster2 -To avoid problems in mobile world by early designing [19] Mobile Multimedia Study (2002), Andersen consulting19

57. MITA 57 Games -Other predictions to year 2006: $3,6 billion and $4,4 billion [21]21 -Mobile gamers in USA 7M in 2002 (IDC estimate), 2007 112,4M [21]21 -New mobile devices (big color displays, joystick, voices..) -Different run-time environments: Java key technology [20]20 -Licenced formats e.g. Codetoys ”Who Wants to be a Millionaire?” -Mobility enabling positioning based and other on-line interactive games -Entertainment in home, bus... Affordable device nearly all (in future) Mobile gaming revenue prediction [19], Play Away: The future of mobile entertainment (2002)19

58. MITA 58 Videos - Requires broad bandwidth -> big growth not yet started - Revenue estimates in Europe 100 MEUR (2004) [20]20 - Sport, news, movies trailers, adult content… - 3G survey:40% would be interested to download video clips [20]20 -OMA DRM in video messages transported with: - MMS: 100kb, small clips, stored - OTA: tcp/ip, little bit longer clips, stored [23]23 -Would audio and video streams require DRM? -Paper [23] says no, because those are not stored23 -e.g. Real media streams downloadable from Internet.

59. MITA 59 DRM revenue scenarious for mobile operators in Western Europe Blue: revenues for operators Violet: revenues for 3rd party [16] Consult reports:16 - Strnd, - Frost&Sullivan, - Baskerville - Ovum

60. MITA 60 Mobile content revenue scenario for mobile operators in Western Europe Estimated revenues for mobile content in world in 2003 $9 billion and in 2007 $39 billion [24, Ovum Research]24 [16] Consult reports:16 - Strnd, - Frost&Sullivan, - Baskerville - Ovum

61. MITA 61 DRM revenus sources scenario for mobile operators in Western Europe DRM revenue sources forecast for Mobile Operators in Western Europe

62. MITA 62 References 1.Nokia – “Mobile Internet Technical Architecture – visions and Implementation”, IT Press, 2002 2.Nordic wireless http://www.nordicwirelesswatch.comhttp://www.nordicwirelesswatch.com 3.3gnewsroom: http://www.3gnewsroom.comhttp://www.3gnewsroom.com 4.Eurescom http://www.eurescom.dehttp://www.eurescom.de 5.Hjelm, Johan : Designing wireless information services, Wiley Computer Publishing 2000 6.Oinas-Kukkonen, H., Kurkela, V.: Developing Successful Mobile Applications in proc, of IASTED int. conf. on Computer science and technology, may 19-21, 2003, Cancun Mexico

63. MITA 63 References 7.Kaikkonen, A., Roto, V. Navigating in a Mobile XHTML Application. Research paper, NRC 8.Next Genetration Mobile Browsing. Nokia White Paper. www.forum.nokia.com/main/1,6566,1_1,00.html. Visited 12.11.2003 www.forum.nokia.com/main/1,6566,1_1,00.html 9.Kennedy C.J. Voice Browsing: How Two Great Ideas Go Great Together. www.wirelessdevnet.com/channels/voice. Visited 29.10.2003www.wirelessdevnet.com/channels/voice 10.VoiceXML Forum Announces Support for W3C’s Multimodal Interaction Activity. www.voicexml.org. Visited 29.10.2003www.voicexml.org

64. MITA 64 References 11.Software AG. VoiceXML for speech-activated information retrieval. www.softwareag.com/xml/library/. Visited 29.10.2003www.softwareag.com/xml/library/ 12.Speechtechnology Magazine. Comverse to Demonstrate Multimodal User Experience. 7.10.2003 www.speechtechmag.com. Visited 11.11.2003www.speechtechmag.com 13.Nokia White Paper: Instant Messaging goes mobile, www.nokia.com/downloads/solutions/mobile_software/ instant_messaging_goes_mobile.pdf www.nokia.com/downloads/solutions/mobile_software/ instant_messaging_goes_mobile.pdf 14.Mobile instant messages website http://www.mobileinstantmessages.com/ Visited 18.11.2003 http://www.mobileinstantmessages.com/

65. MITA 65 References 15.Sonera medialab. Mobile Digital Rights Management (MDRM) Whitepaper. 08.08.2003. http://www.medialab.sonera.fi/2003/08/08 http://www.medialab.sonera.fi/2003/08/08 16.Salz, P.A. The DRM Dilemma. Mobile Communication International. Issue 105, October 2003. Informa telecoms group 17.Proceedings of the first international mobile IPR workshop: Rights management of information prducts on the mobile Internet. Olli Pitkänen (ed.) 27.08.2003 http://www.hiit.fi/publications/pub_files/mobileipr2003-2.pdf http://www.hiit.fi/publications/pub_files/mobileipr2003-2.pdf 18.Durand, J. Nokia. Nokia - Position Paper W3C Workshop on Digital Rights Management http://www.w3.org/2000/12/drm- ws/pp/nokia-durand.htmlhttp://www.w3.org/2000/12/drm- ws/pp/nokia-durand.html

66. MITA 66 References 19.Mobile Entertainment Industry and Culture, Key Actors WP5 - Mobile entertainment business Deliverable D5.1.1. 10.02.2003. http://www.mgain.org/mgain-wp5-d511-delivered.pdf http://www.mgain.org/mgain-wp5-d511-delivered.pdf 20.Mobile Entertainment Industry and Culture, Market Survey WP5 – Business Models Deliverable D5.4.1. 29.08.2003 http://www.mgain.org/mgain-wp5-D541-delivered.pdf http://www.mgain.org/mgain-wp5-D541-delivered.pdf 21.Future Mobile Entertainment Scenarios. Mobile Entertainment Forum. Whitepaper. March 2003. http://www.mobileentertainmentforum.org/pdf/MEF-WP-on-Future- ME-Scenarios.pdf http://www.mobileentertainmentforum.org/pdf/MEF-WP-on-Future- ME-Scenarios.pdf

67. MITA 67 References 22.Ketchell, J. CEN/ISSS - Information Society Standardization System, Global Collaboration for Mobile Commerce Transactions Workshop, 09-10.04.2003. Jointly organized by ETSI and the UMTS Forum. http://www.etsi.org/agreement/Workshops/Workshop_1/John_jkET SI_mcomm.ppt http://www.etsi.org/agreement/Workshops/Workshop_1/John_jkET SI_mcomm.ppt 23. Hyppönen, J. Mobile Video Applications. Nokia Mobile Phones. http://ncsp.forum.nokia.com/downloads/nokia/documents/Nokia_Mobi le_Video_Workshop_Presentation_JH.pdf http://ncsp.forum.nokia.com/downloads/nokia/documents/Nokia_Mobi le_Video_Workshop_Presentation_JH.pdf 24.Deutsch, J., NDS. NDS- Enabling technology for securely mobilizing content. September 2003. http://www.broadcastpapers.com/asset/NDSMobileContent01.htm http://www.broadcastpapers.com/asset/NDSMobileContent01.htm 25.Internet Application usage continues to decline. Nielsen/Netratings http://www.netratings.com/pr/pr_030929.pdf

68. MITA 68 Questions? What problems mobility causes for applications? What kind of applications benefit from rich call? How receiving streaming affects to the mobility of the user? What kind of IM services the users are willing to pay for? Will users want to be reachable any time, anywhere? What does the web-page creator has to take into account mobile browsing? Which applications require seamless Mobile World connectivity?

69. MITA 69 To think about before sleeping Do you believe mobile DRM will succeed (200?) ? –Standardization: DRM & payment methods swamp for operators, device manufacturers, content developers, users –Consumers can and want to use DRM services? –Mobile world will be (quite) free from piratism? DRM enables (new media) business growth?