Bridging troubled water: Quality of Experience in a mobile media context Katrien De Moor, Lieven De Marez MICT-IBBT, Dept. of Communication Sciences, Ghent University Wout Joseph, Tom Deryckere, Luc Martens WiCa-IBBT, Dept. of Information Technology, Ghent University Terena Networking Conference 2008

Introduction and context  Changing ICT environment  User experience – Quality of Experience  ‘Interactionism’ perspective  User-driven innovation  Examples 2 MICT-WiCa De Moor et al.

Today’s Innovation Spiral Technology evolving at a dazzling speed, … intensified competition, … more innovations to distinguish themselves from competitors, … shortening product life cycles, … increased pressure on NPD processes & introduction strategies, … more failing innovations, … 3

4

5 “More and more companies have recognized the need to learn from users in order to develop innovative, user experience’-centred products” (Sleeswijk Visser, Van der Lugt, Stappers, 2007: 35) ”The consumer is king – and needs high QoE” (Kumar, 2005: 39) ”QoE became the ultimate business metric & competitive battleground” (Empirix, 2005: 1) Increased importance of the integration of user- & user experience oriented research in NPD processess

6

7

8 The era of ‘user experience technology’? Quality of Experience = buzzword, USP, ultimate business metric,… Central question How to deliver high QoE for end-users? Problem: ‘Clash’ of visions, fragmented efforts MICT-WiCa De Moor et al.

9 Striving for QoE delivery: ‘clash of visions’ “Natuurlijk vonden veel technici het vervelend dat geen enkel technisch detail genoemd mocht worden, maar in dit geval moesten de emotie en de experience spreken” (Philips, 06/12/2007) A: Gaining insight in what users want, expect, feel, think,… as a means to enhance end-user QoE B: QoS optimisation as a means to optimise end-user QoE QoE = QoS ? MICT-WiCa De Moor et al.

10 QoE = QoS? Delivering high QoE should be the main goal in NPD and striving for QoS optimisation a necessary, but unsatisfactory means to achieve that goal QoS is only one necessary component of QoE! MICT-WiCa De Moor et al.

QoE as a multidimensional concept (1)  5 Building blocks  ‘Quality of Effectiveness’ (~QoS)   Application/Service  Server  Network  Device/Handset  ‘Quality of Efficiency’ (does it work well enough for the user?)   Device/Handset  Network  Application/Service 11 MICT-WiCa De Moor et al.

QoE as a multidimensional concept (2)  5 Building blocks  ‘Usability’ (behavioural and emotional)  ‘Expectations’ (degree up to which expectations are met (or eg. exceeded), will determine part of the QoE  ‘Context’  Environmental Context (eg. public/private, mobility, usage context, …)  Personal & Social Context (eg. Identity creation, emotions, empowerment, …)  Cultural Context (eg. age, values, …)  Technological Context (eg. Tangibles, billing, cost, usage intensity, compatibility, …)  Organisational Context 12 MICT-WiCa De Moor et al.

13 Multidimensional conceptual model

14 Example  Conceptual model, based on desk research & expert panel QoE QoS Example GAMING STUDY Usability Expectations Context Latency, quality of image, speed … Easy navigation, interface… gameplay, content, response time… Community aspect, online, cost, … MICT-WiCa De Moor et al.

15  Currently: too much post development measurements (3) and usability and QoS measurements (2) Prior-to-development & Prior-to-launch ???? Expectations? (Post)Development & Prior-to-launch Prototypes, demo’s, …. Usability testing QoS optimisation Post development & Post-launch User research: Satisfaction measurements Good experience? Delivering high QoE: how? MICT-WiCa De Moor et al.

16 New methodology for bridging the QoE-QoS gap MICT-WiCa De Moor et al.

Empirical case-study  Study of 2 I-City applications, N=10  Interdisciplinary team  Focus on bridging gap between QoE (user-oriented) and QoS (technology-oriented approaches) in NPD  Translation/transformation of insights from user research  test and refinement of new methodology  5 phases 17 MICT-WiCa De Moor et al.

Some results (1) Pre-usage user research: detection of relevant QoE dimensions and user expectations (quali+quantitative) e.g. price, navigation, speed, display size, … 2. Pre-usage translation workshops  social scientists and engineers  e.g. photo-download speed application 3. Monitoring during usage  usage scenarios for test users  different reception levels + monitoring of ‘signal strenght’  probe model (cfr. Deryckere, Joseph et al, 2008) MICT-WiCa De Moor et al.

Some results (2) Post-usage questions on device: after completion of usage scenario  questions on device (general experience, frustration,,…) example: resp. B, male, 30 (Wineguide)

Some results (3) Post-usage user research: comparison of expectations vs. experience: gaps? Explanation? e.g. person B, 30, male (Wineguide) MICT-WiCa De Moor et al.

Conclusion -Integrated approach for QoE-measurement -Both objective and subjective parameters -Future research: Larger scale testings: more test users and usage contexts Extended probe model: more parameters for monitoring (e.g. contextual probe: GPS data, user- generated information about current activities, mood, …) Personalised set of questions on device 21 MICT-WiCa De Moor et al.

22 THANK YOU! QUESTIONS? CONTACT US: