1. Version : 11 December 2008 ITU Workshop on “Monitoring and Benchmarking of QoS and QoE of Multimedia Services in Mobile Networks” (Buenos Aires, Argentina, 24-25 July 2014) Update on ITU-T & ETSI standardization activities for LTE QoS / QoE Dr. Peter Počta Peter.Pocta@fel.uniza.sk University of Žilina, Slovakia Joachim Pomy Consultant@joachimpomy.de OPTICOM, Germany

2. Buenos Aires - Argentina - 24 - 25 Julyl 2014 2  ITU-T SG 12, Question 11:  "Performance interworking and traffic management for Next Generation Networks "

3. Buenos Aires - Argentina - 24 - 25 Julyl 2014 3 G.VoLTE (1)  C-127: Inputs for G.VoLTE (Huawei Technologies Co. Ltd., France Télécom Orange)  At last SG12 meeting, two contributions (C.074 from Orange and C.076 from Ascom) addressed the need for starting the development of specific requirements on end-to-end Qos for voice over 4G mobile networks. The decision has been taken to develop a new Recommendation, provisionally called G.VoLTE, under Q.11 (see the report R.4 of WP3), with the following description:

4. Buenos Aires - Argentina - 24 - 25 Julyl 2014 4 G.VoLTE (2)  This Recommendation describes the key aspects impacting end-to-end performance of managed voice applications over LTE networks.

5. Buenos Aires - Argentina - 24 - 25 Julyl 2014 5 G.VoLTE (3)  This Recommendation makes assumptions about relevant QoS mechanisms used to manage the voice service, and studies the impact of such mechanisms on the end-to-end network performance. Some typical end-to-end scenarios are described, involving cases with LTE access at both sides of the communication, or with a different access technology at one side (wireless or wireline access).

6. Buenos Aires - Argentina - 24 - 25 Julyl 2014 6 G.VoLTE (4)  A typical reference connection is defined in this Recommendation, composed of various segments, including: terminal, wireless access, backhaul network, core network. Considerations regarding the sharing of the budget of some key parameters across these segments, such as latency, are provided.

7. Buenos Aires - Argentina - 24 - 25 Julyl 2014 7 QoS of connections from current technologies to LTE for delay sensitive applications (ETSI STF 437)

8. Buenos Aires - Argentina - 24 - 25 Julyl 2014 8 Outline Introduction of the ETSI STF 437 project (scope, team),  QoS problems encountered by operators,  Shortcomings of standards and implementations,  Possible solutions (actions for standards and implementations),

9. Buenos Aires - Argentina - 24 - 25 Julyl 2014 9 Introduction of the ETSI STF 437  Project scope:  address LTE related QoS problems for delay sensitive applications,  determine possible shortcomings of the standards and possible shortcomings of implementations  propose possible solutions  Project team:  Peter Počta (leader)  University of Žilina, Slovakia  Jan Holub  Mesaqin.com s.r.o. (Ltd.), Czech Republic  Joachim Pomy  Telecommunications & Int’l Standards, Germany  Catherine Quinquis  France Telecom, France

10. Buenos Aires - Argentina - 24 - 25 Julyl 2014 10 QoS problems encountered by operators  Contacted operators and stakeholders only trial LTE technology  Trials mostly focus on a performance of data services (lab implementation).  One of the contacted operators focused on voice service in its trial, following problems have been encountered:  Some problems with CS Fallback (was not working properly),  Problem with routing between radio part (LTE network) and IMS,  All problems have been fixed by vendor (wrong implementation).  Conclusion: operators have not already started to deploy Voice over LTE in their networks.  To sum up current situation, according to market observers and visionaries:  Network operators are planning to deploy VoLTE in upcoming years (2013 or 2014).  The potential subscribers of LTE expect better performing data services than in 3G and services that take advantage of the lower latency, such as voice and video. (Quality of service – crucial role in an adaptation process of VoLTE).  Implication: QoS should be properly implemented and tested in trials to ensure sufficient quality level to speed up/drive the adaptation process of VoLTE.

11. Buenos Aires - Argentina - 24 - 25 Julyl 2014 11 Shortcomings of standards and implementations  Shortcomings of standards:  Standards focusing on IMS Multimedia Telephony, IMS Profiles for voice service, SMS and conversational video service:  According to ETSI TS 126 114, the MTSI client should only support AMR codec for voice service.  The jitter-buffer management mechanisms of video service described in the ETSI TS 126 114 seem to be very weak.  The quality of experience metrics defined in clause 16 of ETSI TS 126 114 can be more or less only considered as QoS parameters.

12. Buenos Aires - Argentina - 24 - 25 Julyl 2014 12 Shortcomings of standards and implementations  Shortcomings of standards:  Standards describing resource and admission control mechanisms developed by 3GPP and ETSI (3GPP Policy and Charging Control (PCC) and TISPAN Resource and Admission Control Subsystem (RACS)):  3GPP PCC and TISPAN RACS vary considerably in architecture, supported networks and node types (big functional differences).  To harmonize them, a following activity called “Framework for Gq' / Rx harmonization” was started in 2007 by 3GPP.  In 2009, the harmonization activity mentioned above has been abandoned and all published specifications has been withdrawn (the developed solutions has been lost).  Main problem: no interface is available for communication between the TISPAN RACS and the 3GPP PCC. Consequently, they can not interact and negotiate with each other in order to finish the operations required for continuous QoS guarantee (e.g. resource reservation).

13. Buenos Aires - Argentina - 24 - 25 Julyl 2014 13 Shortcomings of standards and implementations  Shortcomings of implementations:  Interim solutions:  Circuit-switched fallback (CSFB): Uses 2G/3G for voice and requires interruption of the LTE connection; this is already affected by the signalling of an incoming call.  Voice over LTE Generic Access (VoLGA): Tunnels the voice call from the 2G/3G to the LTE terminal via an LTE data channel; requires additional equipment in the network; lack of QoS.  Fast Track VoLTE: Proprietary solution providing voice service via the LTE technology; adds SIP stack to existing mobile network equipment.  Over-the-Top solutions (OTT): VoIP applications operating via an LTE data channel, running on the LTE terminal; lack of QoS.  Migration of voice service to LTE:  Before commercial deployment, a number of lab, field and market trials should be conducted.  User speech quality expectations are very high. If voice services do not deliver the necessary level of quality and reliability, users will revert back to existing circuit-switched options or, in some cases, simply rely on over- the-top solutions.

14. Buenos Aires - Argentina - 24 - 25 Julyl 2014 14 Shortcomings of standards and implementations  Shortcomings of implementations:  VoLTE Interop testing:  was organised by the Multi-Service Forum (MSF) and backed by the GSMA in September 2011 and October 2012.  a couple of different aspects has been tested, mainly focusing on signalling issues, multivendor interoperability.  MSF reports in the white paper summarizing VoLTE Interop event 2011: " The event proved that VoLTE is a viable solution for providing voice services over LTE access technology, and can be deployed with roaming and interconnect functionality to provide equivalent service as CS-based voice services today.“ – What exactly has been tested? – What kind of understanding of the terms "Quality of Service" and "service" have been prevalent?  After a detailed view into the scenarios of this interoperability event, the STF 437 team has concluded that QoS of the media channel is not tested at all.

15. Buenos Aires - Argentina - 24 - 25 Julyl 2014 15 Possible solutions (actions for standards and implementations)  Actions for standards:  Codec aspects:  In principle, there are three possible solutions: – Tandeming (lower quality and increased delay), – End-to-end connection based on AMR codec (lower quality), – using other codecs supported by IMS for end-to-end connections.  STF 437 team advice: MTSI client defined in ETSI TS 126 114 should have the possibility to access every codec supported by SIP.  Action point: appropriate standard should be extended accordingly.  Jitter-buffer Management Mechanisms of video service: – are currently not or at least not sufficiently standardized. Further study in this area is of critical importance.

16. Buenos Aires - Argentina - 24 - 25 Julyl 2014 16 Possible solutions (actions for standards and implementations)  Actions for standards:  Quality of Experience metrics:  There are still no clear standards in place, which can support network operators in selecting those parameters in the QoE extension.  It is proposed by STF 437 team to follow the approach of a Human Factors Extension to the Seven-Layer OSI Reference Model proposed by Bauer and Patrick: – Main idea: three additional HCI (Human Computer Interface) layers capture the additional quality parameters introduced by the concept of QoE over QoS, whereas the classical 7 OSI layers are synonymous to classical QoS parameters. – The above mentioned three additional layers focusing on HCI are: – Layer No.8: Display (keyboard, GUI/CLI, vocal, Bpp, Ppi, Ppm, etc.), – Layer No.9: Human Performance (perception, cognition, memory, motor control, social, etc.), – Layer No.10: Human Needs (communication, education, acquisition, security, entertainment, etc.).

17. Buenos Aires - Argentina - 24 - 25 Julyl 2014 17 Possible solutions (actions for standards and implementations)  Actions for standards:  Framework for Gq’/Rx harmonization:  Crucial to reopen the activity called “Framework for Gq’/Rx harmonization”: – define interactive interface supporting sufficient exchange of information between TISPAN RACS and 3 GPP PCC – harmonize the communication between the two different resource and admission control solutions to satisfy the subscribers’ service experience

18. Buenos Aires - Argentina - 24 - 25 Julyl 2014 18 Possible solutions (actions for standards and implementations)  Actions for implementations:  Migration of voice service to LTE:  Delay values achieved in trials and real implementations have to be carefully observed.  Sufficient algorithms/techniques should be deployed in the networks in order to ensure continuous data sessions for incoming and outgoing voice/video calls.  A quality of service perceived by the end-user should be carefully measured in trials and monitored in real implementations to ensure the best quality provided to the users.  VoLTE Interop testing:  TC INT has begun to consider inclusion of QoS testing aspects into the specifications for forthcoming VoLTE interoperability events.  STF 437 team has proposed an example of improved test scenario presented on next slide (Fig.1).  While this is a good starting point, the challenge of QoS in heterogeneous scenarios interconnecting current technologies with LTE may require additional effort in the future.  Another option: the organization of a pure Speech & Video Quality Test Event for LTE.

19. Buenos Aires - Argentina - 24 - 25 Julyl 2014 19 Possible solutions (actions for standards and implementations) Figure 1: Example of improved test scenario for VoLTE interop test

20. Buenos Aires - Argentina - 24 - 25 Julyl 2014 20 TOWARDS QUALITY AND QOS ASSESSMENT FOR VOLTE IN ETSI ETSI TC INT Chair: Giulio Maggiore, Telecom Italia giulio.maggiore@telecomitalia.it © ETSI 2013. All rights reserved

21. Buenos Aires - Argentina - 24 - 25 Julyl 2014 21 TC INT (Core Network and Interoperability Testing)  Development of Core Network test specifications (interoperability, conformance, performance, security) based on 3GPP and E2NA specifications.  To initiate and supervise interoperability events (such as Plugtests) related to Core Networks as well as other events (workshops and seminars)  To coordinates interoperability efforts with other organisations GSMA, OMA, IMS/NGN Forum, IMTC, etc.  Endorsement of test specifications from/to other SDOs e.g. ITU-T. 21 © ETSI 2013. All rights reserved

22. Buenos Aires - Argentina - 24 - 25 Julyl 2014 22 TC INT Interoperability Specification Work 2008 2009 2010 2011 2012 R6 R1 R2 R7Late R7 Late R2 R8 R9 R10 2007 TS 186 011 V 1.1.1 TS 186 011 V 2.1.1 TS 186 011 V 2.2.1 TS 186 011 V 3.1.1 TS 102 901 V 1.1.1 TS 186 011 V 4.1.1 TS 102 901 V 2.1.1 TS 103 129 TS 186 011 V 5.0.2 TS 102 901 V 5.0.6 TS 103 129 V2.0.4 2013 E2NA 22

23. Buenos Aires - Argentina - 24 - 25 Julyl 2014 23 2013 – VOLTE QOS ASSESSMENT TECHNOLOGY EVALUATION EVENT 23 © ETSI 2013. All rights reserved ETSI STF#453 STF Leader: Peter Schmitting, FSCom peter.schmitting@fscom.fr

24. Buenos Aires - Argentina - 24 - 25 Julyl 2014 24 VoLTE QoS Assessment event overview © ETSI 2013. All rights reserved 24  Scope: Technology evaluation Plugtests aiming at validating the different methodologies for end to end QoS Assessment for VoLTE described in ETSI TS 103 189 over a deployed LTE/IMS network.  Dates: 18-20 November 2013  Venue: ETSI, Sophia Antipolis, France  Web: http://www.etsi.org/news-events/events/698-volte-qos- assessment-2013 http://www.etsi.org/news-events/events/698-volte-qos- assessment-2013

25. Buenos Aires - Argentina - 24 - 25 Julyl 2014 25 VoLTE QoS Assessment event scope © ETSI 2013. All rights reserved 25  End to end tests performed with several different UEs over a deployed LTE / IMS test network.  4 Test Setups  Acoustic to acoustic  Acoustic to electrical  Electrical to acoustic  Electrical to electrical  Voice quality tests based on instrumental assessment of speech samples performed on different combinations of the acoustic and electrical interfaces which allows evaluating the different methodologies and setups.  Parallel network performance evaluation

26. Buenos Aires - Argentina - 24 - 25 Julyl 2014 26 2013 – VoLTE QoS Assessment Technology Evaluation event – test description TS 103 189  End-to-end Tests based on Instrumental Assessment of Speech Samples (POLQA model deployed):  Mean Opinion Score (MOS)-Listening Quality Objective  Delay variation over time  Difference in loudness level  MOS stability  End-to-end Tests based on Instrumental Assessment of Video Samples (PEVQ model deployed):  DMOS (Difference Mean Opinion Score)  Delay and Quality variation over time  Jerkiness and Blockiness  Blur, Chrominance and Luminance  Temporal Distortion  Frozen and Skipped Frames

27. Buenos Aires - Argentina - 24 - 25 Julyl 2014 27 2013 – VoLTE QoS Assessment Technology Evaluation event – test description TS 103 189  Other end-to-end tests in the voice channel:  End-to-end frequency response  Overall loudness rating  End-to-end delay  Quality of echo cancellation  Double talk performance  End-to-end Tests based on functional parameters:  Telephony Call Setup Time  Streaming Service Access Time  Network Performance Parameters  UNI-to-UNI delay over time  UNI-to-UNI jitter  UNI-to-UNI packet loss

28. Buenos Aires - Argentina - 24 - 25 Julyl 2014 28 VoLTE QoS Assessment – Setup 1 Acoustic-to-Acoustic © ETSI 2013. All rights reserved 28

29. Buenos Aires - Argentina - 24 - 25 Julyl 2014 29 VoLTE QoS Assessment – Setup 2 Acoustic to Electrical © ETSI 2013. All rights reserved 29

30. Buenos Aires - Argentina - 24 - 25 Julyl 2014 30 VoLTE QoS Assessment – Setup 3 Electrical to Acoustic © ETSI 2013. All rights reserved 30

31. Buenos Aires - Argentina - 24 - 25 Julyl 2014 31 VoLTE QoS Assessment – Setup 4 Electrical to Electrical © ETSI 2013. All rights reserved 31

32. Buenos Aires - Argentina - 24 - 25 Julyl 2014 32 What’s next?  Validation of 76% of the test cases in TS 103 189  A number of gaps and potential improvements in TS 103 189 have been identified  A new work item has been open for the update of TS 103 189 with the new setup, acoustic-to-electrical identified during the event  Next event will be launched when VoLTE Native Handset and PEVQ tools will be available for testing 32 © ETSI 2013. All rights reserved

33. Buenos Aires - Argentina - 24 - 25 Julyl 2014 33 Further steps with regard to ETSI TS 103 189  Verification of remaining voice quality test cases  tbd  Verification of video quality test cases (RTP- based video streaming):  Test houses willing to do so are needed  Extension of video quality test cases towards MPEG-DASH  ITU-T SG12 Question 14 currently running a WI called P.NATS

34. Buenos Aires - Argentina - 24 - 25 Julyl 2014 34 ITU-T SG12 Question14 Project P.NATS  P.NATS focusing on a development of a model  consisting of a collection of objective parametric quality assessment modules  predicting the impact of observed IP network impairments on quality experienced by the end-user  in multimedia mobile streaming and fixed network applications  using progressive download, including adaptive streaming methods  The development is divided into 2 parts, namely:  Track No.1: bitstream-based parametric models (payload and/or header information)  Track No.2: full-reference and hybrid models for the video coding quality module.  Track No.2 is of our interest as it correlates well with an aim of VoLTE QoS Test event  Models for both tracks are supposed to be available at the end of this year.

35. Buenos Aires - Argentina - 24 - 25 Julyl 2014 35 Preparation of new Project in EU context  Test Methodologies for QoS, Mobile Broadband & Autonomic Networking  Test methodologies for the interoperability with guarantied Quality of Services (QoS) on the mobile broadband  in a multi-vendor  multi-network  multiservice environment  The test methodologies coming out from this project will be applied to the preparation for the future ETSI Plugtests events.  As holistic test methodologies, the proposal addresses also the test methodologies for the future internet with  autonomic networking,  autonomic control and management.  Such a global network infrastructure has the capabilities of  self-establishing,  self-configuring,  self-optimizing and self-healing.  For autonomics the decisions are made at the communication protocols, functions, nodes and network-wide levels.

36. Buenos Aires - Argentina - 24 - 25 Julyl 2014 36 Any questions ? Contact: Consultant@joachimpomy.de