Version : 11 December 2008 ITU Workshop on “Monitoring and Benchmarking of QoS and QoE of Multimedia Services in Mobile Networks” (Buenos Aires, Argentina, July 2014) Update on ITU-T & ETSI standardization activities for LTE QoS / QoE Dr. Peter Počta University of Žilina, Slovakia Joachim Pomy OPTICOM, Germany

Buenos Aires - Argentina Julyl  ITU-T SG 12, Question 11:  "Performance interworking and traffic management for Next Generation Networks "

Buenos Aires - Argentina Julyl G.VoLTE (1)  C-127: Inputs for G.VoLTE (Huawei Technologies Co. Ltd., France Télécom Orange)  At last SG12 meeting, two contributions (C.074 from Orange and C.076 from Ascom) addressed the need for starting the development of specific requirements on end-to-end Qos for voice over 4G mobile networks. The decision has been taken to develop a new Recommendation, provisionally called G.VoLTE, under Q.11 (see the report R.4 of WP3), with the following description:

Buenos Aires - Argentina Julyl G.VoLTE (2)  This Recommendation describes the key aspects impacting end-to-end performance of managed voice applications over LTE networks.

Buenos Aires - Argentina Julyl G.VoLTE (3)  This Recommendation makes assumptions about relevant QoS mechanisms used to manage the voice service, and studies the impact of such mechanisms on the end-to-end network performance. Some typical end-to-end scenarios are described, involving cases with LTE access at both sides of the communication, or with a different access technology at one side (wireless or wireline access).

Buenos Aires - Argentina Julyl G.VoLTE (4)  A typical reference connection is defined in this Recommendation, composed of various segments, including: terminal, wireless access, backhaul network, core network. Considerations regarding the sharing of the budget of some key parameters across these segments, such as latency, are provided.

Buenos Aires - Argentina Julyl QoS of connections from current technologies to LTE for delay sensitive applications (ETSI STF 437)

Buenos Aires - Argentina Julyl Outline Introduction of the ETSI STF 437 project (scope, team),  QoS problems encountered by operators,  Shortcomings of standards and implementations,  Possible solutions (actions for standards and implementations),

Buenos Aires - Argentina Julyl Introduction of the ETSI STF 437  Project scope:  address LTE related QoS problems for delay sensitive applications,  determine possible shortcomings of the standards and possible shortcomings of implementations  propose possible solutions  Project team:  Peter Počta (leader)  University of Žilina, Slovakia  Jan Holub  Mesaqin.com s.r.o. (Ltd.), Czech Republic  Joachim Pomy  Telecommunications & Int’l Standards, Germany  Catherine Quinquis  France Telecom, France

Buenos Aires - Argentina Julyl QoS problems encountered by operators  Contacted operators and stakeholders only trial LTE technology  Trials mostly focus on a performance of data services (lab implementation).  One of the contacted operators focused on voice service in its trial, following problems have been encountered:  Some problems with CS Fallback (was not working properly),  Problem with routing between radio part (LTE network) and IMS,  All problems have been fixed by vendor (wrong implementation).  Conclusion: operators have not already started to deploy Voice over LTE in their networks.  To sum up current situation, according to market observers and visionaries:  Network operators are planning to deploy VoLTE in upcoming years (2013 or 2014).  The potential subscribers of LTE expect better performing data services than in 3G and services that take advantage of the lower latency, such as voice and video. (Quality of service – crucial role in an adaptation process of VoLTE).  Implication: QoS should be properly implemented and tested in trials to ensure sufficient quality level to speed up/drive the adaptation process of VoLTE.

Buenos Aires - Argentina Julyl Shortcomings of standards and implementations  Shortcomings of standards:  Standards focusing on IMS Multimedia Telephony, IMS Profiles for voice service, SMS and conversational video service:  According to ETSI TS , the MTSI client should only support AMR codec for voice service.  The jitter-buffer management mechanisms of video service described in the ETSI TS seem to be very weak.  The quality of experience metrics defined in clause 16 of ETSI TS can be more or less only considered as QoS parameters.

Buenos Aires - Argentina Julyl Shortcomings of standards and implementations  Shortcomings of standards:  Standards describing resource and admission control mechanisms developed by 3GPP and ETSI (3GPP Policy and Charging Control (PCC) and TISPAN Resource and Admission Control Subsystem (RACS)):  3GPP PCC and TISPAN RACS vary considerably in architecture, supported networks and node types (big functional differences).  To harmonize them, a following activity called “Framework for Gq' / Rx harmonization” was started in 2007 by 3GPP.  In 2009, the harmonization activity mentioned above has been abandoned and all published specifications has been withdrawn (the developed solutions has been lost).  Main problem: no interface is available for communication between the TISPAN RACS and the 3GPP PCC. Consequently, they can not interact and negotiate with each other in order to finish the operations required for continuous QoS guarantee (e.g. resource reservation).

Buenos Aires - Argentina Julyl Shortcomings of standards and implementations  Shortcomings of implementations:  Interim solutions:  Circuit-switched fallback (CSFB): Uses 2G/3G for voice and requires interruption of the LTE connection; this is already affected by the signalling of an incoming call.  Voice over LTE Generic Access (VoLGA): Tunnels the voice call from the 2G/3G to the LTE terminal via an LTE data channel; requires additional equipment in the network; lack of QoS.  Fast Track VoLTE: Proprietary solution providing voice service via the LTE technology; adds SIP stack to existing mobile network equipment.  Over-the-Top solutions (OTT): VoIP applications operating via an LTE data channel, running on the LTE terminal; lack of QoS.  Migration of voice service to LTE:  Before commercial deployment, a number of lab, field and market trials should be conducted.  User speech quality expectations are very high. If voice services do not deliver the necessary level of quality and reliability, users will revert back to existing circuit-switched options or, in some cases, simply rely on over- the-top solutions.

Buenos Aires - Argentina Julyl Shortcomings of standards and implementations  Shortcomings of implementations:  VoLTE Interop testing:  was organised by the Multi-Service Forum (MSF) and backed by the GSMA in September 2011 and October  a couple of different aspects has been tested, mainly focusing on signalling issues, multivendor interoperability.  MSF reports in the white paper summarizing VoLTE Interop event 2011: " The event proved that VoLTE is a viable solution for providing voice services over LTE access technology, and can be deployed with roaming and interconnect functionality to provide equivalent service as CS-based voice services today.“ – What exactly has been tested? – What kind of understanding of the terms "Quality of Service" and "service" have been prevalent?  After a detailed view into the scenarios of this interoperability event, the STF 437 team has concluded that QoS of the media channel is not tested at all.

Buenos Aires - Argentina Julyl Possible solutions (actions for standards and implementations)  Actions for standards:  Codec aspects:  In principle, there are three possible solutions: – Tandeming (lower quality and increased delay), – End-to-end connection based on AMR codec (lower quality), – using other codecs supported by IMS for end-to-end connections.  STF 437 team advice: MTSI client defined in ETSI TS should have the possibility to access every codec supported by SIP.  Action point: appropriate standard should be extended accordingly.  Jitter-buffer Management Mechanisms of video service: – are currently not or at least not sufficiently standardized. Further study in this area is of critical importance.

Buenos Aires - Argentina Julyl Possible solutions (actions for standards and implementations)  Actions for standards:  Quality of Experience metrics:  There are still no clear standards in place, which can support network operators in selecting those parameters in the QoE extension.  It is proposed by STF 437 team to follow the approach of a Human Factors Extension to the Seven-Layer OSI Reference Model proposed by Bauer and Patrick: – Main idea: three additional HCI (Human Computer Interface) layers capture the additional quality parameters introduced by the concept of QoE over QoS, whereas the classical 7 OSI layers are synonymous to classical QoS parameters. – The above mentioned three additional layers focusing on HCI are: – Layer No.8: Display (keyboard, GUI/CLI, vocal, Bpp, Ppi, Ppm, etc.), – Layer No.9: Human Performance (perception, cognition, memory, motor control, social, etc.), – Layer No.10: Human Needs (communication, education, acquisition, security, entertainment, etc.).

Buenos Aires - Argentina Julyl Possible solutions (actions for standards and implementations)  Actions for standards:  Framework for Gq’/Rx harmonization:  Crucial to reopen the activity called “Framework for Gq’/Rx harmonization”: – define interactive interface supporting sufficient exchange of information between TISPAN RACS and 3 GPP PCC – harmonize the communication between the two different resource and admission control solutions to satisfy the subscribers’ service experience

Buenos Aires - Argentina Julyl Possible solutions (actions for standards and implementations)  Actions for implementations:  Migration of voice service to LTE:  Delay values achieved in trials and real implementations have to be carefully observed.  Sufficient algorithms/techniques should be deployed in the networks in order to ensure continuous data sessions for incoming and outgoing voice/video calls.  A quality of service perceived by the end-user should be carefully measured in trials and monitored in real implementations to ensure the best quality provided to the users.  VoLTE Interop testing:  TC INT has begun to consider inclusion of QoS testing aspects into the specifications for forthcoming VoLTE interoperability events.  STF 437 team has proposed an example of improved test scenario presented on next slide (Fig.1).  While this is a good starting point, the challenge of QoS in heterogeneous scenarios interconnecting current technologies with LTE may require additional effort in the future.  Another option: the organization of a pure Speech & Video Quality Test Event for LTE.

Buenos Aires - Argentina Julyl Possible solutions (actions for standards and implementations) Figure 1: Example of improved test scenario for VoLTE interop test

Buenos Aires - Argentina Julyl TOWARDS QUALITY AND QOS ASSESSMENT FOR VOLTE IN ETSI ETSI TC INT Chair: Giulio Maggiore, Telecom Italia © ETSI All rights reserved

Buenos Aires - Argentina Julyl TC INT (Core Network and Interoperability Testing)  Development of Core Network test specifications (interoperability, conformance, performance, security) based on 3GPP and E2NA specifications.  To initiate and supervise interoperability events (such as Plugtests) related to Core Networks as well as other events (workshops and seminars)  To coordinates interoperability efforts with other organisations GSMA, OMA, IMS/NGN Forum, IMTC, etc.  Endorsement of test specifications from/to other SDOs e.g. ITU-T. 21 © ETSI All rights reserved

Buenos Aires - Argentina Julyl TC INT Interoperability Specification Work R6 R1 R2 R7Late R7 Late R2 R8 R9 R TS V TS V TS V TS V TS V TS V TS V TS TS V TS V TS V E2NA 22

Buenos Aires - Argentina Julyl – VOLTE QOS ASSESSMENT TECHNOLOGY EVALUATION EVENT 23 © ETSI All rights reserved ETSI STF#453 STF Leader: Peter Schmitting, FSCom

Buenos Aires - Argentina Julyl VoLTE QoS Assessment event overview © ETSI All rights reserved 24  Scope: Technology evaluation Plugtests aiming at validating the different methodologies for end to end QoS Assessment for VoLTE described in ETSI TS over a deployed LTE/IMS network.  Dates: November 2013  Venue: ETSI, Sophia Antipolis, France  Web: assessment assessment-2013

Buenos Aires - Argentina Julyl VoLTE QoS Assessment event scope © ETSI All rights reserved 25  End to end tests performed with several different UEs over a deployed LTE / IMS test network.  4 Test Setups  Acoustic to acoustic  Acoustic to electrical  Electrical to acoustic  Electrical to electrical  Voice quality tests based on instrumental assessment of speech samples performed on different combinations of the acoustic and electrical interfaces which allows evaluating the different methodologies and setups.  Parallel network performance evaluation

Buenos Aires - Argentina Julyl – VoLTE QoS Assessment Technology Evaluation event – test description TS  End-to-end Tests based on Instrumental Assessment of Speech Samples (POLQA model deployed):  Mean Opinion Score (MOS)-Listening Quality Objective  Delay variation over time  Difference in loudness level  MOS stability  End-to-end Tests based on Instrumental Assessment of Video Samples (PEVQ model deployed):  DMOS (Difference Mean Opinion Score)  Delay and Quality variation over time  Jerkiness and Blockiness  Blur, Chrominance and Luminance  Temporal Distortion  Frozen and Skipped Frames

Buenos Aires - Argentina Julyl – VoLTE QoS Assessment Technology Evaluation event – test description TS  Other end-to-end tests in the voice channel:  End-to-end frequency response  Overall loudness rating  End-to-end delay  Quality of echo cancellation  Double talk performance  End-to-end Tests based on functional parameters:  Telephony Call Setup Time  Streaming Service Access Time  Network Performance Parameters  UNI-to-UNI delay over time  UNI-to-UNI jitter  UNI-to-UNI packet loss

Buenos Aires - Argentina Julyl VoLTE QoS Assessment – Setup 1 Acoustic-to-Acoustic © ETSI All rights reserved 28

Buenos Aires - Argentina Julyl VoLTE QoS Assessment – Setup 2 Acoustic to Electrical © ETSI All rights reserved 29

Buenos Aires - Argentina Julyl VoLTE QoS Assessment – Setup 3 Electrical to Acoustic © ETSI All rights reserved 30

Buenos Aires - Argentina Julyl VoLTE QoS Assessment – Setup 4 Electrical to Electrical © ETSI All rights reserved 31

Buenos Aires - Argentina Julyl What’s next?  Validation of 76% of the test cases in TS  A number of gaps and potential improvements in TS have been identified  A new work item has been open for the update of TS with the new setup, acoustic-to-electrical identified during the event  Next event will be launched when VoLTE Native Handset and PEVQ tools will be available for testing 32 © ETSI All rights reserved

Buenos Aires - Argentina Julyl Further steps with regard to ETSI TS  Verification of remaining voice quality test cases  tbd  Verification of video quality test cases (RTP- based video streaming):  Test houses willing to do so are needed  Extension of video quality test cases towards MPEG-DASH  ITU-T SG12 Question 14 currently running a WI called P.NATS

Buenos Aires - Argentina Julyl ITU-T SG12 Question14 Project P.NATS  P.NATS focusing on a development of a model  consisting of a collection of objective parametric quality assessment modules  predicting the impact of observed IP network impairments on quality experienced by the end-user  in multimedia mobile streaming and fixed network applications  using progressive download, including adaptive streaming methods  The development is divided into 2 parts, namely:  Track No.1: bitstream-based parametric models (payload and/or header information)  Track No.2: full-reference and hybrid models for the video coding quality module.  Track No.2 is of our interest as it correlates well with an aim of VoLTE QoS Test event  Models for both tracks are supposed to be available at the end of this year.

Buenos Aires - Argentina Julyl Preparation of new Project in EU context  Test Methodologies for QoS, Mobile Broadband & Autonomic Networking  Test methodologies for the interoperability with guarantied Quality of Services (QoS) on the mobile broadband  in a multi-vendor  multi-network  multiservice environment  The test methodologies coming out from this project will be applied to the preparation for the future ETSI Plugtests events.  As holistic test methodologies, the proposal addresses also the test methodologies for the future internet with  autonomic networking,  autonomic control and management.  Such a global network infrastructure has the capabilities of  self-establishing,  self-configuring,  self-optimizing and self-healing.  For autonomics the decisions are made at the communication protocols, functions, nodes and network-wide levels.

Buenos Aires - Argentina Julyl Any questions ? Contact: