1. Colombia, 23-24 September 2013 The importance of models and procedures for planning, monitoring and control in the provision of communications services Dr.-Ing. H. W. Gierlich Head of Telecom Division CITEL (PCC.I)/ ITU Forum on Information and Communication Technology Service: Quality, Control and Surveillance (Cartagena de Indias, Colombia, 23-24 September 2013)

2. Colombia, 23-24 September 2013 2 Outline Introduction Communication Services – Underlying System configurations Planning: The ITU-T E-model (G.107 & G.108) Network Monitoring: P.862, P.863 & P.563 Wideband – The new Challenge Summary

3. ITU-T: QoS and QoE Quality of Service (QoS): Totality of characteristics of a telecommunications service that bear on its ability to satisfy stated and implied needs of the user of the service. Quality of Experience (QoE): The overall acceptability of an application or service, as perceived subjectively by the end- user. Quality of experience includes the complete end-to-end system effects (client, terminal, network, services infrastructure, etc.). Overall acceptability may be influenced by user expectations and context. Colombia, 23-24 September 2013 3

4. Speech Quality – what we would like to have Colombia, 23-24 September 2013 4 1m “orthotelefonic reference position”

5. Speech Quality… Colombia, 23-24 September 2013 5 speech quality talking situation listening situation conversational situation … from the user’s perspective

6. End to End Conifguration Colombia, 23-24 September 2013 6 ISCDSLISCPBXGW PSTN 1-15 ms240 ms MSCBSS 90-120 ms IP- GW Netw. 1 IP- GW 10 - 400 ms add. 1 - 100 ms ! Netw. 2 Impact on speech quality: - delay now time variant - echo much more dominant - Coding distortion - background noise problems - insufficient quality of the analog network components (att., noise, distortion…) - insufficient quality of acoustical components A big problem of today‘s networks, interconnection & terminals : delay delay – unpredictable, load dependant, time variant

7. Contributions to Quality The networks Type of networks Interconnection QoS management The endpoints Types of terminals Interoperability Terminal – Network Terminal – Terminal The users‘ location Environmental conditions Colombia, 23-24 September 2013 7 Network planning Network monitoring Network monitoring Laboratory terminal testing Laboratory terminal testing Interoperability testing Interoperability testing

8. Colombia, 23-24 September 2013 8 Outline Introduction Communication Services – Underlying System configurations Planning: The ITU-T E-model (G.107 & G.108) Network Monitoring: P.862, P.863 & P.563 Wideband – The new Challenge Summary

9. Reference Connection in the E-model Colombia, 23-24 September 2013 9

10. Transmission Rating in the E-model Rating factor R: With: R 0 – Basic signal to noise ratio (takes into account circuit noise, room noises) I x – Impairment factors (see next slide) A – Advantage factor (takes into account a potential advantage for a user for a specific transmission in a specific situation) Colombia, 23-24 September 2013 10 Basic assumption: Psychological factors on the psychological scale are additive

11. Network planning & E-model (ITU-T G.107 & G.108) Impairment factors Basis of the E-model: Impairment factors Is Simultaneous impairment factor Is (non optimum loudness rating, non optimum sidetone, PCM coding distortion) Id Delayed impairment factor Id (impact of delay, talker- or listener echo) Ie Equipment impairment factor Ie (all types of impairments in equipment such as coding distortion including the effect of packet loss) Colombia, 23-24 September 2013 11

12. How to derive Ie? Ie is based on subjective tests and can be derived as follows: Equipment impairment factors for the most popular codecs in ITU-T G.113 From subjective listening tests described in ITU-T P.833 From objective models (e.g. ITU-T P.863) following the procedure in ITU-T P. 834 Colombia, 23-24 September 2013 12

13. Some Ie factors from G.113 Colombia, 23-24 September 2013 13 Codec typeReference Operating rate [kbit/s] Ie value PCM (see Note)G.711640 ADPCMG.726, G.727402 G.721, G.726, G.727327 G.726, G.7272425 G.726, G.7271650 LD-CELPG.728167 12.820 CS-ACELPG.729810 G.729-A + VAD811 RPE-LTPGSM 06.10, full-rate1320 VSELPGSM 06.20, half-rate5.623 ACELPGSM 06.60, enhanced full rate 12.25 ACELPG.723.15.319 MP-MLQG.723.16.315

14. The E-model Prediction Range R – value range: prediction on a scale Mapping to MOS: For R  0: For 0  R  100: For R  100: Colombia, 23-24 September 2013 14

15. Guide for User Satisfaction based on R-values Colombia, 23-24 September 2013 15 Range of E-Model Rating R Speech transmission quality category User satisfaction 90  R  100 BestVery satisfied 80  R  90 HighSatisfied 70  R  80 MediumSome users dissatisfied 60  R  70 LowMany users dissatisfied 50  R  60 PoorNearly all users dissatisfied Connections with E-Model Ratings R below 50 are not recommended.

16. Guide on Transmission Planning: ITU-T G.108 Demonstrates the application of the E-model for end-to-end transmission planning in a wide range of networks Keep in mind: G.107 and G.108 are applicable for narrowband networks Colombia, 23-24 September 2013 16

17. Colombia, 23-24 September 2013 17 Outline Introduction Communication Services – Underlying System configurations Planning: The ITU-T E-model (G.107 & G.108) Network Monitoring: P.862, P.863 & P.563 Wideband – The new Challenge Summary

18. Networks & Network Monitoring Main network related impairments: Delay Delay variation (jitter) Listening speech impairment due to Coding Transcoding Interconnection Packet loss/jitter Colombia, 23-24 September 2013 18

19. ITU-T Models for Perceptual based Speech Quality Measurement ITU-T P.862 (2001): PESQ ITU-T P.863 (2011): POLQA The basic principle: Colombia, 23-24 September 2013 19 Typical Processing Steps (Schematic): Adaptation Hearing Model Comparison, Reference, Reduction, Signal Value Results of Listening Tests! S index Q Speech Signal Test Object Hearing Model processed signal reference signal

20. Principle Intrusive test procedure Objective prediction of MOS (mean opinion score) as achieved in listening test subjectively Prediction on a MOS-scale: MOS 5 – excellent MOS 4 – good MOS 3 – fair MOS 2 – poor MOS 1 – bad Substitution of subjective tests for known codecs and impairments Colombia, 23-24 September 2013 20

21. Application Intrusive testing: Insertion of a reference speech sequence Acquiring the transmitted, distorted speech sequence Calculation of the speech quality by comparing to the reference MOS LQOn – Output: MOS LQOn – mean opinion score, objective, narrowband Colombia, 23-24 September 2013 21

22. Typical setups Colombia, 23-24 September 2013 22 Central server - automated call setup -generation of test sequences - analysis & statistics Fixed Network Fixed Network Probe @ NTP - automated call setup -generation of test sequences Probe @ NTP - automated call setup -generation of test sequences Mobile Network Mobile Probe - automated call setup -generation of test sequences Mobile Probe - automated call setup -generation of test sequences Fixed Network 2 Fixed Network 2 Very difficult in monitoring: network one way transmission delay

23. Not Intrusive Testing - ITU-T P.563 Listening speech quality prediction based on speech signal in a call No reference file insertion required Average listening speech quality prediction on statistical evaluation of a high number of calls on the same connection Not recommended for test on a call by call basis Much less reliable than intrusive testing based on P.862 and P.863 Colombia, 23-24 September 2013 23

24. The wideband challenge High quality expectation by the user Wideband must be significantly better in all quality aspects Significant contribution to quality by the terminals First attemt to certify high quality wideband speech transmission: GSMA Colombia, 23-24 September 2013 24

25. Wideband listening examples Colombia, 23-24 September 2013 25 Fullband Narrowband Wideband Mobile Phone Mobile signal processing RF-Interface air interface Speech Transcoder speech noise Wideband Narrowband …in noise

26. Wideband Network Planning – ITU-T G.107.1 The principle of the model is identical to the narrowband version G.107 The model reflects the improvements in quality in wideband: Maximum R = 129 (instead of 100 for narrowband) Model provides new wideband R calculation mapping R to the range of 0-100 All factors – R 0 and Impairment factors are adapted accordingly Colombia, 23-24 September 2013 26

27. Wideband Network Monitoring Setup as in narrowband Only intrusive testing available Network monitoring is based on ITU- T P.863 POLQA ITU-T P.862 PESQ should not be used in wideband Colombia, 23-24 September 2013 27

28. Conclusions and Recommendations ITU-T provides models for planning and monitoring Network planning is essential for achieving high network QoS Network monitoring is essential to detect potential issues – e.g. changing radio conditions, changing network conditions (jitter, packet loss) Both is not sufficient to guarantee a good QoE – terminals play a major role for the overall speech quality Colombia, 23-24 September 2013 28