1. ITU workshop Quality of Service and Quality of Experience of Multimedia Applications and Services Haarlem/Amsterdam, 9-11 May 2016

2. CEM Index for top services in operators’ network to score customer experience in terms of basic performance metrics Mehmet Özdem Quality Assurance Senior Manager, Türk Telekom mehmet.ozdem@turktelekom.com.tr

3. || Agenda  01Türk Telekom Highlights  02Brief Intro to CEM & SQM  03 CEM Works in QSDG  04Idea of CEM Index  05AHP Algorithm applied for CEM Index  06Conclusion Türk Telekom 3

4. | Türk Telekom Highlights 4 Türk Telekom unified mobile, fixed voice, fixed broadband and TV brands under single ‘’Türk Telekom’’ brand.

5. | According to the Tele-management Forum Technical Report TR 148, Customer Experience is defined as “the result of the sum of observations, perceptions, thoughts and feelings arising from interactions and relationships (direct and indirect) over an interval of time between a customer and their provider(s) when using a service”. According to Tele-management Forum Technical Report TR 148 and TR 149, Service Quality Management is defined as the set of features displayed by an operation support system (OSS) that allow the quality management of the different products and services offered by an enterprise/company. Brief Intro to CEM & SQM Türk Telekom6

6. | Supplement E.SUPPL.FTSO: “QoS/QoE Framework for the transition from network oriented to service oriented operations. (TD 840 Rev.1) Identifies key parameters relating to service quality management (SQM), Quality of Experience (QoE) and Performance for supporting operations within service operations centre environment. Key parameters include technical and non-technical parameters. CEM Works in QSDG

7. |  It is proposed to study on creation of a CEM Index which can generate a score for customer experience of top services. This index can have weights for the services by using AHP (Analytic Hierarchy Process) algorithm.  The Document (TD 840 Rev1) can be updated with the most popular services Streaming Media Online Gaming VoIP Web Browser  Each service will get points according to performance in terms of latency, jitter, packet loss, throughput by using AHP algorithm. Our Proposal Türk Telekom13

8. |  For most of service providers, Streaming media, Web Browsing, VoIP and Online Gaming are top services that customers interact in a high volume.  For technical part, CEM Index can provide a score for customer experience of top services. CEM Index can be calculated service, region and customer based.  Packet loss, throughput, latency and jitter are the most important basic technical parameters that indicates quality of service and affects quality of experience. Idea of CEM Index Türk Telekom7

9. | Idea of CEM Index Türk Telekom8  This index will have weight for Streaming Media Service, Web Browsing Service, Voice over IP Service and Online Gaming.  These weights will be calculated through AHP( Analytic Hierarchy Process) algorithm by considering effect of packet loss, throughput, latency and jitter on each service seperately.  Each service will get points according to performance in terms of latency, jitter, packet loss and throughput.  Assigning point will be determined by observing whether service is performing below or above the performance target assigned for latency, jitter, packet loss and throughput for each service. CEM Index Streaming Media VoIP Online Gaming Web Browsing Packet Loss Latency Throughput Jitter

10. |  Analytic Hierarchy Process (AHP) was developed by Professor Thomas Saaty to being used to decide among multiple alternatives.  AHP provides decision-makers choose the best solution  Project prioritization, vendor selection, technology selection, site selection, recruitment…etc  By AHP mechanism, the decision is taken after a few steps[2]:  pairwise comparisons of alternatives by evaluating their importance relative to each other,  transformation of comparisons into weights  check of consistency.  For pairwise comparisons, relative importance of alternatives will be converted to numbers in the scale below. AHP Algorithm Türk Telekom9

11. |  For service alternatives and performance metrics, pairwise comparisons are can be seen in below tables, these table can be read like that “Streaming Media Service is strongly more important than VoIP Service” or “ Web Browsing is weakly more important than VoIP service.” AHP Algorithm applied for CEM Index Türk Telekom10 For Streaming Media Index For VoIP Index CEM Index

12. |  These pairwise comparison ratings are turned into weights by summing columns, dividing each number in each column by column sum and taking the average of each row.  According to this calculation, weight of services and metrics are like below. Also selection of 1 or 0 for service points calculation is determined whether the service is performing below or above the target of related metric. Türk Telekom11 For Online Gaming Index AHP Algorithm applied for CEM Index For Web Browsing Index

13. | AHP Algorithm applied for CEM Index 1212 Türk Telekom CEM Index

14. |  CEM is a key area of interest.  Less of standart  TD 840 Rev 1 can be updated by  CEM Index by using AHP  New popular services  Technical Part  Latency, jitter, packet loss, throughput Conclusion Türk Telekom13

15. || Thank you! Türk Telekom 15