Introduction Prof. Khalid Al Mashouq, King Saud University 3 The world is going through mobile data revolution. Several Mobile Applications delivered on emerging smart devices is driving up the demand of Bandwidths, Quality and Connectivity. Revenues from voice traffic are relatively flat, and operators will count on new revenue streams from data services to re-coup the money they have invested.
Introduction Prof. Khalid Al Mashouq, King Saud University 4 Telecom Operators are competing to add new services and more powerful devices. They investment heavily to upgrade the capacity and enhance performance of their networks. QoS and QoE from an Operator will be key to add or lose subscribers. Authorities have important role to define, control, monitor and enforce QoS and QoE measures not only to ensure quality.
Introduction Prof. Khalid Al Mashouq, King Saud University 5 QoS – Quality of Service ITU-T Rec. E. 800 defines QoS as “collective effect of service performance which determines the degree of satisfaction of a user of the service” (Objective: Blocked calls, Dropped call, throughput, signal level) QoX – Quality of Experience ITU-T Recommendation P.10/G.100, defines Quality of Experience (QoX or QoE) as, “the overall acceptability of an application or service, as perceived subjectively by the end- user” (Subjective: QoS+ perception, customer care, invoicing,..)
Who measures QoS/QoX? Prof. Khalid Al Mashouq, King Saud University 6 Operator a) Network optimization b) Marketing/Auditor?! Authority Consumer protection groups! Public
International Standards Developing Organizations (SDOs) Prof. Khalid Al Mashouq, King Saud University 7 Besides ITU-T, other international telecommunications standardization development organizations (SDOs)exist ISO, ETSI,IETF, INTUG, ETNO, QuEST Forum, etc They deal with the general and special telecommunications quality of service issues. ITU-T organizes its work in study groups subdivided into working parties. SG12 is the lead SG in the performance, QoS and QoE. One of its affiliates is SG12 RG-AFR Most of the ITU-T Recommendations dealing with QoS exist in the E, G, I, M, Q, P, X and Y series.
International Standards Developing Organizations (SDOs) Prof. Khalid Al Mashouq, King Saud University 8 ITU-T gathers together Governments, National telecommunication administrations. Recognized operating agencies. Manufacturers. Service providers. Scientific and industrial organizations Consumer protection groups
Why QoS/QoE Prof. Khalid Al Mashouq, King Saud University 9 Voice & Data communication is facing eternal growth and quality demands in Saudi Arabia. QoS/QoE are measures introduced by Government (CITC) to control telecom operators and service providers in order to control Telecom Hazards, increase quality of service and ensure end customer experience. To define and control the development of telecommunications products and services acceptable to the industry and users/consumers To increase Inter Operability by making interconnected networks work seamlessly well together. Ensure consumers are satisfied with service and experience as per the value of money they pay.
ITU-T Four Views QoS Perspective Prof. Khalid Al Mashouq, King Saud University 10 CustomerService Provider Achieved or delivered QoS Customer received QoS Customer's QoS requirements Service Provider's offerings of QoS Nairobi, Kenya, 26 July 2010 ITU-T Recommendation E.800 has four (4) QoS view points namely: Customer's QoS requirements Service Provider's offerings of QoS (or planned/targeted QoS) QoS achieved or delivered Customer Survey ratings of received QoS The closer the 4 view points are in a given service, the better the QoS delivered.
QoS Parameters/Metrics/Indicators Prof. Khalid Al Mashouq, King Saud University 11 Are also called; metrics, indicators, measures, determinants Are premised on indicators such as: speed, accuracy, availability, reliability, security, simplicity, satisfaction Characterize the QoS level of the service being offered to the satisfaction of the user Ensure the users of a services are getting QoS levels matching what they are paying for Used in development and implementation of SLA Can be classified as; objective-measurements of physical attributes of a circuit, system, network etc or subjective- conducting of customer opinion surveys
QoS Benchmarking/Standards/Thresholds Prof. Khalid Al Mashouq, King Saud University 12 Also called; targets, objectives, benchmarks, thresholds, minimum standards, reference values etc A potential value (or range of values) to be achieved to ensure satisfaction with service delivery Used to maintain good QoS levels based on international, local or benchmarked standards Used to improve QoS levels following consumer complaints
QoS Parameters for Fixed & Mobile Networks Prof. Khalid Al Mashouq, King Saud University 13 Fixed Telephony: Service Supply Time Fault Report Rate Fault Repair Time Call Success Rate Call Set-up Time Response time for operator service (call centers) Response time for directory enquiry service Billing accuracy Billing Complaints Resolution time Call Connection Delay Ratio of working payphones Mobile Telephony Service Supply Time Fault Report Rate Fault Repair Rate Call Center Answer Time Billing and Tariff Accuracy Completed Calls Dropped Calls Blocked Calls Speech Quality (MOS, PESQ) SMS Message Transmission Success Rate Call Set-up Time Handover Success Rate Geographic Coverage Received Signal Strength (RxLv) Received Signal Quality (Rx Qual)
QoS Parameters for Data Networks Prof. Khalid Al Mashouq, King Saud University 14 Data Networks and Internet QoS parameters-cannot be isolated from the fixed or mobile telephony parameters due to convergence of technologies. IP packets are used in both fixed and mobile services. Bandwidth Latency BER Jitter and Jitter Variations Throughput Data Transmission Success Rate Internet Session Login Success Rate
How to Measure - QoS Prof. Khalid Al Mashouq, King Saud University 15 Objective- circuits attributes, networks, signals Subjective-through customer satisfaction surveys Subjective good- fits the QoS definition but time consuming Therefore a mix of the two methods recommended
QoS Benchmarking by QVOICE Prof. Khalid Al Mashouq, King Saud University 16
Prof. Khalid Al Mashouq, King Saud University 17
Prof. Khalid Al Mashouq, King Saud University 18
Prof. Khalid Al Mashouq, King Saud University 19
Samples of Benchmarking in Saudi Arabia Mobile Coverage in Riyadh some time ago! Prof. Khalid Al Mashouq, King Saud University 20
Samples of Benchmarking in Saudi Arabia Riyadh Network Coverage Prof. Khalid Al Mashouq, King Saud University 21
How to achieve QoS Prof. Khalid Al Mashouq, King Saud University 22 Enforcing Compliance Regulators monitor, measure and enforce operators’ obligations Obligations normally in licenses or regulations Enforcements sometimes entails consultations Regulator needs techniques to ensure compliance even after consultation
Enforcing Compliance Prof. Khalid Al Mashouq, King Saud University 23 Some of the enforcement methods are: Publication of QoS measurement results Reprimands Directives Imposition of Penalties Requiring operators/service providers compensate their customers Encourage Self Regulation- Service Level Agreements (SLA) Signed between service providers or between service providers and their customers Define service characteristics, responsibilities and priorities between parties May include statements on specific performance, billing, QoS levels etc May also include remedies for breach Regulator needs to enforce SLA where parties fail to do so
Subjective vs. Objective QoE Prof. Khalid Al Mashouq, King Saud University 24 Direct human QoE scoring is expensive and time-consuming ITU-T has defined objective measures that can be automated These entail algorithms that produce scores that correlate well with human QoE PSQM (ITU-T P.861) and PESQ (ITU-T P.862) are objective comparative MOS-like measures for telephone grade speech They model the human auditory perception system (Bark scale, masking, etc.) PEAQ (ITU-R BS-1387) similarly scores wideband audio These were selected in competitions to have highest correlation with human MOS ITU-T P.563 is a single-ended (absolute) objective MOS-like score It determines un-naturalness of telephone-grade speech sounds and the amount of non-speech-like noise
E-model The E-model defined in ITU-T G.107 is a planning tool It predicts a “mouth-to-ear” transmission rating factor R between 0 and 100 higher values signify better voice quality should be uniquely convertible to a MOS level R = f(QoS 1, … QoS n ) and is additive in individual QoS k degradations R starts with the basic signal to noise ratio R is reduced to account for various impairments, including simultaneous impairments (loudness, sidetone, clipping, quantization noise) delay impairments (delay, echo delay and loudness) equipment impairments (codec distortion, packet loss) R is increased when there are additional advantages such as mobility (cellphone receives A=10) R = R 0 – I s – I d – I e + A Prof. Khalid Al Mashouq, King Saud University 25
R value meanings Prof. Khalid Al Mashouq, King Saud University 26 R values meaningEquivalent MOS 90 - 100Very satisfied4.3-5.0 80- 90Satisfied4.0-4.3 70-80Some users dissatisfied3.6-4.0 60-70Many users dissatisfied3.1-3.6 50-60Nearly all users dissatisfied 2.6-3.1 Below 50 Not recommended1-2.6
VQMON Years before P.563 ETSI specified VQmon TIPHON (Telecommunications and Internet Protocol Harmonization Over Networks) TS 101 329-5 Annex E VQmon (developed by Telchemy) is a single-ended method for estimating the E-model factors for VoIP audio based on QoS parameters (packet loss statistics, delay) Depends on codec type Takes human perception phenomena into account (e.g., recency effect) VQmon was later extended to audio (MOS-A) video (MOS-V) audio-video (MOS-AV) Prof. Khalid Al Mashouq, King Saud University 27
Network planning tools In addition to subjective/objective methods to quantify the QoE of a specific (live or simulated) service instance Network planners need tools to predict service quality in order to efficiently allocate resources G.1030 provides network planners with end-to-end (E-model-like) tools for applications over IP networks It includes an appendix devoted to web browsing that presents empirical perception of users to response times and proposes a MOS measure G.1070 proposes an algorithm for network planners to estimate videophone quality Prof. Khalid Al Mashouq, King Saud University 28
Behavioral QoE All of the above subjective and objective QoE measures are service/application-specific. But new services and applications are created every day and different users use different features of a single application So it is no longer feasible to study each application in depth A new approach is behavioral QoE estimation the user’s satisfaction is estimated based on actions / reactions Example : there is a high measured correlation between a user being unsatisfied with a service level his aborting the application (or at least waiting until the service level improves) Behavioral QoE can be used instead of traditional QoE measurement or to automatically find QoE(new app, QoS) Prof. Khalid Al Mashouq, King Saud University 29
There are also solutions to inject Probes or Robots in the network Prof. Khalid Al Mashouq, King Saud University 30
Prof. Khalid Al Mashouq, King Saud University 31
QoE Measurements by using probes/network robots Prof. Khalid Al Mashouq, King Saud University 32 Mobile Monitoring Robot Video Monitoring Robot Source: Witbe
Mobile Monitoring Robot Prof. Khalid Al Mashouq, King Saud University 33 The Mobile Monitoring Robot is a network appliance dedicated to monitoring mobile network connections and services. Deployed at strategic locations on your network, it gives you a comprehensive user perspective on the quality of your mobile services over 2G, 3G and 4G- LTE networks. Not only does the Mobile Robot test network availability and performance, it interacts with your services like real users: Placing and receiving calls, sending and receiving SMS/MMS, browsing the Web, streaming video content, etc. You now have detailed information to monitor your mobile services and optimize your infrastructure to provide the best user-experience to your subscribers.
Video Monitoring Robot Prof. Khalid Al Mashouq, King Saud University 34 The Video Monitoring Robot connects to set-top boxes, iOS/Android devices, Web browsers, PC/Mac players, game consoles or connected TVs to monitor live and on-demand video services according to test scenarios emulating end-users’ behaviors. It’s the only solution on the market providing an objective, reliable and accurate view of the actual user experience, post device. In addition to technical metrics, it gives you real-time information on the usability of your interactive services and quality of your content. Video Robots are ideal to automate testing of a new service, firmware or application before launch. They are also designed for 24/7 monitoring of your IPTV, VOD, OTT services on any device.
QoX and our approach We develop a software for the MS (downloaded OTA) to give more accurate assessment of quality of experience from the user perspective The collected information is then relayed to the operator (SMS, MMS, etc..) The operator (quality control or optimization) will have access to valuable information that might be difficult or even impossible to get. Improvement can be done based on this information along with network monitoring data Customer will feel special attendance and care from operator This will significantly boost QoX
Plotting Landmarks & Landmarks snapshots The landmarks that were marked with red colors are the calls dropped due to handover and the landmarks that were marked with green colors are normally dropped calls. The different colors landmarks help one to easily visualize and analyze the calls.
Call Quality Parameters Successful call attempts Un-Successful call attempts The successful and un-successful call attempts are classified based on whether the call is successfully connected by the network. Normally Dropped Handover Dropped Normally dropped from either of the party or dropped due to handover during the cell change is recorded.
Call Quality Parameters(example) The Average Call Quality is computed as (successful call attempts rate score+ Normal dropped rate score +Average signal strength of all successful calls)/3
Prepared Exclusively for “Mobily” March 2009 Call Bundle Statistics
Advanced Results Prof. Khalid Al Mashouq, King Saud University 44 Currently we are working on Estimating PSEQ which is ITU standard for objective speech quality which is highly correlated with MOS Developing our software on other platforms like iPhone
Estimating PSEQ from Performance parameters Prepared Exclusively for “Mobily” December 2010
Ideas to Improve QoS/QoX Prof. Khalid Al Mashouq, King Saud University 46 Operator voluntarily asks a group of the user to report their QoX (automated or manually) and present tis information to the public Create compensation mechanism for bad QoS Make an original study to solve the problem of weak penetration inside big houses Rethink of Femto cell solution Make good planning for new mega projects and also new cities/compounds
Conclusion Prof. Khalid Al Mashouq, King Saud University 47 QoS is the monitoring of the discreet infrastructure components QoE is measuring of end customer/subscriber experience and satisfaction. There are Different ways, technologies and products already available in market for QoS and QoE. Governments to play a pivotal role in encouraging service providers to comply certain minimum QoS and QoE standards
Thank You Prof. Khalid Al Mashouq, King Saud University 48