1. Benchmarked Key Performance Indicators
ITU-T Workshop on Delivering Good Quality Telecommunication Service in a Safe Environment in Africa (Nairobi, Kenya, 26th July 2010)
Benchmarked Key Performance Indicators
Crisphine J. Ogongo,
Engineer/Compliance
Communications Commission of Kenya
Nairobi - Kenya July 2010

2. QoS Regulation And Comparison With Int’l Stds Conclusion Questions
Content
ITU/ETSI Standards
QoS Regulation And Comparison With Int’l Stds
Conclusion
Questions
Nairobi - Kenya July 2010

3. ITU-T/ETSI STANDARDS
Nairobi - Kenya July 2010

4. Quality of service (QoS):
Definition
Quality of service (QoS):
The collective effect of service performances, which determine the degree of satisfaction of a user of the service (ITU-T E.800)
Nairobi - Kenya July 2010

5. What QoS Really Is! Network Performance Parameters translated to;
system design, configuration, operation and maintenance
Quality of service parameter influenced by;
Statistics e.g «Call Block Rate»
User/customer requirements
Individual experience e.g «inaccessibility»-
User opinion/requirements feed back in to the network planning process to alter planned performance and/or practical operational standards
Nairobi - Kenya July 2010

6. ITU-T QOS Parameters with Targets
Call Set Up Time/Post Dialing Delay
Call Release Delay
Answer Signal Delay
End to End Blocking
Handover Success Rate/Unsuccessful Handover
Speech Quality
Multimedia QoS
Nairobi - Kenya July 2010

7. Call Set up time Call type F-M M-F M-M Definition
Authentication/ ciphering
0.0
2.5
Time interval between the end of dialing by the user and the reception by him of the appropriate tone or recorded announcement, or the abandon of the call without tone.
Paging/alerting
4.0
Routing number transfer
2.0
Post-selection delay
Local connection
3.0
Toll connection
5.0
International connection
8.0
TOTAL
PLMN
PSTN
Normal Load
High Load
Mean
95% 9
5.5
11.5
3 s
6 s
4.5 s
9 s 11
7.5
13.5 5s
8 s
7.5 s
12 s 14
10.5
16.5
11 s
16.5 s
Nairobi - Kenya July 2010

8. Answer Signal Delay PLMN Connection Definition Call type F-M M-F M-M
Time interval between the establishment of a connection between calling and called users, and the detection of an answer
signal at the originating exchange.
Local connection
1.0
1.25
Toll connection
1.75
2.0
International connection
2.75
3.0
PSTN
Normal Load
High Load
Mean
95%
0.75 sec
1.5 sec
1.0 sec
2.0 sec
3.0 sec
4.0 sec
5.0 sec
3.3 sec
6.5 sec
Nairobi - Kenya July 2010

9. Call release delay (secs)
Connection Type
Call type
F-M, M-F, M-M
PSTN
Calling, or called, party clears
1.0 sec
0.4 – 1sec
Call release delay is defined as the time interval from the instant the first bit of the DISCONNECT message is passed by the user terminal which terminated the call to the access signaling system, until the last bit of the RELEASE message is received by the same terminal (indicating that the terminals can initiate/receive a new call).
Definition
Nairobi - Kenya July 2010

10. End to End Blocking
Definition: - The probability that any call attempt will be unsuccessful due to a lack of network resources
1990
Improvement
PLMN
On radio Channel
5-10% 1%
PLMN to Fixed
0.5%
Normal Load
High Load
PSTN
Local 2% 3%
Toll Connection
4.5%
Int’l Connection 5%
7.5%
Nairobi - Kenya July 2010

11. Unsuccessful Handovers
Is the probability that a handover attempt fails because of lack of radio resources in the target cell, or because of a lack of free resources for establishing the new network connection. The failure condition is based either on a specified time interval since the handover request was first issued or on a threshold on signal strength
Call type
F-M, M-F, M-M
Probability of unsuccessful land cellular handover
0.5%
Nairobi - Kenya July 2010

12. Speech Quality
PESQ: - Perceptual evaluation of speech quality ITU-T P.862
An objective method for end-to-end speech quality assessment of narrow-band telephone networks and speech codecs
PESQ compares an original signal X(t) with a degraded signal Y(t) that is the result of passing X(t) through a communications system. The output of PESQ is a prediction of the perceived quality that would be given to Y(t) by subjects in a subjective listening test(6).
Provides Speech Quality raw values -0.5 to 4.5
Nairobi - Kenya July 2010

13. perceived Speech quality
PESQ: - Perceptual evaluation of speech quality ITU-T P.862
An objective method for end-to-end speech quality assessment of narrow-band telephone networks and speech codecs
Original Signal X (t)
Communication Network
Y(t)
PESQ
Algorithm
perceived Speech quality
Original Signal X(t)
Subjects in Listening Test
Perceived Speech quality
Original Signal X(t)
Nairobi - Kenya July 2010

14. Speech Quality cont’d
ITU-T P Mapping function for transforming P.862 raw result scores to MOS
The mapping ensures a domain rescaling from –0.5 ... 4.5 to 1.02 ... 4.56
Nairobi - Kenya July 2010

15. Speech quality depends on Transmission Rating Factor R
Speech Quality Cont’d
Speech quality depends on Transmission Rating Factor R
The R‑value is a measure of a quality perception to be expected by the average user when communicating via the connection under consideration
Nairobi - Kenya July 2010

16. Transmission Rating Factor conversion to MOS equivalent
Speech Quality Cont’d
Transmission Rating Factor conversion to MOS equivalent
R-value range
Equivalent MOS Values
Speech transmissio n quality category
User satisfaction
90  R < 100 5
Excellent
Very satisfied
80  R < 90 4
Good
Satisfied
70  R < 80 3
Fair
Some users dissatisfied
60  R < 70 2
Bad
Many users dissatisfied
50  R < 60 1
Poor
Nearly all users dissatisfied
Conversion of R‑values into MOS
MOS = MOS = R + R(R-60)(100-R)7.10⁻⁶
Nairobi - Kenya July 2010

17. MOS User Satisfaction Levels
Typical values used in
R-Value
MOS Value
Comment
Country
R ≤ 0 1
Nearly all users dissatisfied
R = 55
2.835
Many users dissatisfied
Kenya
R = 60
3.1
Some users dissatisfied
Nigeria/Kenya in 3yrs
R = 70
3.597
R = 80
4.024
Satisfied
R ≥ 100
4.5
Very satisfied Satisfied
MOS = MOS = R + R(R-60)(100-R)7.10⁻⁶
Nairobi - Kenya July 2010

18. Data/Multimedia Parameters Critical to Users
Delay variations
Occurs at the transport layer in packetized data systems
due to the inherent variable arrival times of individual packets
Solved through buffering
Delay
the time taken to establish a particular service from the initial user request and the time to receive specific information once the service is established.
Packet Loss
Of packets or bits during transmission
Includes coding degradation
Nairobi - Kenya July 2010

19. Multimedia (Audio and Video) QoS Parameters and targets
KPI
1-way delay
Delay Variation
Information Loss (Packet Loss Ratio-PLR)
Audio Conversational Voice
<150 ms
< 1 ms
< 3%
Audio Voice Messaging
< 1 s playback
High Quality Streaming Audio
< 10 s
<< 1 ms
< 1%
Videophone (2-way)
< 150 ms
Videophone (1-way)
Nairobi - Kenya July 2010

20. Multimedia QoS Parameters
KPI
Symmetry
T-Values
Delay
Web-browsing
– HTML
1-way
~10 KB
P< 2 s /page
A < 4 s /page
No delay variation or information Loss
Key
P-preferred
A-Acceptable
T-typical
Bulk data transfer/retrieval
10 KB-10 MB
P < 15 s
A < 60 s
Transaction services –
high priority e.g. e‑commerce
2-way
< 10 KB
P < 2 s
A< 4 s
Command/control
~ 1 KB
< 250 ms
Still image
< 100 KB
Interactive games
< 1 KB
< 200 ms
Telnet
(server access)
P< 2 s
(server to server transfer)
Can be several
minutes
Nairobi - Kenya July 2010

21. QOS REGULATION IN AFRICA AND COMPARISON WITH INTERNATIONAL STANDARDS
Case studies
Kenya
Nigeria
Uganda
Tanzania
Nairobi - Kenya July 2010

22. Status of Country QoS Regulation
Nigeria
Tanzania
Uganda
Kenya
QoS regulated √
QoS Parameters and Targets
Regulations x
Guidelines X
Licenses
Objective parameters
Subjective Parameters
PSTN Network
Cellular Mobile Network
Internet
Data (Leased Lines)
Nairobi - Kenya July 2010

23. Objective KPIs and Targets Benchmarks
ITU Targets
Kenyan Targets
Nigeria
Tanzania
Uganda
Completed Call (%) - 90
>99 99
Speech Quality (MOS)
%age with Good SQ
4.0 ≤ MOS ≥ 4.5
95%>2.7 & 3.1 in 3 yrs
98 >2.0 MOS
>95 95
Call Drop Rate (%) 2
<3
Call Block Rate (%) 1 10
Call Set Up Time (s)
varied
13.5
< 10
Call Set Up Success Rate (%)
Handover Success Rate (%)
99.5% 85
Call release delay 1s 2s
Multimedia (IP) QoS
Varied
ITU
Nairobi - Kenya July 2010

24. Subjective QoS parameters
Account complaint rate
Account complaint resolution time
Disconnection complaint rate
Disconnection complaint resolution time
 Miscellaneous complaint rate
 Miscellaneous complaint resolution time
 Fault report rate
 Fault repair time
Billing Accuracy
 Service supply time
 Call centre answer success ratio
 Call centre answer time
Complaint resolution
Nairobi - Kenya July 2010

25. Conclusion
ITU-T recommendations have covered many QoS issues but very few targets have been specified
Where ITU-T targets are specified, they should be held as minimum thresholds not negotiable downwards
Many regulators monitor QoS parameters
Targets adopted by regulators vary widely
Nairobi - Kenya July 2010

26. Questions How many regulators monitor QoS performance of providers?
What are the KPI’s monitored?
How comprehensive are KPIs in addressing customers requirements?
How do the targets compare with international standards or best practices?
Is there need for African Regulators to benchmark and adopt similar and adequate parameters and targets?
Nairobi - Kenya July 2010

27. Thank You
Nairobi - Kenya July 2010