1. Sources of Failure in the Public Switched Telephone NetworkBY
RAGHU SUNDEEP.T
EE 585: Fault Tolerant
Raghu Sundeep: Sources Of Failure In PSTN

2. DESCRIPTION
Like all telephone switching networks, the PSTN performs a fairly simple task: It connects point A with point B.
The PSTN contains thousands of switches
Switches include redundant hardware and extensive self-checking and recovery software
The PSTN is a large, complex, distributed system with strong dependability guarantees.
Raghu Sundeep: Sources Of Failure In PSTN

3. A Layout Of PSTN DLSU  (Digital Local Switching Unit)
                                                                   
DLSU  (Digital Local Switching Unit)
DLE  (Digital Local Exchange)
DCCE  (Digital Cell Centre Exchange)
DMSU's  (Digital Main Switching Unit)
DLSU  (Digital Local Switching Unit)
DLE  (Digital Local Exchange)
DCCE  (Digital Cell Centre Exchange)
DMSU's  (Digital Main Switching Unit)
Raghu Sundeep: Sources Of Failure In PSTN

4. General Phone Connection
                                                                   
The -48V DC power supply is derived from the mains and is backed up by a massive set of batteries to ensure power is available for the PSTN should the mains fail at the exchange
Master Socket has three components - the Spark Gap (SG) which protects your phone should the exchange develop a problem and produce a line pair voltage greater than about 90V. It also protects the wiring from your phone to the exchange should your phone develop a fault and try to inject mains voltage (for those who have mains powered phones) down the line pair
Raghu Sundeep: Sources Of Failure In PSTN

5. Failure Analysis of the PSTN: 2000 Approach
Find the areas that are failing then try to fix/address the problems
Use PSTN as a case study for ROC:
Large, widely used, networked system
Highly reliable infrastructure
Provides an upper limit for reliable computer service
BEST CASE
Raghu Sundeep: Sources Of Failure In PSTN

6. Collecting Failure Data
Target System: US Public Switched Telephone Network (PSTN)
Detailed telephone service failure data available from the Federal Communications Commission (FCC)
Telephone Disruption reports: company name, duration, time, cause, and event disruption
Required by law for outages affecting 30,000 people or lasting at least 30 minutes
Raghu Sundeep: Sources Of Failure In PSTN

7. Causes of Failure Human Error Acts of Nature Hardware Failure
Software Failure
Call overloads
Vandalism
Raghu Sundeep: Sources Of Failure In PSTN

8. Categorizing the Failures
Human Error
Company workers
Includes Contractors
and Vendors
External
Acts of Nature
Fire
Rain
Lightning
Winds
Floods
Raghu Sundeep: Sources Of Failure In PSTN

9. Categorizing the Failures
Hardware Failure
Network component failure
Cable, power outage
Software Failure
corrupt/incorrect communication software
Call Overloads
Over network capacity
Vandalism
Intentional harm to telephone network equipment
Raghu Sundeep: Sources Of Failure In PSTN

10. Categorization Challenges
Outages may have multiple causes
Terminology
Root Cause - cause behind the outage
Direct Cause - immediate trigger
i.e.
Root Cause – latent error in software
Direct Cause – Maintenance error (human)
Raghu Sundeep: Sources Of Failure In PSTN

11. Outages Breakdown by Number:
Human Error accounts for 55% of the outages for 2000
Human-company
Human-external
Hardware Failure
Software Failure
Overload
Vandalism
Acts of Nature
Total: 202 outages
Raghu Sundeep: Sources Of Failure In PSTN
*Vandalism accounts for < 1%

12. Outage Breakdown by Number (Nature Factored Out)
Human Error accounts for 59% of all Outages
Human-company
Human-external
Hardware Failure
Software Failure
Overload
Vandalism
Raghu Sundeep: Sources Of Failure In PSTN
Total: 187 outages

13. What could humans possibly do wrong?
Cut incorrect cables
Upgrade software incorrectly
Incorrectly repair hardware
Follow instructions incorrectly
Fail to read documentation
Do things out of order
Raghu Sundeep: Sources Of Failure In PSTN

14. Measuring Availability
Number of Outages
Only measures the number of outages.
Does not include the duration of the outages.
There’s more important information than simply the number of outages.
Outage Duration
Customers Affected
Blocked Calls
Raghu Sundeep: Sources Of Failure In PSTN

15. A Second Metric Customer Minutes
Outage duration in Minutes * Customers affected
Captures collective customer experience
Assumes all affected customers or lines attempted to make a call
Raghu Sundeep: Sources Of Failure In PSTN

16. Number of Outages vs. Customer Minutes customer minutes/year
Humans = 54%
Total: about 95 Million
customer minutes/year
Raghu Sundeep: Sources Of Failure In PSTN
Total: 187 outages

17. Summary: Humans were the greatest cause of failure.
Humans caused most of the outages
“Traditional Computing concentrates on tolerating hardware and operating system faults, ignoring faults by human operators…”
(David Patterson, 2001)
Raghu Sundeep: Sources Of Failure In PSTN

18. Trends in Customer Minutes
Cause
Trend
2000
Human Error: Company 98
131
Human Error: external
100
125
Hardware 49 60
Software 15
155
Overload
314 2
Vandalism 5
Raghu Sundeep: Sources Of Failure In PSTN

19. Future Work: Directly apply data to the ROC project
Could the ROC techniques have avoided these outages?
Further categorize the data
More specific categories within each general category
Telephone Company
Geographic location
Breakdown Human error further
Vendors, contractors, technicians, outsiders…
Include more years of outages for further comparison
Raghu Sundeep: Sources Of Failure In PSTN

20. REFERENCES From IEEE Computer, Vol. 30, No. 4 (April, 1997).
Kühn, DR: Sources of Failure in the Public Switched Telephone Network, IEEE Computer (April 1997)
DATA available from FCC
Raghu Sundeep: Sources Of Failure In PSTN