1. CEG 2400 FALL 2012 Chapter 13 Troubleshooting Network Problems 1

2. Approach to Troubleshooting Proceed slowly, logically and methodically Use experience when necessary Benefits of a logical approach –Prevents wasteful, time-consuming efforts –Unnecessary software purchases –Unnecessary hardware replacements 2

3. Troubleshooting Steps Troubleshooting steps 1.Identify problem Gather information Identify symptoms Question users Determine if anything has changed 2.Establish theory Always question the obvious 3

4. Troubleshooting Steps Troubleshooting steps (cont’d.) 3.Test theory to determine cause If theory confirmed, determine next steps If not, establish new theory or escalate Test must not be destructive 4.Establish action plan 5.Implement solution or escalate 6.Verify full functionality 7.Implement preventative measures if applicable 8.Document findings, actions, outcomes 4

5. 1. Identify the Problem and Its Symptoms Ask questions Avoid jumping to conclusions Pay attention to: –Users –System and network behaviors –Error messages Treat each symptom uniquely 5

6. Identify the Problem and Its Symptoms Determine the problem’s scope –Identify affected problem area Find out how many users or network segments are affected –Narrow down the time frame during which problem occurred Discover time and frequency of problem –May reveal more subtle network problems Benefits of narrowing scope –Eliminate causes, may point to other problems Take your time to troubleshoot correctly 6

7. Identify the Problem and Its Symptoms Question users –Ensure human error is not source of problem –Watch the user if possible Use remote desktop software if you can’t be there to watch Watch each step user takes Determine if anything has changed –Be aware of recent network changes Possible actions if network change created problem –Correct problem –Reverse change Have/make network change records available to staff members 7

8. 2. Establish a Theory Re-create the symptoms if can be done safely –Follow same steps as person reporting symptom Verify Physical layer connectivity –Cabling and network adapters a common source of problems –Diagnosing Physical layer problems Ask questions Verify connections between devices Verify soundness of connection hardware Check cable 8

9. Establish a Theory Verify logical connectivity –Consider error messages –Note changes in operating system or applications –Determine if the problem is repeatable –Verify firmware and software configurations of devices in path –Try Ping utility 9

10. 3. Test the Theory to Determine Cause Test Physical layer theories Example approaches –Use a cable testing tool –Check to make sure NIC is seated firmly in slot –Use wireless analyzer –Follow cables to verify physical connectivity –Try different port, data jack –Connect/use another device 10

11. Test the Theory to Determine Cause Test logical connectivity theories Example approaches –View switch configuration to determine which nodes are included in VLANs –Investigate user permissions –Examine NIC configuration (dhcp, ip address, etc) –Ensure routing table includes valid entries –Use tools such as ping, netstat, route, traceroute –Check wireless client settings 11

12. Test the Theory to Determine Cause Escalate if necessary –Ask colleague with more experience or knowledge Help desk analysts/ first-level support –Proficient in basic workstation, network troubleshooting Network specialist / Second-level support Network coordinator/ Third-level support Follow escalation procedures 12

13. 4. Establish a Plan of Action to Resolve the Problem Consider how the solution affects users/network functionality Scope –Assess solution’s scope before implementing Trade-offs –Solution may restore functionality for one user group but may remove functionality for others Security –Be aware of security implications Understand access changes for authorized, unauthorized users 13

14. Establish a Plan of Action to Resolve the Problem Scalability of solution –Position network for future additions, enhancements –Try not to use temporary fix organization will outgrow Cost –Weigh options carefully Use vendor information –Manufacturer documentation –Free online troubleshooting information (phone support/web support) –Consult with others within, outside your organization 14

15. 5. Implement the Solution or Escalate as Necessary Solution implementation –Time required may be minimal or long –Use methodical and logical approach –Problem causing catastrophic outages should be solved as quickly as possible –Follow series of steps Implement a safe, reliable solution Large-scale fixes –Roll out changes in stages 15

16. 6. Verify Full System Functionality Verify problem solution is complete –Type of testing depends on solution –Type of testing depends on area –May not be able to test solution immediately Determine how and why solution worked Evaluate effects on users (no negative consequences) 16

17. 7. Implement preventative measures if applicable Consider how similar problems may be prevented in the future –Increased maintenance –Thoughtful planning –Perform network health checks –Upgrading network Good planning Good design 17

18. 8. Document Findings, Actions, and Outcomes Necessary to record: –Problem symptoms and causes and solutions Justification for recording –Impossible to remember each incident’s circumstances –People leave/Job changes Use centrally located database accessible to all networking personnel 18

19. Document Findings, Actions, and Outcomes Have a good call tracking system (help desk software) Supported services list document –Contact personnel list both local and software companies –Lists all supported hardware, services, and software Follow-up with user who reported problem 19

20. Document Findings, Actions, and Outcomes Notify others in organization of changes –Record resolution in call tracking system –Alert others about problem, solution, and network changes made Have a change management system –Means of documenting network changes 20

21. Troubleshooting Tools Utilities help troubleshoot network problems –Ping –Specialized tools Simple continuity testers Protocol analyzers Tool selection dependencies –Problem being investigated –Network characteristics 21

22. Tone Generator and Tone Locator Probe kit –Generator and locator combination Tone generator (toner) –Small electronic device that issues signal on wire pair Tone locator (probe) –Emits tone when electrical activity detected Testing requires trial and error Used to determine where wire pair terminates Not used to determine cable characteristics 22

23. 23 Use of a tone generator and tone locator

24. Multimeter Specialized tool used to test cables for faults –Isolate problems with network cables –Measures electric circuit characteristics Multimeter –Single instrument for measuring impedance, resistance, and voltage on a wire –Has several uses –Sophistication, features, and costs vary 24

25. 25 A multimeter

26. Cable Continuity Testers Cable checkers (continuity testers, cable testers) –Tests whether cable carries signal to destination Series of lights, audible tone –Used to signal pass/fail Some continuity testers verify UTP, STP wires paired correctly –Not shorted, exposed, crossed Fiber optic continuity tester –Issues light pulses on fiber –Determines whether pulses reach other end 26

27. 27 Cable continuity tester

28. Cable Performance Testers Determines if cable carrying current Continuity testers versus performance testers –Differ in sophistication and price –Performance tester accomplishes same tests Can also perform additional tasks Issue signal, measures signal bounce back Indicates distance between nodes/ measure length Indicates whether terminators properly installed, functional, determine faulty splice locations, breaks, connectors, bends Expensive 28

29. 29 A high-end cable performance tester

30. Butt Set Lineman’s handset, telephone test set Uses –Determine if line functioning (detects dial tone) –Receives signal –Picks up noise affecting signal 30

31. 31 Butt set

32. Network Monitors Software-based tool –Continually monitors network traffic –Interprets up to Layer 3 –Determines protocols passed by each frame –Cannot interpret frame data Tools developed by other software companies –Purchase or free download Network adapter –Must support promiscuous mode 32

33. Protocol Analyzers Protocol analyzer (network analyzer) –Captures traffic, analyzes frames Typically to Layer 7 Same features as network monitor but includes additional features –Generates traffic to reproduce network problem –Can collect more information than can be reasonably processed –Set filters on gathered data 33

34. Protocol Analyzers Before using network monitor or protocol analyzer: –Learn what network traffic normally looks like –Capture data for time period on regular basis Establish a baseline to compare with future analyses Sniffer (packet sniffer) –Older term referring to hardware device 34

35. 35 Traffic captured by a protocol analyzer

36. Wireless Network Testers Tools containing wireless NICs, running wireless protocols A workstation wireless network connection –Shows little about wireless environment –Only applies to one workstation Programs that scan for wireless signals –Both software and hardware based –Discover access points, wireless stations transmitting 36

37. Wireless Network Testers (cont’d.) Hardware instruments for wireless network testing –Typically more portable than workstation with software tools –Preinstalled with network analysis tools –Accessible from simple, graphical interface –Contain powerful antennas Spectrum analyzer tool –Tool to assess wireless signal quality 37

38. 38 Wireless network testing tool

39. Network Monitors/Protocol analyzer Network Monitor software –Microsoft’s Network Monitor –Nagios –Ganglia –Plus there are commercial software also Variety of protocol analyzer software available –Wireshark –Ethereal –SmartSniff –Plus there are commercial software also 39

40. Summary Methodical, logical troubleshooting methodology –8 troubleshooting steps Troubleshooting tools –Tone generator, tone locator –Multimeters –Cable continuity testers –Cable performance tester –Butt set –Network monitors and protocol analyzers –Wireless network testing tools 40

41. End of Chapter 13 Questions 41