1. Advanced Troubleshooting with Debug Diagnostics on IIS 6 Draft 2.5 5/13/06 NameTitleGroup Microsoft Corporation

2. Agenda Introduction A look at Debug Diagnostics Dig into crash scenarios Understanding hangs Investigating memory leaks Session Summary

3. Introduction Overview of Debugging: Past and Present Integral tools for resolving Debug scenarios Variables impacting Debugging

4. Overview of Debugging Past and Present Past: offered many tools, fewer resolutions Previous tools were often not server-ready No single tool to address different debugging scenarios Required expertise in debugging Present: more advanced tools, still time- consuming to solve Still lacks any analytical piece Existing tools are not “integrated” into single tool Do you really want to learn “debugging?”

5. Talking about tools… There are several tools designed at resolving the same problems As Windows Servers have grown, so have the debugging tools Example of debugging tools Exception Monitor ADPlus.vbs (i.e. – cdb.exe) IIS Debug Toolkit 1.1 IIS State

6. Variables Impacting Debugging (2) IIS’s Extensibility Examples ISAPIASP.NET Active Server Pages (ASP) COM components PHP (ISAPI) Perl (CGI or ISAPI) Oracle\MySQL These “extensible” parts are often integrated and executing in same process Ever felt like this?

7. Variables Impacting Debugging (3) Web applications often have high dependency factors Multiple technologies “chewing” on the same memory space Inexperienced server side programmers W3WP.exe ISAPI Extensions ISAPI Filters Not to mention CGI

8. Debug Diagnostics 1.0 “Core” pieces of DebugDiag Understanding “Rules” Using DebugDiag’s User Interface

9. Core pieces of Debug Diag 3 pieces make up Debug Diag 1.0 Service Host (also called Controller) User Interface DbgSvc.exe DbgSvc.exe DebugDiag.exe

10. Core pieces of Debug Diag Service is multi- purposed Interacts with the Debugger Host Allows: Service required to attach invasively to process To run from Terminal Services The Service – DbgSvc.exe DbgSvc.exe To register the service: C:\DebugDiag> DbgSvc /service To start the service: C:\DebugDiag> net start DbgSvc To stop the service C:\DebugDiag> net stop DbgSvc Unregister the service C:\DebugDiag> DbgSvc /unregserver

11. Core pieces of Debug Diag Host Where the DbgEng.dll instance is loaded Exposes 3 classes The Host – DbgHost.exe DbgControl: Attach/Detach from processes or open/analyze a memory dump DbgObj: Collect process or memory dump information Manager: Mainly to output analysis data to the report file DbgSvc.exe DbgHost.exe Attac h

12. Core pieces of Debug Diag User Interface designed to simplify: Rule creation View processess Initiate analysis of memory dumps Add analysis scripts Change properties for Debug Diag The User Interface – DebugDiag.exe DebugDiag.exe DbgHost.exe ControlScripts

13. Crash Scenarios Background of crashes Using DebugDiag to quickly identify root cause of crashes

14. Background: Exceptions What is an exception? How do applications cause exceptions What is a “handled” exception? Unhandled exceptions need debugging and resolution Access Violation: C000 0005 Breakpoint Exception: 8000 0003 Stack Overflow: C000 00FD C++ exception: E06D7363 Managed exception: E0434F4D E0434F4D Invalid Handle: C000 0008

15. Background: Heap (2) What is a heap? Heap is a data structure Heap Corruption: Certain rules must be followed when using the heap When rules are violated, heap corruption occurs Heap corruption crashes Code that corrupts heap is usually not the ones impacted

16. Debugging Crashes with Debug Diagnostics: Crash Rule Name Title Group

17. Hang Scenarios Investigating the theory of “process hangs” Using DebugDiag to debug “process hangs”

18. Theory of “Process Hangs” What is a process? Memory boundary for execution of code Allocated by Operating System done by calling CreateProcess or CreateProcessAsUser API Processes do not execute code Contains a minimum of 1 thread What is a thread? An independent sequence of instructions executing within a process

19. Theory of “Process Hangs” Thread implications on applications The level of concurrency (virtual) is dictated by the number of threads Real concurrency is dictated by the number of execution units 1 processor = one or more execution units What are common thread pools for IIS? ATQ thread pool ASP thread pool ASP.NET thread pool

20. Runaway Threads A runaway thread is a thread stuck in a infinite (or near) loop Performing a task that is consuming CPU Either by design or because of error conditions Runaway thread is taking CPU resources away from other threads Often takes away from good threads Causes performance degradation for server

21. SQL Thread4 Thread3 Thread1 Defining a Blocked Thread A thread that isn’t using any CPU resources Waiting on database calls to return Waiting on web service calls to return Waiting on a socket/network call to return Waiting on a lock IIS

22. Locks and Deadlocks What is a lock? A synchronization mechanism to protect resources from being corrupted due to simultaneous access by multiple threads Why do developers use locks? Protecting data Protecting other resources that are not memory based Impact on applications if locks are poorly implemented Causes threads to block Can cause deadlock situations when using multiple locks in the application Process hangs when no threads are available

23. Debugging Deadlocks and Process Hangs with DebugDiag: Hang Rule Name Title Group

24. Looking into Leaks… What is a leak? Types of Leaks Debugging a Leak with Debug Diagnostics

25. Leak Scenarios (2) Why are leaks difficult to identify and resolve? Server applications often cache memory allocations The allocations are long term These applications, to outsiders, look to have memory leaks There are many other caches Using Performance Monitor and seeing memory growth does not equal a memory leak

26. Leak Scenarios (3) What are short-term allocations? Memory expensive applications use “spike” allocations but are destroyed immediately after request (not long- sustaining) These are not Memory Leaks

27. Leak Scenarios (4) Difference between cache and memory leaks? Memory leaks are caused by unbounded allocations Cache memory usage rises quickly, but tops out after application is stabilized

28. Nailing a Memory Leak in a Web Application using DebugDiag: Memory Rule Name Title Group

29. Session Summary Debugging Live Web Applications without downtime is challenging Crashes cause processes to terminate due to unhandled exceptions Hangs are caused due to race conditions, lock contention, network blocking, database blocking, infinite loops and deadlocks Leaks are caused by the failure to release memory allocated beyond typical short term allocations and caching Debug Diagnostics has an extensibility model that allows it to successfully capture and analyze common debugging scenarios to determine the root cause Know what DebugDiag can do…and also what it can’t do

30. Debug Diagnostics 1.0 web resources http://www.microsoft.com/windowsserver20 03/iis/diagnostictools/default.mspx

31. © 2005 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.