Advanced Troubleshooting with Debug Diagnostics on IIS 6 Draft 2.5 5/13/06 NameTitleGroup Microsoft Corporation

Agenda Introduction A look at Debug Diagnostics Dig into crash scenarios Understanding hangs Investigating memory leaks Session Summary

Introduction Overview of Debugging: Past and Present Integral tools for resolving Debug scenarios Variables impacting Debugging

Overview of Debugging Past and Present Past: offered many tools, fewer resolutions Previous tools were often not server-ready No single tool to address different debugging scenarios Required expertise in debugging Present: more advanced tools, still time- consuming to solve Still lacks any analytical piece Existing tools are not “integrated” into single tool Do you really want to learn “debugging?”

Talking about tools… There are several tools designed at resolving the same problems As Windows Servers have grown, so have the debugging tools Example of debugging tools Exception Monitor ADPlus.vbs (i.e. – cdb.exe) IIS Debug Toolkit 1.1 IIS State

Variables Impacting Debugging (2) IIS’s Extensibility Examples ISAPIASP.NET Active Server Pages (ASP) COM components PHP (ISAPI) Perl (CGI or ISAPI) Oracle\MySQL These “extensible” parts are often integrated and executing in same process Ever felt like this?

Variables Impacting Debugging (3) Web applications often have high dependency factors Multiple technologies “chewing” on the same memory space Inexperienced server side programmers W3WP.exe ISAPI Extensions ISAPI Filters Not to mention CGI

Debug Diagnostics 1.0 “Core” pieces of DebugDiag Understanding “Rules” Using DebugDiag’s User Interface

Core pieces of Debug Diag 3 pieces make up Debug Diag 1.0 Service Host (also called Controller) User Interface DbgSvc.exe DbgSvc.exe DebugDiag.exe

Core pieces of Debug Diag Service is multi- purposed Interacts with the Debugger Host Allows: Service required to attach invasively to process To run from Terminal Services The Service – DbgSvc.exe DbgSvc.exe To register the service: C:\DebugDiag> DbgSvc /service To start the service: C:\DebugDiag> net start DbgSvc To stop the service C:\DebugDiag> net stop DbgSvc Unregister the service C:\DebugDiag> DbgSvc /unregserver

Core pieces of Debug Diag Host Where the DbgEng.dll instance is loaded Exposes 3 classes The Host – DbgHost.exe DbgControl: Attach/Detach from processes or open/analyze a memory dump DbgObj: Collect process or memory dump information Manager: Mainly to output analysis data to the report file DbgSvc.exe DbgHost.exe Attac h

Core pieces of Debug Diag User Interface designed to simplify: Rule creation View processess Initiate analysis of memory dumps Add analysis scripts Change properties for Debug Diag The User Interface – DebugDiag.exe DebugDiag.exe DbgHost.exe ControlScripts

Crash Scenarios Background of crashes Using DebugDiag to quickly identify root cause of crashes

Background: Exceptions What is an exception? How do applications cause exceptions What is a “handled” exception? Unhandled exceptions need debugging and resolution Access Violation: C Breakpoint Exception: Stack Overflow: C000 00FD C++ exception: E06D7363 Managed exception: E0434F4D E0434F4D Invalid Handle: C

Background: Heap (2) What is a heap? Heap is a data structure Heap Corruption: Certain rules must be followed when using the heap When rules are violated, heap corruption occurs Heap corruption crashes Code that corrupts heap is usually not the ones impacted

Debugging Crashes with Debug Diagnostics: Crash Rule Name Title Group

Hang Scenarios Investigating the theory of “process hangs” Using DebugDiag to debug “process hangs”

Theory of “Process Hangs” What is a process? Memory boundary for execution of code Allocated by Operating System done by calling CreateProcess or CreateProcessAsUser API Processes do not execute code Contains a minimum of 1 thread What is a thread? An independent sequence of instructions executing within a process

Theory of “Process Hangs” Thread implications on applications The level of concurrency (virtual) is dictated by the number of threads Real concurrency is dictated by the number of execution units 1 processor = one or more execution units What are common thread pools for IIS? ATQ thread pool ASP thread pool ASP.NET thread pool

Runaway Threads A runaway thread is a thread stuck in a infinite (or near) loop Performing a task that is consuming CPU Either by design or because of error conditions Runaway thread is taking CPU resources away from other threads Often takes away from good threads Causes performance degradation for server

SQL Thread4 Thread3 Thread1 Defining a Blocked Thread A thread that isn’t using any CPU resources Waiting on database calls to return Waiting on web service calls to return Waiting on a socket/network call to return Waiting on a lock IIS

Locks and Deadlocks What is a lock? A synchronization mechanism to protect resources from being corrupted due to simultaneous access by multiple threads Why do developers use locks? Protecting data Protecting other resources that are not memory based Impact on applications if locks are poorly implemented Causes threads to block Can cause deadlock situations when using multiple locks in the application Process hangs when no threads are available

Debugging Deadlocks and Process Hangs with DebugDiag: Hang Rule Name Title Group

Looking into Leaks… What is a leak? Types of Leaks Debugging a Leak with Debug Diagnostics

Leak Scenarios (2) Why are leaks difficult to identify and resolve? Server applications often cache memory allocations The allocations are long term These applications, to outsiders, look to have memory leaks There are many other caches Using Performance Monitor and seeing memory growth does not equal a memory leak

Leak Scenarios (3) What are short-term allocations? Memory expensive applications use “spike” allocations but are destroyed immediately after request (not long- sustaining) These are not Memory Leaks

Leak Scenarios (4) Difference between cache and memory leaks? Memory leaks are caused by unbounded allocations Cache memory usage rises quickly, but tops out after application is stabilized

Nailing a Memory Leak in a Web Application using DebugDiag: Memory Rule Name Title Group

Session Summary Debugging Live Web Applications without downtime is challenging Crashes cause processes to terminate due to unhandled exceptions Hangs are caused due to race conditions, lock contention, network blocking, database blocking, infinite loops and deadlocks Leaks are caused by the failure to release memory allocated beyond typical short term allocations and caching Debug Diagnostics has an extensibility model that allows it to successfully capture and analyze common debugging scenarios to determine the root cause Know what DebugDiag can do…and also what it can’t do

Debug Diagnostics 1.0 web resources 03/iis/diagnostictools/default.mspx

© 2005 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.