1 Thread Synchronization (Un)Fairness in the.NET Common Language Runtime If you are building an application that absolutely requires fair thread synchronization, you should not use the.NET Framework at all.“ Jeffrey Richter, In his article "Thread Synchronization Fairness in the.NET CLR”, published 6 February, 2003)

2 What?... Why? The CLR manages memory via garbage collection. When the CLR wants to start a garbage collection, … the CLR will suspend the threads executing managed code. When Windows suspends a thread, it stops the thread from waiting for any thread synchronization object. Later, when the thread is resumed, all the suspended threads race back to wait on the object that it was waiting on before it got suspended. This means that threads are not guaranteed to gain ownership of an object on a first-in-first-out basis. Jeffrey Richter, In his article "Thread Synchronization Fairness in the.NET CLR”, published 6 February, 2003)

3 So what is this presentation about? Thread Synchronization Garbage Collection in.NET managed application. How these seemingly unrelated areas conspire to cause unfair thread synchronization in the.NET CLR.

4 First, Thread Synchronization: Thread Synch support exposed in the underlying OS (and used by the.NET CLR) OS Kernel Support for Thread Synch Thread Synch in the CLR

5 Managed execution in the common language runtime (CLR) Figure 2-1, Introducing Microsoft.NET, Second Edition by David S. Platt

6 Thread Synch Support in Win2k User Mode User Mode – No transition to Kernel Mode* Interlocked Functions – Set simple 32 bit values atomically. Critical Sections – Data structure owned by one thread at a time to synchronize access to common code. Can be used to manipulate complex data structures in a thread safe manner. * When entering a critical section, the thread stays in user mode only as long as there is no contention. If the critical section is owned by another thread, the thread will transition to kernel mode and enter a wait state.

7 Thread Synch Support in Win2k Kernel Mode Provided Primarily through the Win32 subsystem Wait functions (i.e. WaitForSingleObject) A thread calls a Wait function to place itself in a wait state until a kernel object is signaled.

8 Waiting on a dispatcher object Figure 3-19, Inside Microsoft Windows 2000, Third Edition by David A. Solomon and Mark E. Russinovich

9 Definitions of the Signaled State Table 3-9: Inside Microsoft Windows 2000, Third Edition by David A. Solomon and Mark E. Russinovich

10 Running in the CLR Loading and initializing the CLR Figure 1-3, Applied Microsoft.NET Framework Programming by Jeffrey Richter

11 Threads in a Managed Process Each application thread in a managed application has 1:1 correspondence with an OS thread. The CLR will create additional threads for its own purposes: Finalizer, Debug, Threadpool, Asynch Ops (Timers, Delegates) All thread creation, dispatch and execution is done in the OS. The CLR itself provides none of this functionality.

12 Thread Synch Classes in.NET Adaptation of Table in Robert Burns, Multithreaded Programming with Visual Basic.NET, MSDN Online Library, February 2002

13 The other piece of the puzzle … Automatic Memory Management… a.k.a. Garbage Collection What is it and why do we use it?

14 Traditional Manual Memory Management (e.g. C-Runtime) Memory Allocation (malloc): User code requests a certain number of bytes. CRT walks linked list to find large enough memory block then updates list. Memory Deallocation (free): CRT updates linked list to free up the memory block.

15 Automatic Memory Management (e.g. Garbage Collection) CLR reserves contiguous block of memory when process initialized, the managed heap. Pointer maintained to next free location. Memory Allocation (newobj IL instruction): CLR determines memory required based on object metadata. If enough memory available, the CLR returns a pointer to next free location, and updates the pointer to the next free location. If not enough memory in the heap, a garbage collection (details later). Memory Deallocation: Memory is never explicitly deallocated, and is not freed until the next garbage collection is done.

16 Memory Management Comparison Manual Memory MgtAutomatic Memory Mgt AdvMore power to programmerSimple and fast allocation Predictable time delayGood Object Locality Deterministic Finalization DisadvMore power to programmerNot predictable – usually short / sometimes (unpredictably) long Introduces global dependenciesNon-deterministic finalization Most allocations slowerUser has less power Less object locality Memory leaks Early deallocation – memory corruption Fragile Code (static allocations) Application specific GC awkward & buggy

17 Garbage Collection in the.NET CLR The.NET CLR uses a Mark-Compact type garbage collector that utilizes generations and a separate Large Object Heap …

18 Garbage Collection = Detection + Reclamation Mark-Compact type garbage collection Mark – garbage detection GC creates a list of live objects by starting at a set of roots and walking all referenced objects, marking them as reachable. Anything not marked is reclaimable garbage. Other types of marking: Reference counting – Number of object references maintained. Copying – copy live objects as they are found to another space.

19 Garbage Collection = Detection + Reclamation Mark-Compact type garbage collection Compact – garbage reclamation GC walks linearly through the heap looking for contiguous garbage (now free) blocks that it can move live objects down to. In this way live objects are moved to the front of the heap and free space is ‘squeezed’ to the end. Pointers to moved objects are updated. Other types of reclamation: Copying: copy live objects as they are found to another space. Sweep: Garbage marked as free, but memory left fragmented.

20 Mark-Compact GC Mark-Compact Type Garbage Collection AdvProvides contiguous section of memory for easy future memory allocation. Object ordering maintained / locality somewhat maintained Circular references not a problem (e.g. garbage objects that reference each other – keeping themselves alive) DisadvMemory cannot be accessed during garbage collection. Several Passes over data required – marking, compute new locations, update pointers, actual object move. Time consuming if a lot of live data left. Generations used to help this situation …

21 Generations Most objects live a very short time. “80-98% of new objects die within a few million CPU instructions.” Paul R. Wilson, Uniprocessor Garbage Collection Techniques, 1992 International Workshop on Memory Management, St. Malo, France, September A small percent live much longer. Using generations capitilizes on these statistics: Each time an object survives a GC, it is graduated to a higher generation (.NET currently supports generations 0,1, and 2). The garbage collector will normally do a generation 0 collection. This will lessen the amount of long lived live objects that must be marked and copied during a GC.

22 Large Object Heap Any objects that are 85,000 bytes or more are allocated on a separate heap – the Large Object Heap. The Large Object Heap is never compacted. Done to alleviate the need to copy large objects during the compact phase of the garbage reclamation.

23 How does this all tie together? Lets reexamine one of the disadvantages listed for the Mark-Compact type GC: Memory cannot be accessed during garbage collection. In the case we are examining, the.NET CLR on a uniprocessor system, this means that all threads that are running managed code in the process must be suspended while the GC moves memory and fixes up pointers.

24 The problem restated A request is made to instantiate a new object, if there is not enough room in generation 0 for this object, then a garbage collection is run. During a garbage collection, all threads that are running managed code are suspended. “When Windows suspends a thread, it stops the thread from waiting for any thread synchronization object. Later, when the thread is resumed, all the suspended threads race back to wait on the object that it was waiting on before it got suspended. This means that threads are not guaranteed to gain ownership of an object on a first-in-first-out basis. Since a garbage collection can start at any time (and cannot be prevented), the architecture of the CLR just doesn't support fair thread synchronization — period. “ Jeffrey Richter, In his article "Thread Synchronization Fairness in the.NET CLR”, published 6 February, 2003)

25 References General: Jeffrey Richter, Thread Synchronization Fairness in the.NET CLR. published 6 February, Francesco Balena, Programming Microsoft Visual Basic.NET, Microsoft Press © Aaron Barth, Jackie Richards, Production Debugging for.NET Framework Applications, Microsoft Patterns and Practices, MSDN Online Library, November 2002, ( ) Robert Burns, Multithreaded Programming with Visual Basic.NET, MSDN Online Library, February 2002 ( us/dv_vstechart/html/vbtchasyncprocvb.asp ) us/dv_vstechart/html/vbtchasyncprocvb.asp Jeffrey Richter, Applied Microsoft.NET Framework Programming. Microsoft Press © Jeffrey Richter, Thread Synchronization Fairness in the.NET CLR. published 6 February, Windows David A. Solomon and Mark E. Russinovich, Inside Microsoft Windows 2000, Third Edition. Microsoft Press © Jeffrey Richter, Programming Applications for Microsoft Windows, Fourth Edition. Microsoft Press © Garbage Collection Paul R. Wilson, Uniprocessor Garbage Collection Techniques, 1992 International Workshop on Memory Management, St. Malo, France, September 1992

26 Questions? As clear as mud?