Message Analysis-Guided Allocation and Low-Pause Incremental Garbage Collection in a Concurrent Language Konstantinos Sagonas Jesper Wilhelmsson Uppsala University, Sweden

Goals of this work Efficiently implement concurrency through asynchronous message-passing Memory management with real-time characteristics o Short stop-times o High mutator utilization Design for multithreading

Our context: Erlang Designed for highly concurrent applications Soft Real-Time Light-weight processes No destructive updates Data types: atoms, numbers, PIDs, tuples, cons cells (lists), binaries heap data

Our context: the Erlang/OTP system Industrial-strength implementation Used in embedded applications Three memory architectures: [ISMM’02] o Private o Shared o Hybrid

Stack Heap Private heaps PP

PP O(|message|) copy

Private heaps PP Garbage collection is a private business Fast memory reclamation of terminated processes

O(1) Shared heap PP Global synchronization Longer stop-times No fast reclamation of process-local data

Hybrid architecture PP Message area Process-local heaps Big objects area

Several possible methods o User annotations o Dynamic monitoring [Petrank et al ISMM’02] o Static analysis guided allocation Allocating messages in the message area

Static message analysis [SAS’03] Similar to escape analysis Allocation is process-local by default o Possible messages allocated on message area o Copy on demand Analysis is quite precise o Typically finds 99% of all messages

Process-local heaps Private business: No synchronization required Message area Two generations Copying collector in young generation o Fast allocation Mark-and-sweep in old generation o Prevents repeated copying of old objects Garbage Collection in Hybrid Arch.

GC of the message area is a bottleneck 1.Generational process scanning 2.Remembered set in local heaps The root-set for the message area consists of all stacks and process-local heaps This is not enough... We need an incremental collector in the Message Area!

Properties of incremental collector No overhead on mutator No space overhead on heap objects Short stop-times High mutator utilization

Old generation Organization of the Message Area Fwd Black-map Young generation Nursery From- space Nursery and from- space always have a constant size,  (=100k words) Storage area for forwarding pointers. Size bound by  (currently =  ) List of arbitrary sized areas Free-list, first-fit allocation Bit-array used to mark objects in mark- and-sweep

N limit N top allocation limit Nursery Organization of the Message Area

Incremental collector Two approaches to choose from: Work-based Reclaim n live words each step Time-based A step takes no more than t ms n and t are user-specified

Work-based collection The mutator wants to allocate need words reclaim = max( n, need ) N limit N top allocation limit Allocation limit = N top + reclaim

Time-based collection 1.User annotations (as in Metronome) 2.Dynamic worst-case calculation How much can the mutator allocate? How much live data is there?

Time-based collection  GC = reclaimed after GC – reclaimed before GC GC steps =  – reclaimed after GC  GC w M = N free GC steps N limit N top allocation limit Allocation limit = N top + w M 

Collecting the Message Area P1P2P3 FwdNurseryFromspace

Process Queue Collecting the Message Area P1P2P3 FwdFromspaceNursery

Process Queue Collecting the Message Area P1P2P3 FwdFromspaceNursery

Process Queue Collecting the Message Area P1P2P3 FwdFromspaceNursery P1

Process Queue P1 Collecting the Message Area P2P3 FwdFromspaceNursery

Process Queue P1 Collecting the Message Area P2P3 FwdFromspaceNursery

Process Queue P1 Collecting the Message Area P2P3 FwdFromspaceNursery allocation limit Cheap write barrier Link receiver to a list in the send operation

Process Queue Collecting the Message Area P2P3 FwdFromspaceNursery P1 allocation limit

Process Queue Collecting the Message Area P2P3 FwdFromspaceNursery P1 allocation limit

Process Queue Collecting the Message Area P2P3 FwdFromspaceNursery P1 allocation limit

Process Queue Collecting the Message Area P2P3 FwdFromspaceNursery allocation limit P1

Process Queue Collecting the Message Area P2P3 FwdFromspaceNursery P1 allocation limit

Process Queue Collecting the Message Area P2P3 FwdFromspaceNursery P1 allocation limit

Process Queue Collecting the Message Area P2P3 FwdFromspaceNursery P1 allocation limit

Process Queue Collecting the Message Area P2P3 FwdFromspaceNursery P1 allocation limit

Process Queue Collecting the Message Area P2P3 FwdFromspaceNursery allocation limit P1

Collecting the Message Area P2P3 FwdFromspaceNurseryallocation limit P1

Performance evaluation: Settings Intel Xeon 2.4 GHz, 1GB RAM, Linux Start with small process-local heaps (233 words, grows when needed) Measure active CPU time o using hardware performance monitors

Performance evaluation: Benchmarks Mnesia – Distributed database system 1,109 processes 2,892,855 messages Yaws – HTTP Web server 420 processes 2,275,467 messages Adhoc – Data mining application 137 processes 246,021 messages

Stop-times – Time-based Mnesia Yaws t = 1ms

Stop-times – Work-based AdhocYaws n = 2 words Mean: 3 Geo. Mean: 2 Mean: 9 Geo. Mean: 1

Stop-times – Work-based AdhocYaws n = 100 words Mean: 53 Geo. Mean: 46 Mean: 268 Geo. Mean: 36 Time (  s)

Bench- mark n = 2 MA GC n = 100 MA GC n = 1000 MA GC Non-Inc. MA GC Mnesia Yaws Adhoc Message area total GC times incremental vs. non-incremental Times in ms

Bench- mark Mutator Local GC MA n = 2 MA n = 100 MA n = 1000 Mnesia52,9064, Yaws237,62911, Adhoc61,0458, Runtimes – Incremental Times in ms

Minimum Mutator Utilization The fraction of time that the mutator executes in any time window [Cheng & Blelloch PLDI 2001]

Mutator Utilization – Work-based Adhoc Yaws n = 100 words

Concluding Remarks Memory allocator is guided by the intended use of data Incremental Garbage Collector High mutator utilization Small overhead on total runtime No mutator overhead Small space overhead Really short stop-times!

Runtimes incremental vs. non-incremental Times in ms Bench- mark Inc. Mutator Non-Inc. Mutator Mnesia52,90653,276 Yaws237,629240,985 Adhoc61,04561,578

Total GC times incremental vs. non-incremental Times in ms Bench- mark Inc. Local GC Non-Inc. Local GC Mnesia4,4394,487 Yaws11,72811,359 Adhoc8,1947,848

Mutator Utilization – Time-based Mnesia Yaws t = 1ms