1 Why Events Are A Bad Idea (for high-concurrency servers) By Rob von Behren, Jeremy Condit and Eric Brewer (May 2003) CS533 – Spring 2006 – DONG, QIN TINA

2 Arguments for Events & Threads Which One is better for high concurrent apps? Ousterhout In favor of using events; Support the use of threads for true CPU concurrency Welsh & Culler &Brewer SEDA (staged event-driven architecture) In favor of using events; But SEDA is designed with events and threads.

3 Arguments for Events & Threads Lauer and Needham Two strategies should yield equivalent performance. Message-oriented System (Events) Procedure-oriented System (Thread) event handlers events accepted by a handler SendMessage / AwaitReply SendReply waiting for messages Monitors functions exported by a module procedure call, or fork/join return from procedure waiting on condition variables

4 Arguments for Events & Threads Behren, Condit, Brewer (This paper authors) 1. Threads can perform as well as events for high concurrency. 2. Events don’t show substantial qualitative advantages. 3. The absence of scalable user-level threads is caused by an artifact of poor implementation rather than a basic property of the thread abstraction.

5 Behren’s arguments for events & threads: Are Events better ? In favor of eventsNot support events (This paper authors) Inexpensive synchronization due to cooperative multitasking; Lower overhead for managing state (no stacks); Better scheduling and locality (based on apps); flexible control flow (not just call/return) These properties could be implemented by threads as well.

6 Behren’s arguments for events & threads: Are threads bad? Performance. Threads - BadThreads – not Bad (This paper authors) O(n) in the number of threads. high context switch compared with events This disadvantages are not properties of threads Experiment: remove most of O(n) operations from the scheduler in GNU Pth user-level threading package, match SEDA’s performance.

7 Behren’s arguments for events & threads: Are threads bad? Control Flow Threads - BadThreads – not Bad (This paper authors) Threads have restrictive control flow. Analyze control flow patterns in Ninja, SEDA, Flash (call/return, parallel calls, and pipelines) These patterns could be expressed by threads as well. Thread Run-time call stack encapsulates all live state for a task (good for debugging)

8 Events – problem with control folw. (This paper authors) Events control flow requires mentally match call/return pairs, which could cause race condition / logic error due to unexpected message arrives. Events call/return requires manually save and restore live state. Control Flow

9 Behren’s arguments for events & threads: Are threads bad? - Synchronization Threads - BadThreads – not Bad (This paper authors) too heavyweightSupport Adya’s point: Event lightweight is due to cooperative multitasking, not events themselves. Threads could use lightweight as events do. Cooperative multitasking only provides sync on uni- processors (many high concurrency servers run on multiprocessors).

10 Behren’s arguments for events & threads: Are threads bad? - State Management Threads - BadThreads – not Bad (This paper authors) Thread stacks are an ineffective way to manage live state. Threads face tradeoff between risking stack overflow and wasting virtual address space on large stacks. Enabling dynamic stack growth could solve ineffective way to manage live state.

11 Behren’s arguments for events & threads: Are threads bad? - Scheduling Threads - BadThreads – not Bad (This paper authors) The virtual processor model forces the runtime system to be too generic and prevents it from making optimal scheduling decisions. Support Lauer-Needham's point : "we can apply the same scheduling tricks to cooperatively scheduled threads."

12 Behren’s arguments for events & threads: Threads is better : 1. Control Flow (Thread Run-time call stack encapsulates all live state for a task ) 2. Exception Handling and State Lifetime: Using threads: Easy cleaning up task state since the thread stack tracks the live state for the task. Using Events: a). Task state is typically heap allocated. Freeing this state at the correct time is extremely difficult. b). Garbage collection mechanism (used in SEDA) is inappropriate for high performance. Maintaining correct reference counting is difficult for managing state (used in Inktomi’s Traffic Server). 3. Even in existing event-driven systems is using threads for the most complex parts (eg. recovery) 4.The applications (not need high concurrency) are written with thread (eg, FTP server in Harvest) in many existing systems.

13 Idea with Compiler Support for Threads Developing a mechanism - Dynamic Stack Growth To avoid overflow / wasted space issues. 1. Compiler analysis provides an upper bound on the amount of stack space needed. 2. Compiler analysis helps determine which call sites may require stack growth. Live State Management Compilers could easily purge unnecessary state from the stack before making function calls. Synchronization 1. Compile analysis helps determine which atomic sections are safe to run concurrently. 2. Compile analysis can reduce the occurrence of bugs by warning the programmer about data races.

14 Evaluation Designed and implemented a simple a user-level cooperative threading package for Linux. 1. uses the coroutine library for minimalist context switching 2. translates blocking I/O requests to asynchronous requests internally. I. asynchronous socket I/O (use the UNIX poll() system call) II.asynchronous disk I/O (use a thread pool that performs blocking I/O operations) 3. The library also overrides blocking system calls and provides a simple emulation of pthreads. Design a test web server (Knot) with this thread pkg. accepts static data requests. allows persistent connections. includes a basic page cache.

15 Evaluation Comparison KnotHaboob user-level cooperative threading pkg Knot-C favors connections Knot-A favors accepting SEDA’s event-driven web server Have the same general performance patterns as Haboob Initially: linear increase in bandwidth as simultaneous connections increases; server saturated: the bandwidth levels out. Have the same general performance patterns as Knot. Initially: linear increase in bandwidth as simultaneous connections increases; server saturated: the bandwidth levels out.

16 Web server bandwidth versus concurrent clients (Failed to run in Haboob with clients >= 16384)

17 KnotHaboob The performance degradation in Knot-A is due to the poor scalability of poll(). The performance degradation in Haboob is due to the poor scalability of poll(). The steady-state bandwidth (Knot-C) is ~700 Mbit/s Maximum bandwidth is 500 Mbit/s: Why it is lower than Knot? requires more context switches (6 times than Knot) by thread pool per handler model create crossings and queuing operations by large # of small modules create many temporary objects and relie heavily on garbage collection. require various forms of run-time dispatch, since executing next event handler is unknown statically.

18 Conclusions A well designed thread package can achieve the same scaling behavior as a well-designed event system. Simpler programming model combined with compiler analyses could gives threads an advantage over events when writing highly concurrency Authors advocate tight integration between the compiler and the thread system