Multithreading vs. Event Driven in Code Development of High Performance Servers

Goals Maximize server utilization and decrease its latency. –Server process doesn’t block! Robust under heavy loads. Relatively easy to develop.

Key Idea Event Driven Multi-Threaded Programming level: Implementation level: Multi-threaded serversEvent driven servers

Key Idea Get the best of the two worlds. Event Driven Multi-Threaded Programming level: Implementation level: Multi-threaded serversEvent driven servers Lazy threaded servers

Backgound: AIO Asynchronous I/O mainly used for disk reads/writes. Issue disk request and return immediately without blocking for results. On completion an event is raised.

Servers Architectures: a- Multi-threaded Servers  Kernel thread per request executing the request handler.  A thread blocking on I/O doesn’t block the server.  Ex: Apache, MySql. Request Handler A B C Function Calls

Multi-threaded Servers (Cont.) Advantages: + Relatively easy to develop. Disadvantages: - Not good performance. Thread scheduling. Context switching. - Doesn’t scale. Under heavy load, many threads would be created exhausting kernel’s memory, crashing the server.

Needed for Event-Driven servers Kernel event notification mechanism. Supported by FreeBSD Kernel. Events queued in a kernel queue, and delivered to user by kernel calls. Events usually correspond to AIO completions, or socket reads/writes. Asynchronous I/O

 Single thread (event loop).  AIO operations.  I/O completions / Requests (events) put in Event Queue.  Continuations.  Ex:Flash web server (slightly different architecture). + High performance. + Scalable/Robust. - Hard to write. b- Event Driven Servers Event Loop A1 B1 C1 A2 B2 C2 Events From Kernel “Blocking” operation Event Queue (KQueue)

Lazy Threading Similar design to event driven but no continuations. Threads yield control on fixed points (I/O).  If request handler doesn’t “block” –Run handler to completion then use same thread to handle next event.  Else –Suspend current thread, create another user thread to handle next event.

Wrappers / Thread Suspension IO_Wrapper(..) { uc = save_user_context(..); /* save the current user thread context to be able to resume it later */ aio_func(..); /* non-blocking I/O call */ stack=allocate_stack(); switch_thread(event_loop,stack); /* suspend current thread and create a new thread running the event loop using the newly allocated stack */. } Wrappers around “blocking” operations to create the blocking illusion. Uses AIO. Saves current stack instead of unwinding it. Creates a user thread by jumping to the event loop and using a new stack.

Thread Resumption Event_loop() {. e = get_event(..); /* read new event from the kernel’s kqueue */ if e is completion of aio_func(..) run_thread(uc); /* dispose current thread and resume the IO_Wrapper from where it has stopped */. } Restores registers and jumps back to wrapper as if blocking has finished. Current thread is disposed as it’s useless. Wrapper’s thread eventually returns to the event loop after handling its request.

Summary Yes NoHigh Performance No YesThread / request? Yes (user) NoYes (kernel) >1 thread NoN/AYesArbitrary thread yield YesNoYes“Easy” to build Yes NoScalable Lazy- Threaded Event Driven Multi- Threaded