SEDA: An Architecture for Well-Conditioned, Scalable Internet Services Matt Welsh, David Culler, and Eric Brewer Computer Science Division University of California, Berkley Presented By Oindrila Mukherjee

Agenda What is SEDA ? Why SEDA ? Thread-based Concurrency Event-driven Concurrency SEDA Goals Stages Applications Dynamic Resource Controllers Implement SEDA concepts using existing OS primitives Asynchronous Socket I/O Asynchronous File I/O Conclusion

What is SEDA ? SEDA – Staged Event-Driven Architecture Enables high concurrency and load conditioning Simplifies construction of well conditioned services Combines threads and event-based programs

Why SEDA ? Internet services must be responsive, robust and always available Service request for dynamic content Fluctuation of load SEDA combines threads and event-based programming models

Thread-Based Concurrency Relatively easy to program Associated overheads Cache and TLB misses Scheduling overhead Lock Contention

Thread-Based Concurrency (contd.) Can be achieved in two ways Unbounded thread allocation Thread-per-task model Eat up memory Too many context switches Bounded thread pools Fixed limit on number of threads allocated Avoids throughput degradation Unfairness to clients Large waiting times For servicing internet services

Event-Driven Concurrency Smaller number of threads No Context Switches Little degradation in throughput Designing the scheduler – complex Difficult to achieve modularity 4. Events trigger state transitions in the FSM

Threads vs. Events

SEDA Application is a network of stages Stages connected by event queues Uses dynamic resource controllers

SEDA - Goals Support massive concurrency Simplify the construction of well-conditioned services Enable Introspection Support self-tuning resource management

SEDA - Stages Event Handler – provides logic of the stage Incoming Event Queue – queues events processed by the event handler for that stage Thread Pool – for dynamic allocation of threads Controller – manages the stage, affects scheduling and thread allocation

SEDA - Applications A network of stages Queue decouples execution amongst stages Independent load management Improves debugging and performance analysis

SEDA - Dynamic Resource Controllers Thread Pool Controller Avoids allocating too many threads Queue length > threshold, controller adds a thread Threads idle, controller removes thread

SEDA - Dynamic Resource Controllers (contd.) Batching Controller Adjusts number of events processed by each invocation of the event handler within a stage Observes output rate of events Decreases batching factor until throughput begins to degrade

Asynchronous I/O Primitives Asynchronous Socket I/O Provides a non-blocking sockets interface for services Outperforms the thread-based compatibility layer Asynchronous File I/O Implemented by using blocking I/O and a bounded thread pool One thread processes events for a particular file at a time

Conclusion Internet services experience huge variations in service load Slashdot Effect Resources must be utilized to make best use of them Dynamic allocation of resources enables this SEDA uses dynamic controllers for resource management and scheduling of each stage SEDA isolates threads within each stage and uses event-driven design to maximize throughput at high load