1. PA3: Improving Performance with I/O Multiplexing Part 1-1: Nov. 7, Part 1-2: Nov. 10 Part 2-1: Nov. 17, Part 2-2: Nov.20

2. Dealing with Concurrency Creating new child or new thread Expensive Only handling 1 connection in a thread I/O multiplexing with event-driven programming Multiplex input/output between multiple files (or sockets) Select() When event comes, select returns Buffered I/O (POSIX AIO) I/O asynchronously done in background Libevent When event comes, pre-designated function is called

3. Select() Select returns when there’s new event After select returns, it checks whether the fd is set. If fd is set, process the event (e.g., handle HTTP request) // server example ret = select(n, &fds, NULL, NULL, NULL); if (ret < 0) { perror(“select”); } else if (ret) { for (i = 0; i < n; i++) { if (FD_ISSET(fd_set[i], &fds)) { /* if listening fd, accept* / /* otherwise, process request */ if (fd_set[i] == listenfd) { accept(listenfd, …); } else { read(fd_set[i], …); write(fd_set[i], …); }

4. Part 1: POSIX AIO POSIX asynchronous I/O aio_read(), aio_write() immediately returns The actual read and write are processed in the background The application can do another work without being blocked by read() or write() The completion of I/O can be notified by a signal

5. Server Example using AIO struct aiocb aiocbList[MAX_FD]; sa.sa_flags = SA_RESTART | SA_SIGINFO; sa.sa_sigaction = aioSigHandler; if (sigaction(IO_SIGNAL, &sa, NULL) == -1) exit(1); while (1) { fd = accept(listenfd, …); if (fd < 0) continue; /* build aio control block */ aiocbList[fd].aio_fildes = fd; aiocbList[fd].aio_buf = malloc(BUF_SIZE); aiocbList[fd].aio_nbytes = BUF_SIZE; aiocbList[fd].aio_reqprio = 0; aiocbList[fd].aio_offset = 0; aiocbList[fd].aio_sigevent.sigev_notify = SIGEV_SIGNAL; aiocbList[fd].aio_sigevent.sigev_signo = IO_SIGNAL; aiocbList[fd].aio_sigevent.sigev_value.sival_ptr = &aiocbList[fd]; s = aio_read(&aiocbList[fd]); if (s == -1) errExit("aio_read"); }

6. Server Example using AIO /* Handler for I/O completion signal */ static void aioSigHandler(int sig, siginfo_t *si, void *ucontext) { struct aiocb *acb = si->value.sival_ptr; int fd = acb->aio_fildes; /* process the request stored in acb->aio_buf /* write a response using aio_write() */ /* if aio_write() completes, close the connection */ } For more detail, refer to the man page of aio (in shell, $ man aio)

7. Part 2: Libevent Event notification library Execute a callback function when a specified event occurs on a file descriptor Availability-based If incoming connection available, it notifies If incoming request available, it notifies If sending buffer available, it notifies Calls pre-registered callback function when there is event Use nonblocking for I/O operations Set socket fds to nonblocking by fcntl()

8. Server Example using libevent #include /* for libevent */ int main() { struct event ev_accept; … /* create listening socket and set it nonblock */ … event_init(); /* register the event */ event_set(&ev_accept, listenfd, EV_READ | EV_PERSIST, OnAcceptEvent, NULL); event_add(&ev_accept, NULL); /* start event loop */ event_dispatch(); … close(listenfd); return 0; }

9. Server Example using libevent static void OnAcceptEvent(int fd, short event, void *arg) { struct event ev_read; int new_fd = accept(fd, NULL, NULL); /* set new_fd nonblocking */ event_set(&ev_read, new_fd, EV_READ, OnReadEvent, NULL); event_add(&ev_read, NULL); } static void OnReadEvent(int fd, short event, void *arg) { struct event ev_write; char buf[BUFSIZE]; read(fd, buf, BUFSIZE); /* process the request in buffer */ /* write the response */ write(fd, response, len); }

10. Buffer Management What if send buffer is full? write() will return -1 errno will be set to EAGAIN or EWOULDBLOCK What you need to do? You need to send the response again later Remember the “state” Until where it was sent Register a write event Retry later when write event comes When send buffer becomes available, write event will come

11. Remembering the State struct context { char w_buf[BUFSIZE]; /* write buffer */ int fd; struct event ev_read; struct event ev_write; int off; /* buffer offset */ int rem_len; /* remaining length */ }; static void OnReadEvent(int fd, short event, void *arg) { struct context *ctx = (struct context *)arg; char buf[BUFSIZE]; rd = read(fd, buf, BUFSIZE); /* process the request in buffer */ /* start writing the response */ /* if not sent fully, register an event */ wr = write(fd, ctx->w_buf, ctx->rem_len); ctx->off += wr; ctx->rem_len -= wr; event_del(ctx->ev_read, NULL); event_set(ctx->ev_write, fd, EV_WRITE, OnWriteEvent, ctx); event_add(ctx->ev_write, NULL); }

12. Remembering the State static void OnWriteEvent(int fd, short event, void *arg) { struct context *ctx = (struct context *)arg; wr = write(fd, ctx->w_buf + ctx->off, ctx->rem_len); ctx->off += wr; ctx->rem_len -= wr; /* if done writing */ if (ctx->rem_len == 0) { event_del(ctx->ev_write, NULL); close(fd); free(ctx); }

13. Evaluation 50 pt for each part 20 pt for basic functionality testing Ina.kaist.ac.kr, yahoo, naver 20 pt for performance and robustness testing Testing with ab (ApachBench) Many consecutive requests Many concurrent connections 10 pt for report Performance comparison Threaded vs. AIO vs. libevent