1 Data Communications and Networking Socket Programming Part II: Design of Server Software Reference: Internetworking with TCP/IP, Volume III Client-Server Programming and Applications By Douglas E. Comer, David L. Stevens

2 Outline Review of multithreading Server Design —Iterative, Connectionless —Iterative, Connection-Oriented —Concurrent, Connectionless —Concurrent, Connection-Oriented Multi-thread Singly threaded (not required by COMP2330)

3 Multithreading In modern operating systems, a process can have multiple threads —They share the process’s resources such as memory space, opened files, sockets, etc. Advantages —Good for CPUs with multiple cores —Can overlap different IO. E.g., a thread can wait for an input from the user, while another thread is receiving data from the network —Context switching between threads is faster than that between processes

4 Threads in MS Windows A thread is executed independently of other threads. All threads in a process share: —Global variables —Resources that the OS allocates to the process When multiple threads execute a piece of code concurrently —Each thread has its own, independent copy of the local variables associated with the code. —Each has its own run-time stack of procedure activation records.

5 Threads in MS Windows (Cont.) A program can call _beginthread() to create a new thread to execute a specified function. Then the program and the newly created thread are executed concurrently. unsigned long _beginthread( void(* Func )(void *), unsigned stack_size, void *arglist ); The 1st parameter: start address of the function to be executed The 2nd parameter: stack size for the new thread; 0 for system default The 3rd parameter: pointer to a list of parameters to be passed to the function; NULL for no parameter Two remarks: 1. must be included 2. The application must link with one of the multithreaded C run-time libraries. E.g., using the “/MT” option of cl.exe command.

6 Threads in MS Windows (Cont.) int addem(int count) { int i, sum; sum = 0; for(i = 1; i <= count; i++) { printf(“The value of i is %d\n”, i); fflush(stdout); sum += i; } printf( “The sum of i is %d \n”, sum); fflush(stdout); return 0; } /* threadsum.c */ /* To compile: > cl threadsum.c /MT */ #include int addem(int); int main() { _beginthread( (void (*)(void())addem, 0, (void *)5); addem(5); return 0; }

7 Threads in MS Windows (Cont.)

8 Server Design Issues Connectionless vs. connection-oriented —UDP vs. TCP —Reliability issue? TCP wins. —Support of broadcast or multicast? UDP wins. —Real time applications? UDP may win. Client and server must use the same protocol

9 Server Design Issues (Cont.) Iterative vs. concurrent —Iterative server handles one request at a time. —If another request arrives while the server is busy handling an existing request, the new request has to wait. Iterative servers are easier to design, implement, and maintain. Iterative server is not good for a service with a long “request processing time”. —Concurrent server handles multiple requests concurrently. It can decrease the response time. It can achieve high performance on a server with multiple processors. It can achieve high performance by overlapping processing and I/O.

10 Iterative Server server client being served waiting clients

11 Concurrent Server

12 (1) Iterative, Connectionless Server Algorithm 1.Create a datagram (UDP) socket and bind to the well-known address for the service being offered. 2.Repeatedly receive the next request from a client, formulate a response, and send a reply back to the client according to the application protocol. It is the most common form of connectionless server, used especially for services that require a trivial amount of processing for each request. Example:

13 (2) Iterative, Connection-Oriented Server Algorithm 1.Create a stream (TCP) socket and bind to the well-known address for the service being offered. 2.Place the socket in passive mode, making it ready for use by a server. 3.Accept the next connection request from the socket, and obtain a new socket for the connection. 4.Repeatedly receive a request from the client, formulate a response, and send a reply back to the client according to the application protocol. 5.When finished with a particular client, close the connection and return to step 3 to accept a new connection. A less common server type used for services that require a trivial amount of processing for each request, but for which reliable transport is necessary. Example:

14 (3) Concurrent, Connectionless Server Algorithm Master 1.Create a socket and bind to the well-known address for the service being offered. Leave the socket unconnected. Master 2.Repeatedly call recvfrom to receive the next request from a client, and create a new slave thread to handle the response. Slave 1.Receive a specific request upon creation as well as access to the socket. Slave 2.Form a reply according to the application protocol and send it back to the client using sendto. Slave 3.Exit (i.e., a slave thread terminates after handling one request). It is an uncommon type in which the server creates a new thread to handle each request.

15 (4) Concurrent, Connection- Oriented Server Algorithm Master 1.Create a socket and bind to the well-known address for the service being offered. Leave the socket unconnected. Master 2.Place the socket in passive mode, making it ready for use by a server. Master 3.Repeatedly call accept to receive the next request from a client, and create a new slave thread to handle the response. Slave 1.Receive a connection request (i.e., socket for the connection) upon creation. Slave 2.Interact with the client using the connection: receive request(s) and send back response(s). Slave 3.Close the connection and exit. The slave thread exists after handling all requests from one client. The most general type of server, and the most common implementation. Example:

16 Thread Structure 1.The master server thread doesn’t communicate with clients directly. It merely waits at the well-known port for the next connection request (accept()). 2.Once a request has arrived, the master server thread returns the socket descriptor, and creates a slave thread to handle the connection. The new socket descriptor is passed to the slave thread as a parameter.

17 (5) Concurrent, Connection-Oriented Server Algorithm: single thread 1.Create a stream socket and bind to the well-known port for the service. Add socket to the list of those on which I/O is possible. 2.Use select to wait for I/O on existing sockets. 3.If original socket is ready, use accept to obtain the next connection, and add the new socket to the list of those on which I/O is possible. 4.If some socket other than the original is ready, use recv to obtain the next request, form a response, and use send to send the response back to the client. 5.Continue processing with step 2 above. The most general type of server, and a less common implementation. Example:

18 Thread Structure 1.The single thread maintains all the sockets, including the one listening socket and all other connected sockets. 2.The single thread use select() to monitor all the interested sockets: event-driven! 3.If nothing happens, select() just blocks itself. If select() returns, it means some event(s) happened and we need to response!

19 Thread vs. select() There is a debate between thread programming and event-based programming. J.K. Ousterhout, “Why Threads Are a Bad Idea (for most purposes),” Presentation given at the 1996 Usenix Annual Technical Conference, January — R. Behren, J. Condit, and E. Brewer, “Why Events Are a Bad Idea (for high-concurrency servers),” Usenix HotOS, May (from University of California at Berkeley)

20 Appendix: select() in MS Windows The normal Blocking I/O Model: application recvfrom() system call kernel no datagram ready datagram ready wait for data copy datagram copy complete return OK Process blocks in call to recvfrom() process the datagram copy data from kernel to user

21 application select() system call kernel no datagram ready datagram ready wait for data copy datagram copy complete return OK Process blocks in call to select(), waiting for one of possibly many sockets to become readable process the datagram copy data from kernel to user return readable recvfrom() system call Process blocks while data copied into application buffer Appendix: select() in MS Windows

22 Appendix: select() in MS Windows int select ( int nfds, fd_set * readfds, fd_set * writefds, fd_set * exceptfds, const struct timeval * timeout ); select() returns the number of ready file descriptors if successful, 0 if the time limit was reached. From winsock2.h: #ifndef FD_SETSIZE #define FD_SETSIZE 64 #endif typedef struct fd_set { u_int fd_count; /* how many are SET? */ SOCKET fd_array[FD_SETSIZE]; /* an array of SOCKETs */ } fd_set; fd_set is used to record an array of socket descriptors that we are interested in. select() can monitor three sets of sockets, each for a different purpose: read, write, and exception. select() tells the OS to wait for any one of multiple events to occur, and to wake up the process only when one or more of these events occurs, or when a specified amount of time has passed.

23 A set of macros are defined to manipulate fd_set Example: fd_set readfd; socket s; FD_ZERO(&readfd); /* initialize readfd to empty */ FD_SET(s, &readfd); /* add s to readfd */ FD_CLR(s, &readfd); /* remove s from readfd */ FD_ISSET(s, &readfd); /* check if s belongs to readfd */ You can check the definition of these macros in winsock2.h ! Appendix: select() in MS Windows

24 When will select() return? 1.For socket descriptors in readfds: If it is a listening socket, an arrival of a new TCP connection request can cause select() to return. For this case, normally the program will call accept() to create a new socket, and put that new socket into the readfds (or writefds) using FD_SET. If it is a connected socket, the arrival of new data may cause select() to return. Normally the program can call read() to get the data. If read() returns 0, it means the client has closed the socket. 2.For socket descriptors in writefds: Normally it means data are ready to be sent. Appendix: select() in MS Windows