1. 1 Networking Programming (I)

2. A Client-Server Transaction Client process Server process 1. Client sends request 2. Server handles request 3. Server sends response 4. Client handles response Resource Note: clients and servers are processes running on hosts (can be the same or different hosts). 2

3. Hardware Organization of a Network Host main memory I/O bridge MI ALU register file CPU chip system busmemory bus disk controller graphics adapter USB controller mousekeyboardmonitor disk I/O bus Expansion slots network adapter network 3

4. Logical Structure of an internet Ad hoc interconnection of networks –No particular topology –Vastly different router & link capacities Send packets from source to destination by hopping through networks –Router forms bridge from one network to another –Different packets may take different routes router host 4

5. Transferring data over an internet LAN2 protocol software client LAN1 adapter Host A LAN1 data (1) dataPHFH1 (4) dataPHFH2 (6) data (8) dataPHFH2 (5) LAN2 frame protocol software LAN1 adapter LAN2 adapter Router dataPH (3) FH1 dataPH FH1 (2) internet packet LAN1 frame (7) dataPHFH2 protocol software server LAN2 adapter Host B PH: Internet packet header FH: LAN frame header 5

6. 6 Global IP Internet Based on the TCP/IP protocol family. –UDP (Unreliable Datagram Protocol) uses IP to provide unreliable datagram delivery from process-to-process. –TCP (Transmission Control Protocol) uses IP to provide reliable byte streams (like files) from process-to-process. Accessed via a mix of Unix file I/O and functions from the Berkeley sockets interface.

7. 7 Hardware and software organization of an Internet application TCP/IP Client Network adapter Global IP Internet TCP/IP Server Network adapter Internet client hostInternet server host Sockets interface (system calls) Hardware interface (interrupts) User code Kernel code Hardware and firmware

8. 8 Programmer’s view of the Internet Hosts are mapped to a set of 32-bit IP addresses. –202.120.225.9 The set of IP addresses is mapped to a set of identifiers called Internet domain names. –202.120.225.9 is mapped to bbs.fudan.edu.cn A process on one host communicates with a process on another host over a connection.

9. 9 Dotted decimal notation By convention, each by in a 32-bit IP address is represented by its decimal value and separated by a period IP address 0x8002C2F2 = 128.2.194.242

10. 10 Dotted decimal notation Functions for converting between binary IP addresses and dotted decimal strings: –inet_aton : converts a dotted decimal string to an IP address in network byte order. –inet_ntoa : converts an IP address in network by order to its corresponding dotted decimal string. –“n” denotes network representation. “a” denotes application representation.

11. 11 IP Addresses 32-bit IP addresses are stored in an IP address struct –IP addresses are always stored in memory in network byte order (big-endian byte order) /* Internet address structure */ struct in_addr { unsigned int s_addr; /* network byte order (big-endian) */ };

12. 12 IP Addresses Handy network byte-order functions: –htonl: convert long int from host to network byte order. –htons: convert short int from host to network byte order. –ntohl: convert long int from network to host byte order. –ntohs: convert short int from network to host byte order.

13. IP Address Structure IP (V4) Address space divided into classes: Network ID Written in form w.x.y.z/n –n = number of bits in host address –E.g., CMU written as 128.2.0.0/16 Class B address Unrouted (private) IP addresses: 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 Class A Class B Class C Class D Class E 0 1 2 3 8 16 24 31 0 Net IDHost ID Net ID Multicast address Reserved for experiments 10 101 1101 1111 13

14. 14 miledugovcom cmuberkeleymit csece kittyhawk 128.2.194.242 cmcl unnamed root pdl imperial 128.2.189.40 amazon www 208.216.181.15 first-level domain names second-level domain names third-level domain names Internet Domain Names

15. 15 Domain Naming System (DNS) The Internet maintains a mapping between IP addresses and domain names in a distributed database called DNS. –Conceptually, we can think of the DNS database as being millions of host entry structures:

16. 16 Domain Naming System (DNS) /* DNS host entry structure */ struct hostent { /* official domain name of host */ char *h_name; /* null-terminated array of domain names */ char **h_aliases; /* host address type (AF_INET) */ int h_addrtype; /* length of an address, in bytes */ int h_length; /* null-terminated array of in_addr structs*/ char **h_addr_list; };

17. 17 Domain Naming System (DNS) Functions for retrieving host entries from DNS: –gethostbyname : query key is a DNS domain name –gethostbyaddr: query key is a an IP address

18. 18 hostname: a program that queries DNS int main(int argc, char **argv) { /* argv[1] is a domain name char **pp; * or dotted decimal IP addr */ struct in_addr addr; struct hostent *hostp; if (inet_aton(argv[1], &addr) != 0) hostp = Gethostbyaddr((const char *)&addr, sizeof(addr), AF_INET); else hostp = Gethostbyname(argv[1]); printf("official hostname: %s\n", hostp->h_name); for (pp = hostp->h_aliases; *pp != NULL; pp++) printf("alias: %s\n", *pp); for (pp = hostp->h_addr_list; *pp != NULL; pp++) { addr.s_addr = *((unsigned int *)*pp); printf("address: %s\n", inet_ntoa(addr)); }

19. 19 Networking Programming

20. 20 Internet connections Clients and servers communicate by sending streams of bytes of connections. –point-to-point, full-duplex, and reliable A connection is uniquely identified by the socket addresses of its endpoints (socket pair) –(cliaddr:cliport, servaddr:servport)

21. 21 Internet connections A port is a 16-bit integer that identifies a process: –Ephemeral port : assigned automatically on client when client makes a connection request –Well-known port : associated with some service provided by a server (e.g., port 80 is associated with Web servers)

22. Putting it all Together: Anatomy of an Internet Connection Connection socket pair (128.2.194.242:51213, 208.216.181.15:80) Server (port 80) Client Client socket address 128.2.194.242:51213 Server socket address 208.216.181.15:80 Client host address 128.2.194.242 Server host address 208.216.181.15 51213 is an ephemeral port allocated by the kernel 80 is a well-known port associated with Web servers 22

23. 23 To the kernel, a socket is an endpoint of communication To an application, a socket is a descriptor that lets an application read/write from/to the network. Remember: All Unix I/O devices, including networks, are modeled as files What is a socket?

24. 24 Clients and servers communicate with each other by reading from and writing to socket descriptors –Using regular Unix read and write I/O functions What is a socket? Client clientfd Server serverfd

25. 25 The main difference between file I/O and socket I/O is how the application “opens” the socket descriptors What is a socket?

26. 26 Overview of the Sockets Interface Client / Server Session socket bind listen rio_readlineb rio_writenrio_readlineb rio_writen Connection request rio_readlineb close EOF Await connection request from next client open_listenfd open_clientfd acceptconnect ClientServer

27. 27 Internet-style sockets are characterized by –a 32-bit IP address and a port Defined in /usr/include/netinet/in.h /* Internet address */ struct in_addr { unsigned int s_addr; /* 32-bit IP address */ }; Key data structures

28. 28 Defined in /usr/include/netdb.h /* Domain Name Service (DNS) host entry */ struct hostent { char *h_name; /* official name of host */ char **h_aliases; /* alias list */ int h_addrtype; /* host address type */ int h_length; /* length of address */ char **h_addr_list;/*list of addresses */ } Key data structures

29. Socket Address Structures Internet-specific socket address: –Must cast ( sockaddr_in * ) to ( sockaddr * ) for connect, bind, and accept 00000000 sa_family Family Specific struct sockaddr_in { unsigned short sin_family; /* address family (always AF_INET) */ unsigned short sin_port; /* port num in network byte order */ struct in_addr sin_addr; /* IP addr in network byte order */ unsigned char sin_zero[8]; /* pad to sizeof(struct sockaddr) */ }; sin_port AF_INET sin_addr sin_family 29

30. Overview of the Sockets Interface ClientServer socket bind listen Connection request open_listenfd open_clientfd acceptconnect 30

31. Echo Client: open_clientfd int open_clientfd(char *hostname, int port) { int clientfd; struct hostent *hp; struct sockaddr_in serveraddr; if ((clientfd = socket(AF_INET, SOCK_STREAM, 0)) < 0) return -1; /* check errno for cause of error */ /* Fill in the server's IP address and port */ if ((hp = gethostbyname(hostname)) == NULL) return -2; /* check h_errno for cause of error */ bzero((char *) &serveraddr, sizeof(serveraddr)); serveraddr.sin_family = AF_INET; bcopy((char *)hp->h_addr_list[0], (char *)&serveraddr.sin_addr.s_addr, hp->h_length); serveraddr.sin_port = htons(port); /* Establish a connection with the server */ if (connect(clientfd, (SA *) &serveraddr, sizeof(serveraddr)) < 0) return -1; return clientfd; } This function opens a connection from the client to the server at hostname:port Create socket Create address Establish connection 31

32. Echo Client: open_clientfd (socket) int clientfd; /* socket descriptor */ if ((clientfd = socket(AF_INET, SOCK_STREAM, 0)) < 0) return -1; /* check errno for cause of error */... creates a socket descriptor on the client –Just allocates & initializes some internal data structures –AF_INET: indicates that the socket is associated with Internet protocols –SOCK_STREAM: selects a reliable byte stream connection provided by TCP 32

33. 33 Echo client: open_clientfd()(connect) int clientfd; /* socket descriptor */ struct sockaddr_in serveraddr; /* server address */ typedef struct sockaddr SA; /* generic sockaddr */... /* Establish a connection with the server */ if (connect(clientfd, (SA *)&serveraddr, sizeof(serveraddr))< 0) return -1; return clientfd; }

34. 34 Then the client creates a connection with the server –The client process suspends (blocks) until the connection is created with the server. –At this point the client is ready to begin exchanging messages with the server via Unix I/O calls on the descriptor clientfd. Echo client: open_clientfd()(connect)

35. 35 Echo server: open_listenfd() int open_listenfd(int port) { int listenfd, optval=1; struct sockaddr_in serveraddr; /* Create a socket descriptor */ if ((listenfd = socket(AF_INET, SOCK_STREAM, 0)) < 0) return -1; /* Eliminates "Address already in use" error from bind. */ if (setsockopt(listenfd, SOL_SOCKET, SO_REUSEADDR, (const void *)&optval, sizeof(int)) < 0) return -1;...

36. 36 Echo server: open_listenfd() (cont)... /* Listenfd will be an endpoint for all requests to port on any IP address for this host */ bzero((char *) &serveraddr, sizeof(serveraddr)); serveraddr.sin_family = AF_INET; serveraddr.sin_addr.s_addr = htonl(INADDR_ANY); serveraddr.sin_port = htons((unsigned short)port); if (bind(listenfd, (SA *)&serveraddr, sizeof(serveraddr)) < 0) return -1; /* Make it a listening socket ready to accept connection requests */ if (listen(listenfd, LISTENQ) < 0) return -1; return listenfd; }

37. 37 Initialize socket with server port number Accept connection from any IP address Echo server: open_listenfd() struct sockaddr_in serveraddr; /* server's socket addr */... /* listenfd will be an endpoint for all requests to port on any IP address for this host */ bzero((char *) &serveraddr, sizeof(serveraddr)); serveraddr.sin_family = AF_INET; serveraddr.sin_port = htons((unsigned short)port); serveraddr.sin_addr.s_addr = htonl(INADDR_ANY);

38. 38 bind() associates the socket with the socket address we just created. Echo server: open_listenfd() (bind) int listenfd; /* listening socket */ struct sockaddr_in serveraddr; /* server’s socket addr */... /* listenfd will be an endpoint for all requests to port on any IP address for this host */ if (bind(listenfd, (SA *)&serveraddr, sizeof(serveraddr)) < 0) return -1;

39. 39 listen() indicates that this socket will accept connection ( connect ) requests from clients. LISTENQ is constant indicating how many pending requests allowed We’re finally ready to enter the main server loop that accepts and processes client connection requests. Echo server: open_listenfd (listen) int listenfd; /* listening socket */... /* Make it a listening socket ready to accept connection requests */ if (listen(listenfd, LISTENQ) < 0) return -1; return listenfd; }

40. 40 The server loops endlessly, waiting for connection requests, then reading input from the client, and echoing the input back to the client. main() { /* create and configure the listening socket */ while(1) { /* Accept(): wait for a connection request */ /* echo(): read and echo input line from client */ /* Close(): close the connection */ } Echo server: main loop

41. 41 accept() returns a connected socket descriptor ( connfd ) with the same properties as the listening descriptor ( listenfd) –Returns when connection between client and server is created and ready for I/O transfers. –All I/O with the client will be done via the connected socket. accept() also fills in client’s address. Echo server: accept()

42. Echo Server: accept Illustrated listenfd(3) Client 1. Server blocks in accept, waiting for connection request on listening descriptor listenfd clientfd Server listenfd(3) Client clientfd Server 2. Client makes connection request by calling and blocking in connect Connection request listenfd(3) Client clientfd Server 3. Server returns connfd from accept. Client returns from connect. Connection is now established between clientfd and connfd connfd(4) 42

43. Connected vs. Listening Descriptors Listening descriptor –End point for client connection requests –Created once and exists for lifetime of the server Connected descriptor –End point of the connection between client and server –A new descriptor is created each time the server accepts a connection request from a client –Exists only as long as it takes to service client Why the distinction? –Allows for concurrent servers that can communicate over many client connections simultaneously E.g., Each time we receive a new request, we fork a child to handle the request 43

44. Networking Programming --Web Server(I) 44

45. Clients and servers communicate using the HyperText Transfer Protocol (HTTP) –client and server establish TCP connection –Client requests content –Server responds with requested content –client and server close connection (eventually) Current version is HTTP/1.1 –RFC 2616, June, 1999. Web servers 45

46. Web servers Web server HTTP request HTTP response (content) Web client (browser) IP TCP HTTP Datagrams Streams Web content http://www.w3.org/Protocols/rfc2616/rfc2616.html 46

47. Web servers return content to clients –content: a sequence of bytes with an associated MIME (Multipurpose Internet Mail Extensions) type Web content 47

48. Example MIME types –text/htmlHTML page –text/plainUnformatted text –application/postscript Postcript document –image/gifBinary image encoded in GIF format –image/jpgBinary image encoded in JPG format Web content 48

49. The content returned in HTTP responses can be either static or dynamic –Static content: content stored in files and retrieved in response to an HTTP request Examples: HTML files, images, audio clips. Request identifies content file –Dynamic content: content produced on-the-fly in response to an HTTP request Example: content produced by a program executed by the server on behalf of the client. Request identifies file containing executable code Static and dynamic content Bottom line: All Web content is associated with a file that is managed by the server. 49

50. Each file managed by a server has a unique name called a URL (Universal Resource Locator) URLs for static content: –http://www.cs.cmu.edu:80/index.html –http://www.cs.cmu.edu/index.html –http://www.cs.cmu.edu identifies a file called index.html, managed by a Web server at www.cs.cmu.edu that is listening on port 80. URLs 50

51. URLs for dynamic content: –http://www.cs.cmu.edu:8000/cgi- bin/adder?15000&213 identifies an executable file called adder, managed by a Web server at www.cs.cmu.edu that is listening on port 8000, that should be called with two argument strings: 15000 and 213. URLs 51

52. Anatomy of an HTTP Transaction unix> telnet www.cmu.edu 80 Client: open connection to server Trying 128.2.10.162... Telnet prints 3 lines to the terminal Connected to www.cmu.edu. Escape character is '^]'. GET /index.shtml HTTP/1.1 Client: request line HOST: www.cmu.edu Client: required HTTP/1.1 HOST header Client: empty line terminates headers. HTTP/1.1 200 OK Server: responds with web page Date: Fri, 29 Oct 2010 19:41:08 GMT Server: Apache/1.3.39 (Unix) mod_pubcookie/3.3.3... Transfer-Encoding: chunked Content-Type: text/html... Lots of stuff Connection closed by foreign host. Server: closes connection unix> Client: closes connection and terminates 52

53. HTTP Requests HTTP request is a request line, followed by zero or more request headers Request line: – is HTTP version of request ( HTTP/1.0 or HTTP/1.1 ) – is typically a URL for proxies, a suffix for servers. A URL is a type of URI (Uniform Resource Identifier) See http://www.ietf.org/rfc/rfc2396.txt – is either GET, POST, OPTIONS, HEAD, PUT, DELETE, or TRACE. 53

54. HTTP Requests (cont) HTTP methods: –GET : Retrieve static or dynamic content Arguments for dynamic content are in URI Workhorse method (99% of requests) –POST : Retrieve dynamic content Arguments for dynamic content are in the request body –OPTIONS : Get server or file attributes –HEAD : Like GET but no data in response body –PUT : Write a file to the server! –DELETE : Delete a file on the server! –TRACE : Echo request in response body Useful for debugging. Request headers: : –Provide additional information to the server. 54

55. HTTP Responses HTTP response is a response line followed by zero or more response headers. Response line: – is HTTP version of the response. – is numeric status. – is corresponding English text. 200 OKRequest was handled without error 301MovedProvide alternate URL 403ForbiddenServer lacks permission to access file 404Not foundServer couldn’t find the file. Response headers: : –Provide additional information about response –Content-Type: MIME type of content in response body. –Content-Length: Length of content in response body. 55