1. 1 2: Application Layer Programming application protocols Goal: learn how to build client/server application that communicate using sockets without understanding underlying functions of TCP/IP  Many possible programming interfaces Socket APIs (most common) –BSD C Socket API (most common) –Java socket API –Python socket API Other APIs –Client-side Java URLconnections –Server-side Java servlets –Python urllib –Python HTTPServer –RPC, CORBA, Java RMI (not covered)

2. 2 2: Application Layer Socket programming Socket API r introduced in BSD4.1 UNIX, 1981 r client/server paradigm r two types of transport service via socket API:  unreliable datagram  reliable, byte stream-oriented a host-local, application-created, OS-controlled interface (a “door”) into which application process can both send and receive messages to/from another application process socket

3. 3 2: Application Layer Socket-programming using TCP Socket: a door between application process and end- end-transport protocol (UCP or TCP) TCP service: reliable transfer of bytes from one process to another process TCP with buffers, variables socket controlled by application developer controlled by operating system host or server process TCP with buffers, variables socket controlled by application developer controlled by operating system host or server internet

4. 4 2: Application Layer Socket programming using TCP Server setup r server process must first be running r server must have created a “listening” socket (door) that welcomes client’s contact Client contacts server by: r creating client-local TCP socket r specifying IP address, port number of server process r When client connects a socket: client TCP establishes connection to server TCP r When client connects, server TCP creates new socket for it to communicate with client  different than listening socket (allows server to talk with multiple clients)  source IP and port numbers used to distinguish b/w clients (more in Chap 3) TCP provides reliable, in-order transfer of bytes (“pipe”) between client and server application viewpoint

5. 5 2: Application Layer Addressing processes w/ port numbers r For a process to receive messages, it must have a unique identifier r Each host has a unique32- bit IP address r Q: Does the IP address of the host on which the process runs suffice for identifying the process?  A: No, many processes can be running on same host r Identifier includes both the IP address and port numbers associated with the process on the host. r Example port numbers:  HTTP server: 80  Mail server: 25

6. 6 2: Application Layer TCP sockets in action *,SIP:80 CIP:1099,SIP:80*,SIP:80CIP:1099,SIP:80

7. 7 2: Application Layer TCP sockets in action *,SIP:80CIP:1099,SIP:80 CIP:1100,SIP:80 *,SIP:80CIP:1099,SIP:80 CIP:1100,SIP:80

8. 8 2: Application Layer BSD sockets in C/Unix r Socket API (socket.h) socket(): create unnamed socket (data structure) –UDP (SOCK_DGRAM), TCP (SOCK_STREAM) –IP (SOCK_RAW) bind(): name socket (bind local source port to socket) listen(): enable socket to accept connections accept(): get connect() request from listen queue, allocate file descriptor for new socket connect(): initiate connection on a socket (TCP handshake) send(), sendto(), sendmsg(), writev(), write(): send data recv(), recvfrom(), recvmsg(), readv(), read(): receive data setsockopt(), getsockopt(): set socket options (such as buffer sizes, flag fields) close(), shutdown(): teardown connection

9. 9 2: Application Layer BSD TCP sockets in action connect() socket() bind() server client listen() accept() write() read() write() read()

10. 10 2: Application Layer Simple C socket example r http://www.thefengs.com/wuchang/cou rses/cs594/socket_example http://www.thefengs.com/wuchang/cou rses/cs594/socket_example r TCP socket code  Client tcpcli.c  Server tcpserv.c

11. 11 2: Application Layer Java network programming Java network applications java.net package System-dependent implementations

12. 12 2: Application Layer Java installation on linuxlab r J2SE  javac java compiler  java java interpreter  http://docs.oracle.com/javase/8/ http://docs.oracle.com/javase/8/

13. 13 2: Application Layer java.net classes r Low-level networking classes  Sockets and Packets  java.net.Socket  java.net.ServerSocket  java.net.DatagramSocket  java.net.DatagramPacket  java.net.InetAddress r High-level URL networking classes  java.net.URL  java.net.URLConnection java.net.HttpURLConnection  java.net.URLencoder

14. 14 2: Application Layer java.net.Socket r Constructors  Socket(InetAddress, int)  Socket(String, int)  Socket(InetAddress, int, InetAddress, int) r Some methods  getInputStream()  getOutputStream()  getInetAddress()  getPort()  getLocalAddress()  getLocalPort()  get/set individual socket options

15. 15 2: Application Layer java.net.ServerSocket r Constructors  ServerSocket(int) // port specified  ServerSocket(int, int) // port, backlog specified  ServerSocket(int, int, InetAddress) // port, backlog, IP address r Some methods  accept()  getInetAddress()  getLocalPort()

16. 16 2: Application Layer Stream jargon for Java network programming r A stream is a sequence of characters that flow into or out of a process. r An input stream is attached to some input source for the process, e.g., keyboard or socket. r An output stream is attached to an output source, e.g., monitor or socket.

17. 17 2: Application Layer Socket programming with TCP Example client-server app: 1) client reads line from standard input ( inFromUser stream), sends to server via socket ( outToServer stream) 2) server reads line from socket 3) server converts line to uppercase, sends back to client 4) client reads, prints modified line from socket ( inFromServer stream) Client process client TCP socket

18. 18 2: Application Layer Client/server socket interaction: TCP wait for incoming connection request connectionSocket = welcomeSocket.accept() create socket, port=x, for incoming request: welcomeSocket = ServerSocket() create socket, connect to hostid, port=x clientSocket = Socket() close connectionSocket read reply from clientSocket close clientSocket Server (running on hostid ) Client send request using clientSocket read request from connectionSocket write reply to connectionSocket TCP connection setup

19. 19 2: Application Layer Example: Java client (TCP) import java.io.*; import java.net.*; class TCPClient { public static void main(String argv[]) throws Exception { String sentence; String modifiedSentence; BufferedReader inFromUser = new BufferedReader(new InputStreamReader(System.in)); Socket clientSocket = new Socket("hostname", 6789); DataOutputStream outToServer = new DataOutputStream(clientSocket.getOutputStream()); Create input stream Create client socket, connect to server Create output stream attached to socket

20. 20 2: Application Layer Example: Java client (TCP), cont. BufferedReader inFromServer = new BufferedReader(new InputStreamReader(clientSocket.getInputStream())); sentence = inFromUser.readLine(); outToServer.writeBytes(sentence + '\n'); modifiedSentence = inFromServer.readLine(); System.out.println ("FROM SERVER: " + modifiedSentence ); clientSocket.close(); } Create input stream attached to socket Send line to server Read line from server

21. 21 2: Application Layer Example: Java server (TCP) import java.io.*; import java.net.*; class TCPServer { public static void main(String argv[]) throws Exception { String clientSentence; String capitalizedSentence; ServerSocket welcomeSocket = new ServerSocket(6789); while(true) { Socket connectionSocket = welcomeSocket.accept(); BufferedReader inFromClient = new BufferedReader(new InputStreamReader(connectionSocket.getInputStream())); Create welcoming socket at port 6789 Wait, on welcoming socket for contact by client Create input stream, attached to socket

22. 22 2: Application Layer Example: Java server (TCP), cont DataOutputStream outToClient = new DataOutputStream (connectionSocket.getOutputStream()); clientSentence = inFromClient.readLine(); capitalizedSentence = clientSentence.toUpperCase() + '\n'; outToClient.writeBytes(capitalizedSentence); } Read in line from socket Create output stream, attached to socket Write out line to socket End of while loop, loop back and wait for another client connection

23. 23 2: Application Layer Socket programming using UDP UDP: no “connection” between client and server r no handshaking r sender explicitly attaches IP address and port of destination to each packet r server must extract IP address, port of sender from received packet UDP: transmitted data may be received out of order, or lost application viewpoint UDP provides unreliable transfer of groups of bytes (“datagrams”) between client and server

24. 24 2: Application Layer BSD UDP sockets in action server client bind() socket() recvfrom() sendto() bind() recvfrom() sendto()

25. 25 2: Application Layer BSD example r http://www.thefengs.com/wuchang/cou rses/cs594/socket_example http://www.thefengs.com/wuchang/cou rses/cs594/socket_example r UDP socket code  Client udpcli.c  Server udpserv.c

26. 26 2: Application Layer java.net.DatagramSocket r Constructors  DatagramSocket()  DatagramSocket(int) // bind to specific port  DatagramSocket(int, InetAddress) // specify local address r Some methods  getLocalAddress()  getLocalPort()  receive(DatagramPacket)  send(DatagramPacket)  get/set individual socket options

27. 27 2: Application Layer java.net.DatagramPacket r Constructors  DatagramPacket(byte[], int) // receiving packets  DatagramPacket(byte[], int, InetAddress, int) // sending packets r Some methods  getAddress() // remote address  getPort() // remote port  getLength() // get packet length  getData() // return data received or to be sent  setAddress(InetAddress) // set remote address  setData(byte[]) // set packet data  setLength(int) // set packet length  setPort(int) // set remote port

28. 28 2: Application Layer Client/server socket interaction: UDP close clientSocket Server (running on hostid ) read reply from clientSocket create socket, clientSocket = DatagramSocket() Client Create, address (hostid, port=x, send datagram request using clientSocket create socket, port=x, for incoming request: serverSocket = DatagramSocket() read request from serverSocket write reply to serverSocket specifying client host address, port number

29. 29 2: Application Layer Example: Java client (UDP) Output: sends packet (recall that TCP sent “byte stream”) Input: receives packet (recall that TCP received “byte stream”) Client process client UDP socket

30. 30 2: Application Layer Example: Java client (UDP) import java.io.*; import java.net.*; class UDPClient { public static void main(String args[]) throws Exception { BufferedReader inFromUser = new BufferedReader(new InputStreamReader(System.in)); DatagramSocket clientSocket = new DatagramSocket(); InetAddress IPAddress = InetAddress.getByName("hostname"); byte[] sendData = new byte[1024]; byte[] receiveData = new byte[1024]; String sentence = inFromUser.readLine(); sendData = sentence.getBytes(); Create input stream Create client socket Translate hostname to IP address using DNS

31. 31 2: Application Layer Example: Java client (UDP), cont. DatagramPacket sendPacket = new DatagramPacket(sendData, sendData.length, IPAddress, 9876); clientSocket.send(sendPacket); DatagramPacket receivePacket = new DatagramPacket(receiveData, receiveData.length); clientSocket.receive(receivePacket); String modifiedSentence = new String(receivePacket.getData()); System.out.println("FROM SERVER:" + modifiedSentence); clientSocket.close(); } Create datagram with data-to-send, length, IP addr, port Send datagram to server Read datagram from server

32. 32 2: Application Layer Example: Java server (UDP) import java.io.*; import java.net.*; class UDPServer { public static void main(String args[]) throws Exception { DatagramSocket serverSocket = new DatagramSocket(9876); byte[] receiveData = new byte[1024]; byte[] sendData = new byte[1024]; while(true) { DatagramPacket receivePacket = new DatagramPacket(receiveData, receiveData.length); serverSocket.receive(receivePacket); Create datagram socket at port 9876 Create space for received datagram Receive datagram

33. 33 2: Application Layer Example: Java server (UDP), cont String sentence = new String(receivePacket.getData()); InetAddress IPAddress = receivePacket.getAddress(); int port = receivePacket.getPort(); String capitalizedSentence = sentence.toUpperCase(); sendData = capitalizedSentence.getBytes(); DatagramPacket sendPacket = new DatagramPacket(sendData, sendData.length, IPAddress, port); serverSocket.send(sendPacket); } Get IP addr port #, of sender Write out datagram to socket End of while loop, loop back and wait for another datagram Create datagram to send to client

34. 34 2: Application Layer High-level Java networking classes r Socket/Packet  Low level building blocks  Must implement all application-level logic r Many protocols based on URLs and/or tunneled in HTTP  Program at a higher-level to hide underlying protocol details  Do not re-implement HTTP, URL parsing, MIME handling for each application

35. 35 2: Application Layer High-level client-side Java networking classes r java.net.URL  Represent a URL object r java.net.URLConnection  Represent a connection to a URL which can be read and written from r java.net.HttpURLConnection  Subclass of URLConnection for http:// URLs

36. 36 2: Application Layer Java high-level client-side networking classes example r http://www.thefengs.com/wuchang/courses /cs594/java_example http://www.thefengs.com/wuchang/courses /cs594/java_example import java.net.*; import java.io.*; public class Test { public static void main(String argv[]) { try { URL u=new URL(http://www.google.com/);http://www.google.com/ URLConnection uc=u.openConnection(); Object o = (Object) uc.getContent(); } catch (Exception e) { }

37. 37 2: Application Layer High-level server-side Java networking classes r Servlets  Dynamically generate content  Implement common protocol header logic Example http servlets –Cookies –Content-type –Content-length r Servlet classes  javax.servlet.Servlet javax.servlet.HttpServlet init() service() destroy()  javax.servlet.ServletRequest javax.servlet.ServletResponse  javax.servlet.HttpServletRequest javax.servlet.HttpServletResponse

38. 38 2: Application Layer Python network programming Python network applications System-dependent implementations Python network packages (socket, URLlib, HTTPServer)

39. 39 2: Application Layer Python network programming r Python  Scripting language  No compilation required  Language reference: http://www.python.orghttp://www.python.org  Provides APIs similar to Java socket –Low-level socket interface urllib –HTTP client SimpleHTTPServer –HTTP server

40. 40 2: Application Layer Python sockets r Similar to C and Java  Client import socket host = “localhost” port = 7 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.connect((host,port)) s.send(“some data to echo”) print s.recv(20) s.close

41. 41 2: Application Layer Python sockets r Similar to C and Java  Server import socket host = “” port = 7 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.bind((host,port)) s.listen(1) while (1): conn, addr = s.accept() data = conn.recv(20) conn.send(data) conn.close()

42. 42 2: Application Layer Python urllib/urllib2 r Client-side HTTP code similar to Java’s java.net.URLConnection  Hides socket creation, HTTP request formatting, HTTP response parsing url = sys.argv[1] sock = urllib.urlopen(url) htmlSource = sock.read() sock.close() print htmlSource url = sys.argv[1] txdata = None txheaders = { ‘Accept-Language’: ‘en-us’ } req = urllib2.Request(url, txdata, txheaders) u = urlib2.urlopen(req) headers = u.info() print u.read() Python urllib Python urllib2

43. 43 2: Application Layer Python BasicHTTPServer r Server-side HTTP processing libraries similar to javax.servlet.HTTPServlet  Hides socket creation, HTTP request parsing, HTTP response formatting  SocketServer.TCPServer.BaseHTTPServer

44. 44 2: Application Layer Python network programming URLs r http://docs.python.org/lib/internet.html r http://docs.python.org/lib/module-socket.html r http://docs.python.org/lib/module-urllib.html r http://docs.python.org/lib/module-urllib2.html r http://docs.python.org/lib/module- SimpleHTTPServer.html r http://docs.python.org/lib/module- BaseHTTPServer.html r http://www.w3journal.com/6/s3.vanrossum.html

45. 45 2: Application Layer Programming your projects r Multiple clients per server r Two options  Single threaded server that manages all client connections simultaneously  Multi-threaded server with one thread per client connection

46. 46 2: Application Layer Source code examples r http://www.thefengs.com/wuchang/courses /cs594/select_example http://www.thefengs.com/wuchang/courses /cs594/select_example  Manage I/O across multiple file descriptors simultaneously Can be blocking (wait until any FD has event) Can be non-blocking (return immediately with status of which FDs need service, if any)  May use poll() instead of select() as well Provides the same functionality with a different interface (fd_set bit mask versus pollfd structures) Some systems map them to one another –System V => select() is a wrapper to poll() –BSD => poll() is a wrapper to select()