1. Networking with Java

2. Introduction to Networking

3. Protocols Hi TCP connection request Hi TCP connection reply Got the time? GET http://www.cs.huji.ac.il/~dbi 2:00 time

4. Internet Architecture Model Application (HTTP, FTP) DATA Transport (TCP,UDP) HEADERDATA Network (IP) HEADER DATA Link (LINK) HEADER DATA

5. TCP ( Transmission-Control Protocol) Enables symmetric byte-stream transmission between two endpoints (applications) Reliable communication channel TCP perform these tasks: -connection establishment by handshake (relatively slow) -division to numbered packets (transferred by IP) -error correction of packets (checksum) -acknowledgement and retransmission of packets -connection termination by handshake

6. UDP (User Datagram Protocol) Enables direct datagram (packet) transmission from one endpoint to another No reliability (except for data correction) -sender does not wait for acknowledgements -arrival order is not guaranteed -arrival is not guaranteed Used when speed is essential, even in cost of reliability -e.g., streaming media, games, Internet telephony, etc.

7. Ports A computer may have several applications that communicate with applications on remote computers through the same physical connection to the network When receiving a packet, how can the computer tell which application is the destination? Solution: each channel endpoint is assigned a unique port that is known to both the computer and the other endpoint

8. Ports (cont) Thus, an endpoint application on the Internet is identified by -A host name → 32 bits IP-address -A 16 bits port Q: Why don’t we specify the port in a Web browser?

9. Known Ports Some known ports are -20, 21: FTP -23: TELNET -25: SMTP -110: POP3 -80: HTTP -119: NNTP 21 23 25 110 80 119 Client Application web browser mail client

10. Sockets A socket is a construct that represents one end-point of a two-way communication channel between two programs running on the network Using sockets, the OS provides processes a file-like access to the channel -i.e., sockets are allocated a file descriptor, and processes can access (read/write) the socket by specifying that descriptor A specific socket is identified by the machine's IP and a port within that machine

11. Sockets (cont) A socket stores the IP and port number of the other end- point computer of the channel When writing to a socket, the written bytes are sent to the other computer and port (e.g., over TCP/IP) -That is, remote IP and port are attached to the packets When OS receives packets on the network, it uses their destination port to decide which socket should get the received bytes

12. Client-Server Model A common paradigm for distributed applications Asymmetry in connection establishment: -Server waits for client requests (daemon) at a well known address (IP+port) -Connection is established upon client request Once the connection is made, it can be either symmetric (TELNET) or asymmetric (HTTP) For example: Web servers and browsers

13. Client-Server Interaction

18. Java Sockets Low-Level Networking

19. Java Sockets Java wraps OS sockets (over TCP) by the objects of class java.net.Socket new Socket(String remoteHost, int remotePort) creates a TCP socket and connects it to the remote host on the remote port (hand shake) Write and read using streams: - InputStream getInputStream() - OutputStream getOutputStream()

20. A Socket Example import java.net.*; import java.io.*; public class SimpleSocket { public static void main(String[] args) throws IOException {... next slide... }

21. Socket socket = new Socket("www.cs.huji.ac.il", 80); InputStream istream = socket.getInputStream(); OutputStream ostream = socket.getOutputStream(); String request = "GET /~dbi/admin.html HTTP/1.1\r\n" + "Host: www.cs.huji.ac.il\r\n" + "Connection: close\r\n\r\n"; ostream.write(request.getBytes()); byte[] response = new byte[4096]; int bytesRead = -1; while ((bytesRead = istream.read(response)) >= 0) { System.out.write(response, 0, bytesRead); } socket.close();

23. Java ServerSocket ServerSocket represents a socket that listens and waits for requests from clients Construction: - new ServerSocket(int port) -Why do want to specify the port? Listen and accept incoming connections - Socket accept() -returns a new socket (with a new port) for the new channel -blocks until connection is made

24. public class EchoServer { public static void main(String[] args) throws IOException { ServerSocket serverSocket = new ServerSocket(8000); Socket socket = null; while (true) { try {... next slide... } catch (IOException exp) {... } finally { try {if (!socket.isClosed()) socket.close(); } catch (IOException e) {} } }}}

25. socket = serverSocket.accept(); String clientName = socket.getInetAddress().getHostName(); BufferedReader reader = new BufferedReader( new InputStreamReader(socket.getInputStream())); PrintStream writer = new PrintStream(socket.getOutputStream()); writer.println("Hello " + clientName + "!"); writer.flush(); String lineRead = null; while ((lineRead = reader.readLine()) != null) { writer.println("You wrote: " + lineRead); writer.flush(); }

26. Accepting Connections Usually, the accept() method is executed within an infinite loop -i.e., while(true){...} Whenever accept() returns, a new thread is launched to handle that interaction Hence, the server can handle several requests concurrently

27. Timeout You can set timeout values to these blocking methods: - read() of Socket - accept() of ServerSocket Use the method setSoTimeout(milliseconds) If timeout is reached before the method returns, java.net.SocketTimeoutException is thrown

28. Java Sockets and HTTP

29. HTTP Message Structure A HTTP message has the following structure: Request/Status-Line \r\n Header1: value1 \r\n Header2: value2 \r\n... HeaderN: valueN \r\n \r\n Message-Body

30. Reading HTTP Messages Several ways to interpret the bytes of the body -Binary: images, compressed files, class files,... -Text: ASCII, Latin-1, UTF-8,... Commonly, applications parse the headers of the message, and process the body according to the information supplied by the headers -E.g., Content-Type, Content-Encoding, Transfer- Encoding

31. An Example

32. Parsing the Headers So how are the headers themselves are represented? Answer: The headers of a HTTP message must be in US-ASCII format (1 byte per character) However, in order to parse the headers, it may be impossible to use readLine() of BufferedReader, since the body should be later read as binary bytes Hence, we convert bytes to characters

33. Socket socket = new Socket(argv[0], 80); InputStream istream = socket.getInputStream(); OutputStream ostream = socket.getOutputStream(); String request = "GET / HTTP/1.1\r\n" + "Host: " + argv[0] + "\r\n" + "Connection: close\r\n\r\n"; ostream.write(request.getBytes()); StringBuffer headers = new StringBuffer(); int byteRead = 0; while ( !endOfHeaders(headers) && (byteRead = istream.read()) >= 0) { headers.append((char) byteRead); } System.out.print(headers); socket.close(); Example: Extracting the Headers

34. public static boolean endOfHeaders(StringBuffer headers) { int lastIndex = headers.length() - 1; if (lastIndex < 3 || headers.charAt(lastIndex) != '\n') return false; return ( headers.substring(lastIndex - 3, lastIndex + 1).equals("\r\n\r\n")); } Example: Extracting the Headers (cont)

35. Persistent Connections According to HTTP/1.1, a server does not have to close the connection after fulfilling your request One connection (socket) can be used for several requests and responses send more requests -even while earlier responses are being transferred (pipelining) Question: how can the client know when one response ends and a new one begins? To avoid persistency, require explicitly by the header: Connection: close

36. URL and URLConnection High-Level Networking

37. Protocol Host Name Port Number File Name Reference Working with URLs URL (Uniform Resource Locator): a reference (an address) to a resource on the Internet http://www.cs.huji.ac.il:80/~dbi/main.html#notes

38. The Class URL The class URL is used for parsing URLs Constructing URLs: - URL w3c1 = new URL("http://www.w3.org/TR/"); - URL w3c2 = new URL("http","www.w3.org",80,"TR/"); - URL w3c3 = new URL(w3c2, "xhtml1/"); If the string is not an absolute URL, then it is considered relative to the URL

39. Parsing URLs The following methods of URL can be used for parsing URLs getProtocol(), getHost(), getPort(), getPath(), getFile(), getQuery(), getRef(), getQuery()

40. The class URLConnection To establish the actual resource, we can use the object URLConnection obtained by url.openConnection() If the protocol of the URL is HTTP, the returned object is of class HttpURLConnection This class encapsulates all socket management and HTTP directions required to obtain the resource

41. public class ContentExtractor { public static void main(String[] argv) throws Exception { URL url = new URL(argv[0]); System.out.println("Host: " + url.getHost()); System.out.println("Protocol: " + url.getProtocol()); System.out.println("----"); URLConnection con = url.openConnection(); InputStream stream = con.getInputStream(); byte[] data = new byte[4096]; int bytesRead = 0; while((bytesRead=stream.read(data))>=0) { System.out.write(data,0,bytesRead); }}}

43. About URLConnection The life cycle of a URLConnection object has two parts: -Before actual connection establishment Connection configuration -After actual connection establishment Content retrieval Passage from the first phase to the second is implicit -A result of calling some committing methods, like getDate()

44. About HttpURLConnection The HttpURLConnection class encapsulates all HTTP transaction over sockets, e.g., -Content decoding -Redirection -Proxy indirection However, you can control requests by its methods - setRequestMethod, setFollowRedirects, setRequestProperty,...

45. Sending POST Requests In order to send POST requests with HttpURLConnection, you have to do the following: -Enable connection output: con.setDoOutput(true); -Get the output stream: con.getOutputStream() This changes the method from GET to POST -Write the message body into the stream -close the stream (important!)

46. URL url = new URL("http://find.walla.co.il/"); URLConnection connection = url.openConnection(); connection.setDoOutput(true); String query = "q=" + URLEncoder.encode(args[0], "UTF-8"); OutputStream out = connection.getOutputStream(); out.write(query.getBytes()); out.close(); InputStream in = connection.getInputStream(); int bytesRead = -1; byte[] response = new byte[4096]; while ((bytesRead = in.read(response)) >= 0) System.out.write(response, 0, bytesRead); in.close();

47. POST / HTTP/1.1 User-Agent: Java/1.5.0_01 Host: find.walla.co.il Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 Connection: keep-alive Content-type: application/x-www-form-urlencoded Content-Length: 18 q=Java+networking java SearchWalla "Java networking" Sent Headers

48. URLEncoder Contains a utility method encode for converting a string into an encoded format (used in URLs) To convert a string, each character is examined in turn: -Space is converted into a plus sign + - a - z, A - Z, 0 - 9,., -, * and _ remain the same. -The bytes of all special characters are replaced by hexadecimal numbers, preceded with % To decode an encoded string, use decode() of the class URLDecoder

49. Defining Default Proxy For reading a URL using a proxy, we run java with the environment variables for http.proxyHost and http.proxyPort set properly: java -DproxyHost=wwwproxy.huji.ac.il –DproxyPort=8080... Another option is to set the environment variables in the program itself System.getProperties().put( "proxyHost", "wwwproxy.huji.ac.il" ); System.getProperties().put( "proxyPort", "8080" );

50. Not Covered: UDP Connections The URL, URLConnection, Socket and ServerSocket classes all use the TCP protocol to communicate over the network Java programs can also use UDP protocol for communication For that, use DatagramPacket, DatagramSocket, and MulticastSocket classes