. Computer Networks Application Layer

Chapter 2 Outline 2.1 Principles of app layer protocols clients and servers app requirements 2.2 Web and HTTP 2.3 FTP 2.4 Electronic Mail SMTP, POP3, IMAP 2.5 DNS 2.6 Socket programming with TCP 2.7 Socket programming with UDP 2.8 Building a Web server 2.9 Content distribution Network Web caching Content distribution networks P2P file sharing

Network Applications within same host, two processes communicate using inter-process communication (defined by OS). processes running in different hosts communicate with an application-layer protocol implements user interface & application-level protocol Web: browser E-mail: mail reader streaming audio/video: media player user agent: interfaces with user “above” and network “below”. Process: program running within a host.

Network Application Architectures Client-Server Architecture P2P Architecture Hybrid of the Client-Server and P2P Architecture.

Client-Server Paradigm Typical network app has two pieces: client and server Client: initiates contact with server (“speaks first”) typically requests service from server, Web: client implemented in browser; e-mail: in mail reader client process application transport network data link physical request reply Server: provides requested service to client e.g., Web server sends requested Web page, mail server delivers e-mail Server process application transport network data link physical

Pure P2P architecture Arbitrary end systems directly communicate Peers are intermittently connected and change IP addresses Example: Gnutella Highly scalable But difficult to manage

Applications and Application-Layer Protocols Application: communicating, distributed processes e.g., e-mail, Web, P2P file sharing, instant messaging running in end systems (hosts) exchange messages to implement application Application-layer protocols one “piece” of an app define messages exchanged by apps and actions taken use communication services provided by lower layer protocols. Server process application transport network data link physical Server process application transport network data link physical Server process application transport network data link physical

Application-Layer Protocols Types of messages exchanged: request & response messages Syntax of message types: what fields in messages & how fields are delineated Semantics of the fields: meaning of information in fields Rules for when and how processes send & respond to messages Public-domain protocols: defined in RFCs allows for interoperability eg, HTTP, SMTP Proprietary protocols: eg, Napster, KaZaA, …

Process Identifier For a process to receive messages, it must have an identifier Every host has a unique 32-bit IP address Q: does the IP address of the host on which the process runs suffice for identifying the process? Answer: No, many processes can be running on same host Identifier includes both the IP address and port numbers associated with the process on the host. Example port numbers: HTTP server: 80 Mail server: 25 Example IP address: 113.45.12.201 Example Identifier: (80; 113.45.12.201)

Chapter 2 Outline 2.2 Web and HTTP 2.1 Principles of app layer protocols clients and servers app requirements 2.2 Web and HTTP 2.3 FTP 2.4 Electronic Mail SMTP, POP3, IMAP 2.5 DNS 2.6 Socket programming with TCP 2.7 Socket programming with UDP 2.8 Building a Web server 2.9 Content distribution Network Web caching Content distribution networks P2P file sharing

What is a Web Browser It is a software application that enables a user to display and interact with html, text, images, audio, and other information typically located on a web page at a web server. Text and images on a web page can contain hyperlinks to other web pages at the same or different websites. Web browsers allow a user to quickly and easily access information provided on many web pages at many websites by traversing these links. Web browser hides http underneath a nice GUI. Browser must handle server error messages in some elegant way.

Web Browsers Mozilla Firefox AOL Netscape Microsoft Internet Explorer Google Chrome AOL Netscape Opera Apple Safari Flock

What is a Web Server? The term Web server can mean one of two things: A computer or A program that is: responsible for accepting HTTP requests from clients, which are known as Web browsers, and serving them HTTP responses along with optional data contents, which usually are Web pages such as HTML documents and linked objects (images, etc.).

Modern Web Servers On client side, higher-end client capabilities: Java applets DHTML (css, xml, javascript, etc). Increased form support in http multimedia (flash, etc.) On server side, this led to: dynamic web pages – asp, jsp, servlets, Cold Fusion, PHP, etc. improvements in cgi

Market Share for Top Servers Across All Domains Source: http://news.netcraft.com/

Total Sites Across All Domain , ,

URIs, URLs and URNs Uniform Resource Identifier (URI = URL or URN) Generic term for all textual names/addresses Uniform Resource Locator (URL) The set of URI schemes that have explicit instructions on how to access the resource over the Internet, e.g. http, ftp, gopher Uniform Resource Name (URN) is location-independent resource identifier urn: ietf: rfc:3187 urn: isbn: 0451450523

Web Web page consists of objects Object can be HTML file, JPEG image, Java applet, audio file,… Web page consists of base HTML-file which includes several referenced objects Each object is addressable by a URL Example URL: http://www.iust.ac.ir/computer/home.html host name path name

HTTP Overview HTTP: hypertext transfer protocol Web’s app layer protocol client/server model client: browser that requests, receives, “displays” Web objects server: Web server sends objects in response to requests HTTP 1.0: RFC 1945 HTTP 1.1: RFC 2068 Server running Apache Web server HTTP request PC running Explorer HTTP response HTTP request Storage contents: base files, objects HTTP response Mac running Navigator

HTTP Overview (cont.) HTTP is “stateless” Uses TCP: aside server maintains no information about past client requests Uses TCP: client initiates TCP connection (creates socket) to server, port 80 server accepts TCP connection from client HTTP messages (application-layer protocol messages) exchanged between browser (HTTP client) and Web server (HTTP server) TCP connection closed aside Protocols that maintain “state” are complex! past history (state) must be maintained if server/client crashes, their views of “state” may be inconsistent, must be reconciled

HTTP Connections Nonpersistent HTTP At most one object is sent over a TCP connection. HTTP/1.0 uses nonpersistent HTTP Persistent HTTP Multiple objects can be sent over single TCP connection between client and server. HTTP/1.1 uses persistent connections in default mode

Non-persistent HTTP Suppose user enters URL www.iust.ac.ir/computer/home.html (contains text, references to 10 jpeg images) 1. HTTP client initiates TCP connection to HTTP server (process) at www.iust.ac.ir on port 80 2. HTTP server at host www.iust.ac.ir waiting for TCP connection at port 80. “accepts” connection, notifying client 3. HTTP client sends HTTP request message (containing URL) into TCP connection socket. Message indicates that client wants object Computer//home.html 4. HTTP server receives request message, forms response message containing requested object, and sends message into its socket Next slide time

Non-persistent HTTP (cont.) 4. … 5. HTTP client receives response message containing html file, displays html. Parsing html file, finds 10 referenced jpeg objects 6. HTTP server closes TCP connection. 7. Steps 1-6 repeated for each of 10 jpeg objects time

Non-Persistent &Parallel Connections Clients creates first TCP connection and then ask for the base file. Client creates several TCP connections with the server(s) simultaneously. Client requests one object on each connection. Each TCP connection is terminated when the requested object is retrieved.

Persistent HTTP Without Pipeline1 initiate TCP connection RTT Request base (html) file RTT time to transmit base file Base file received Request first object time to transmit first object First object received Request second object Second object received time Definition of RTT: time to send a small packet to travel from client to server and back.

Response Time Model (Per.-no pip.) one RTT to initiate TCP connection one RTT for HTTP request and first few bytes of HTTP response to return Time ≈ (N+2)RTT + (N+1) file transmit time N = number of objects in the web page. initiate TCP connection RTT request base file time to transmit base file RTT base file received time

Persistent HTTP With Pipeline default in HTTP/1.1 initiate TCP connection RTT Request base (html) file RTT time to transmit base file Base file received 10 2 1- Request first object 2- Request second object … 10- Tenth second object 1 time to transmit first object First object received Tenth object received time Response Time ≈ 3RTT+ (N+1) file transmit time

HTTP Request Message: General Format request line method SP URL version CR LF header field name : value CR LF --- - http Header header lines CR LF Entity Body (Application Message, Object, …)

HTTP Response Message: General Format status line version SP status code phrase CR LF header field name : value CR LF --- - http Header header lines CR LF Entity Body (Application Message, Object, …)

HTTP Request Message HTTP request message: ASCII (human-readable format) request line (GET, POST, HEAD commands) GET /somedir/page.html HTTP/1.1 Host: www.someschool.edu User-agent: Mozilla/5.0 Connection: close Accept-language:fr If-modified-since:Sun,27 Sep 2009 13:45:12 (carriage return, line feed) header lines Carriage return, line feed indicates end of message

HTTP Response Message status line (protocol status code status phrase) HTTP/1.1 200 OK Connection: close Date: Thu,15 Oct 2009 12:00:15 GMT Server: Apache/1.3.0 (Unix) Last-Modified: Mon, 12 Oct 2009 …... Content-Length: 6821 Content-Type: text/html data data data data data ... header lines MIME lines data, e.g., requested HTML file

GET and HEAD Commands GET HEAD The GET method means retrieve whatever information (in the form of an entity) is identified by the Request-URI. The semantics of the GET method changes to a "conditional GET" if the request message includes an If-Modified-Since header field. HEAD The HEAD method is identical to GET except that the server must not return any Entity-Body in the response. There is no"conditional HEAD"request analogous to the conditional GET.

Conditional GET: Client-Side Caching server Goal: don’t send object if client has up-to-date cached version client: specify date of cached copy in HTTP request If-modified-since: <date> server: response contains no object if cached copy is up-to-date: HTTP/1.0 304 Not Modified HTTP request msg If-modified-since: <date> object not modified HTTP response HTTP/1.0 304 Not Modified HTTP request msg If-modified-since: <date> object modified HTTP response HTTP/1.0 200 OK <data>

POST Command POST The POST method is used to request that the destination server accept the entity enclosed in the request as a new subordinate of the resource identified by the Request-URL in the Request-Line. POST is designed to allow a uniform method to cover the following functions: Annotation of existing resources; Posting a message to a bulletin board, newsgroup, mailing list,or similar group of articles; Providing a block of data, such as the result of submitting a form, to a data-handling process; Extending a database through an append operation.

Uploading Form Input Get method: Web page often includes form input Input is uploaded to server in URL field of request line: URL encoding A-Z A-Z a-z a-z 0-9 0-9 *-_ *-_ Space + Others %HEX www.somesite.com/cgi.bin/form.cgi ?firstname=morteza&lastnme=analoui URL Encoded data Post method: Web page often includes form input Input is uploaded to server in entity body

Method Types HTTP/1.0 GET POST HEAD HTTP/1.1 GET, POST, HEAD PUT asks server to leave requested object out of response HTTP/1.1 GET, POST, HEAD PUT uploads file in entity body to path specified in URL field DELETE deletes file specified in the URL field Other Methods OPTIONS, PATCH, COPY, MOVE, DELETE, LINK, UNLINK, TRACE, WRAPPED, extension method.

Header Fields General Header Fields Request Header Fields Response Header Fields Entity Body Hearer Fields

Header Fields - 1 1. General Header Fields Keep-Alive Upgrade Connection Forwarded Cache-Control Data Pragma

Header Fields - 2 2. Request Header Fields Accept: Accept-Language: Authorization: From: Host: If-Modified-Since: Proxy-Authorization: Range: Refer: Unless: User-Agent: Accept-Encoding: Cookie:

Header Fields - 3 3. Response Header Fields Location: Proxy-Authentication: Public: Retry-After: Server: WWW-Authenticate: Set-Cookie:

Header Fields - 4 4. Entity Body Header Fields Allow Content-Encoding Content-Language Content-Length Content-Range Content-Type Content-Version Content-MD5 Derived-From Expires Last-Modified Link Title Transfer-Encoding URL-Header extension-header

Header Fields – 4 (Content-Type) Multipurpose Internet Mail Extensions (rfcs822obs, 2045, 2046, 2077) Type Application Text Image Audio Multipart Video Subtypes Postscript, msword, Msexel,… Html, plain(ASCII),… Jpeg, gif, ief,… Basic, x-pn-realaudio,… mixed mpeg, quicktime

HTTP Response Status Codes In first line in server -> client response message. A few sample codes: 200 OK request succeeded, requested object later in this message 301 Moved Permanently requested object moved, new location specified later in this message (Location:) 400 Bad Request request message not understood by server 404 Not Found requested document not found on this server

Status Codes Categories Response Categories Informational :100 Successful :200 Redirection :300 Client Error :400 (eg. 404 Not found) Server Error :500

Status Codes Status-Codes "201" ; Created "202" ; Accepted "204" ; No Content "301" ; Moved Permanently "200" ; OK "302" ; Moved Temporarily "304" ; Not Modified "400" ; Bad Request "401" ; Unauthorized "403" ; Forbidden "404" ; Not Found "500" ; Internal Server Error "501" ; Not Implemented "502" ; Bad Gateway "503" ; Service Unavailable “505” ; HTTP Version Not Supported

Trying out HTTP (client side) for Yourself 1. Telnet to your favorite Web server: telnet www.eurecom.fr 80 Opens TCP connection to port 80 (default HTTP server port) at www.eurecom.fr. Anything typed in sent to port 80 at www.eurecom.fr 2. Type in a GET HTTP request: By typing this in (hit carriage return twice), you send this minimal (but complete) GET request to HTTP server GET /~ross/index.html HTTP/1.0 3. Look at response message sent by HTTP server!

User-Server Interaction: Authorization Authorization : control access to server content authorization credentials: typically name, password stateless: client must present authorization in each request authorization: header line in each request if no authorization: header, server refuses access, sends WWW authenticate: header line in response client server usual http request msg 401: authorization req. WWW authenticate: usual http request msg + Authorization: <cred> usual http response msg usual http request msg + Authorization: <cred> time usual http response msg

Observing HTTP in action WireShark (Ethereal) packet-sniffer http://www.wireshark.org/ captures, records link-layer frames being sent/received recall: HTTP encapsulated inside TCP inside IP inside Ethernet (see chapter 6)

Ethereal Screen Shot

Cookies: Keeping “State”-RFC2109 Some Web sites store information in a small text file on your computer. This file is called a cookie. There are several types of cookies, you can choose whether to allow some, none, or all of them to be saved on your computer. If you do not allow cookies at all, you may not be able to view some Web sites or take advantage of customization features (such as local news and weather, or stock quotes).

Cookies and Privacy What cookies can bring: authorization aside What cookies can bring: authorization shopping carts recommendations user session state (Web e-mail) Cookies and privacy: cookies permit sites to learn a lot about you you may supply name and e-mail to sites search engines use redirection & cookies to learn yet more advertising companies obtain info across sites

Anatomy of a (Simple) Cookie String of text with these 6 attributes: Name of the cookie Value (content) of the cookie Site (host) for which the cookie is valid Path for which the cookie is valid Whether a secure connection needed to use the cookie. An expiration date A Cookie may not exceed 4 KB in size.

How cookies are used Created by an Internet site to store information on your computer. Example, you visit a Web site, it saves a record of the pages you looked at, to help the site customize the view for you the next time you visit. The file is a text file, usually stored in directory named Cookies and having username@WebServerName[1].txt name. Cookies can also store personally identifiable information, that can be used to identify or contact you, such as your name, e-mail address, or telephone number. A Web site only has access to the information you provide. Once a cookie is saved on your computer, only the Web site that created the cookie can read it.

HTTP and Cookies Many major Web sites use cookies There are four components: 1) cookie header line in the HTTP response message SET-COOKIE: (name, value, expires, path, domain, secure) 2) cookie header line in HTTP request message COOKIE: name 3) cookie file kept on user’s host and managed by user’s browser 4) back-end database at Web site

A Simple Example client server server creates ID 1678 for user Cookie file ebay: 8734 usual http request msg server creates ID 1678 for user usual http response + Set-cookie: 1678 entry in backend database Cookie file usual http request msg cookie: 1678 cookie- specific action amazon: 1678 ebay: 8734 access usual http response msg one week later: access usual http request msg cookie: 1678 Cookie file cookie- specific action amazon: 1678 ebay: 8734 usual http response msg

Cookie Example - 1 Name: Lang Content: en Host: webmail.iust.ac.ir Useranaloui@iust.ac.irwebmail.iust.ac.ir/15362988154882988652277920158429813096*Langenwebmail.iust.ac.ir/153634615495682987529279016025629801864*Themefarsiwebmail.iust.ac.ir/153634615495682987529279016025629801864* Name: Lang Content: en Host: webmail.iust.ac.ir Path: / Send For: Any type of connection (secure and no-secure) Expires: Monday, July 16, 2007 12:00:55 AM

Cookie Example - 2 SRCHUIDV=1&GUID=2A674B205B514D43BF918B49294F41B5search.msn.com/10247911939843045869690096649629869799*AFORMMSNHsearch.msn.com/102479119398430458696154618190429869801*SRCHUSRAUTOREDIR=0&GEOVAR=1&DOB=20070715search.msn.com/102440699493123060405390127649629869799* vid213.160.98.227.1198190542941908shockwave.com/1536226640563230048508249129052829901657* Apache3f07293ece0600007fe0fc4688290400016a0000www.toyota.com/1536324757811230245003340226955229884863* Array Cookie Simple Cookies

Where are cookies stored on client? Client-specific locations No standard IE stores in a folder called “Temporary Internet Files” Each cookie stored in a separate file. Firefox stores them in a user-specific profile file that ends in “.default” Safari stores them in ~/Library/Cookies/Cookies.plist

Cookie Types and Taxonomy By Origination First Party Cookies Third Party Cookies By Lifespan Session (temporary) Cookies (RAM) Persistent Cookies (Disk) By Read-Write Mechanism (see next slide) Server-Side Cookies (HTTP Header) Client-Side Cookies (JavaScript) By Structure Simple Cookies Array Cookies

Cookie Code Cookies may be read/written by server-side code, or client-side code. Server-side Cookies are executed by the web server and instructions included in HTTP header for the page. Server-side Cookie languages: Perl/CGI, ASP/VBScript. Client-side scripts: JavaScript embedded in page HTML. function setCookie (name, value, expires, path, domain, secure) function getCookie (name) function deleteCookie (name, path, domain)

First-Party vs. Third-Party cookies A first-party cookie either originates on or is sent to the Web site you are currently viewing. A third-party cookie either originates on or is sent to a Web site different from the one you are currently viewing. Third-party Web sites usually provide some content on the Web site you are viewing. For example, many sites use advertising from third-party Web sites and those third-party Web sites may use cookies. A common use for this type of cookie is to track your Web page use for advertising or other marketing purposes. Third-party cookies can either be persistent or temporary.

Cookie Code: JavaScript JavaScript code uses 3 standard functions that are defined in the HTML <head> tag: getCookie(cookieName) setCookie(cookieName, value, expDate) delCookie(cookieName) All Cookie manipulation is performed using these 3 functions and regular algorithmic constructs All functions are automatically performed on the cookie object of that domain/path

HTTP over TCP Routers Routers HTTP Files TCP IP packets packets

Chapter 2 Outline 2.3 FTP 2.1 Principles of app layer protocols clients and servers app requirements 2.2 Web and HTTP 2.3 FTP 2.4 Electronic Mail SMTP, POP3, IMAP 2.5 DNS 2.6 Socket programming with TCP 2.7 Socket programming with UDP 2.8 Building a Web server 2.9 Content distribution Network Web caching Content distribution networks P2P file sharing

FTP: the File Transfer Protocol user interface client file transfer FTP server user at host local file system remote file system transfer file to/from remote host client/server model client: side that initiates transfer (either to/from remote) server: remote host ftp: RFC 959 ftp server: port 21

FTP: Separate Control, Data Connections TCP control connection port 21 (persistent) TCP data connection port 20(nonpersistent) FTP client FTP server FTP client contacts FTP server at port 21, specifying TCP as transport protocol. Client obtains authorization over control connection. Client browses remote directory by sending commands over control connection. When server receives a command for a file transfer, server opens a TCP data conn. to client at port 20.

FTP MODEL - 1 USER PI: Protocol Interpreter DTP: Data Transfer Process INTERFACE USER PI FTP COMMANDS SERVER PI FTP REPLIES USER DTP SERVER DTP DATA CONNECTION FILE SYSTEM FILE SYSTEM

Control Connection Initiation In the model, the user-protocol interpreter (PI) initiates the control connection. The control connection follows the Telnet protocol. FTP commands are generated by the user-PI and transmitted to the server process via the control connection. Standard replies are sent from the server-PI to the user-PI over the control connection in response to the commands.

Data Connection Initiation The FTP commands specify the parameters for the data connection (data port, transfer mode, representation type, and structure) and the nature of file system operation (store, retrieve, append, delete, etc.). The user-DTP should "listen" on the specified data port, and the server initiate the data connection and data transfer in accordance with the specified parameters. Note that the data connection may be used for simultaneous sending and receiving.

FTP: Separate Control, Data Connections After transferring a file, server closes port 20. Server opens a second TCP data connection to transfer the next file. Control connection: “out of band” (http is also a file transfer protocol. It is stateless, in-band and a push/pull protocol. FTP is also a push/pull one) FTP server maintains “state”: current directory, earlier authentication

FTP Commands, Responses Sample commands: sent as ASCII text over control channel (4 uppercase ASCII characters) USER username PASS password LIST return list of file in current directory PORT ip and port addr. for datat connection RETR filename retrieves (gets) file STOR filename stores (puts) file onto remote host Sample return codes status code and phrase (as in HTTP) 331 Username OK, password required 125 data connection already open; transfer starting 425 Can’t open data connection 452 Error writing file ftp://ftpserver.iust.ac.ir/dir1/filename.extension

Example FTP client tells FTP server how to open the data connection: supplies IP address and port

FTP Reply Status Code - 1 First digit of status code indicates type of reply: ‘1’: Positive Preliminary Reply (got it, but wait). ‘2’: Positive Completion Reply (success). ‘3’: Positive Intermediate Reply (waiting for more information). ‘4’: Transient Negative Completion (error - try again). ‘5’: Permanent Negative Reply (error - can’t do).

FTP Reply Status Code - 2 2nd digit indicates function groupings. ‘0’: Syntax (problem with command syntax). ‘1’: Information (reply to help or status cmds). ‘2’: Connections (problem with a connection). ‘3’: Authentication (problem with login). ‘4’: Unspecified. ‘5’: File system (related to file system). 3rd digit indicates specific problem within function group.

Chapter 2 Outline 2.4 Electronic Mail 2.1 Principles of app layer protocols clients and servers app requirements 2.2 Web and HTTP 2.3 FTP 2.4 Electronic Mail SMTP, POP3, IMAP, HTTP 2.5 DNS 2.6 Socket programming with TCP 2.7 Socket programming with UDP 2.8 Building a Web server 2.9 Content distribution Network Web caching Content distribution networks P2P file sharing

Electronic Mail Three major components: user agents mail servers user mailbox outgoing message queue Three major components: user agents mail servers simple mail transfer protocol: SMTP User Agent Sometimes is called: “mail reader” composing, editing, reading mail messages e.g., Eudora, Outlook, elm, Netscape Messenger outgoing, incoming messages stored on server user agent mail server user agent SMTP mail server user agent SMTP mail server SMTP user agent user agent user agent

Mail Clients Microsoft Outlook Microsoft Live Mail Qualcomm Eroda Mozila Thunderbird Mailreader (pop3) Netscape Messenger Microsoft Outlook Microsoft Live Mail Qualcomm Eroda

Electronic Mail: Mail Servers mailbox contains incoming messages for user message queue of outgoing (to be sent) mail messages SMTP SMTP protocol between mail servers to send email messages “client”: sending mail server (sending agent) “server”: receiving mail server (receiving agent) user agent mail server user agent SMTP mail server user agent SMTP mail server SMTP user agent user agent user agent

SMTP Mail Flow Mail Queue Internet

Electronic Mail: SMTP [RFC 2821] uses TCP to reliably transfer email message from client to server, port 25 direct transfer: sending server to receiving server three phases of transfer handshaking (greeting) transfer of messages closure command/response interaction commands: ASCII text response: status code and phrase messages must be in 7-bit ASCII

Scenario: User1 sends message to User2 1) User1 (sender) uses UA to compose message to user2@yahoo.com . 2) User1’s UA sends message to his mail server; message placed in message queue. 3) Client side of SMTP opens TCP connection with User2’s mail server. 4) SMTP client sends User1’s message over the TCP connection. 5) User2’s mail server places the message in User2’s mailbox. 6) User2 invokes his/her user agent to read message. UA: User Agent webmail.iust.ac.ir 1 mail server mail server User2 user agent user agent User1 2 6 3 4 5 user2@yahoo.com yahoo.com user1@webmail.iust.ac.ir

SMTP: Final Words SMTP uses persistent connections. SMTP requires message (header & body) to be in 7-bit ASCII SMTP server uses CRLF.CRLF to determine end of message Comparison with HTTP: HTTP: pull SMTP: push both have ASCII command/response interaction, status codes. HTTP: each object encapsulated in its own response message. SMTP: multiple objects sent in multipart message.

Mail Message Format . header body SMTP: protocol for exchanging email msgs RFC 822: standard for text message format: header lines, e.g., To: From: Subject: different from SMTP commands! body the “message”, ASCII characters only header blank line body .

SMTP Message Transfer Between Mail Servers sms = sending mail server (client side) rms = receiving mail server (server side) sms rms: TCP connection establishment on port 25 sms rms: SMTP handshake (HELO, MAIL, FROM, RCPT TO, DATA, commands, and 220, 250, 354, 221 status codes) sms header <CRLF> body <CRLF>.<CRLF> sms QUIT rms 221 yahoo.com closing connection Mail Massage

Example webmail.iust.ac.ir User2 User1 user2@yahoo.com yahoo.com server User1 mail server user agent 1 6 3 4 user agent 2 5 user2@yahoo.com yahoo.com user1@webmail.iust.ac.ir

Example: SMTP Interaction sms: TCP connection request on port 25 rms: 220 yahoo.com accepts connection sms: HELO webmail.iust.ac.ir is ready rms: 250 Hello webmail.iust.ac.ir sms: MAIL FROM: <user1@webmail.iust.ac.ir> rms: 250 user1@webmail.iust.ac.ir... Sender ok sms: RCPT TO: <user2@yahoo.com> rms: 250 user2@yahoo.com ... Recipient ok sms: DATA rms: 354 Enter mail, end with "." on a line by itself sms: From: user1@webmail.iust.ac.ir sms: To: user2@yahoo.com . sms: . rms: 250 Message accepted for delivery sms: QUIT rms: 221 yahoo.com closing connection Mail Massage from user1 to user2

Message Format: Multimedia Extensions MIME: multimedia mail extension, RFC 2045, 2056 additional lines in message header declare MIME content type From: User1@webmail.iust.ac.ir To: User2@yahoo.com Subject: Picture of iust cc: User3@tu.ac.ir MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Type: image/jpeg base64 encoded data ..... ......................... ......base64 encoded data . MIME version method used to encode data multimedia data type, subtype, parameter declaration encoded data

MIME (RFCs2045, 2046, 2077) Types Content-Type: type/subtype; parameters Text example subtypes: plain, html Image example subtypes: jpeg, gif Audio example subtypes: basic (8-bit mu-law encoded), 32kadpcm (32 kbps coding) Video example subtypes: mpeg, quicktime Application other data that must be processed by reader before “viewable” example subtypes: msword, octet-stream

Multipart Type From: user1@webmail.iust.ac.ir To: user2@yahoo.com Subject: photo of iust. MIME-Version: 1.0 Content-Type: multipart/mixed; boundary=StartOfNextPart --StartOfNextPart Dear User1, Hear is a photo of iust in 1383 winter. Content-Transfer-Encoding: base64 Content-Type: image/jpeg base64 encoded data ..... ......................... ......base64 encoded data Do you like to receive more pictures? .

Example: SMTP Response Time 1RTT sms: TCP connection request on port 25 rms: 220 yahoo.com accepts connection sms: HELO webmail.iust.ac.ir is ready rms: 250 Hello webmail.iust.ac.ir sms: MAIL FROM: <user1@webmail.iust.ac.ir> rms: 250 user1@webmail.iust.ac.ir... Sender ok sms: RCPT TO: <user2@yahoo.com> rms: 250 user2@yahoo.com ... Recipient ok sms: DATA rms: 354 Enter mail, end with "." on a line by itself sms: From: user1@webmail.iust.ac.ir sms: To: user2@yahoo.com . sms: . rms: 250 Message accepted for delivery sms: QUIT rms: 221 yahoo.com closing connection 1RTT 1RTT 1RTT 1RTT 1RTT 4RTT Mail Sending Time (MST) 1RTT Next mail 1RTT

SMTP Response Time - 1 1RTT 4RTT 6RTT 7RTT+M/B 8RTT+M/B TCP 220 Helo 250 Mail From Rcp To Data 354 Quit 221 time 1RTT 6RTT 8RTT+M/B 7RTT+M/B 4RTT

SMTP Response Time - 2 1 Message from sms to rms SRT = 8RTT + M/B N messages from sms to rms N SRT = 4(N+1)RTT +∑ Mi /Bi B is the throughput of sms-rms connection (Byte/sec) M is the email message size (Byte)

Example: 1 mail for 2 RCPTs sms: TCP connection request on port 25 rms: 220 yahoo.com accepts connection sms: HELO webmail.iust.ac.ir is ready rms: 250 Hello webmail.iust.ac.ir sms: MAIL FROM: <user1@webmail.iust.ac.ir> rms: 250 user1@webmail.iust.ac.ir... Sender ok sms: RCPT TO: <user2@yahoo.com> rms: 250 user2@yahoo.com ... Recipient ok sms: RCPT TO: <user3@yahoo.com> rms: 250 user3@yahoo.com ... Recipient ok sms: DATA rms: 354 Enter mail, end with "." on a line by itself sms: From: user1@webmail.iust.ac.ir sms: To: user2@yahoo.com; user2@yahoo.com . sms: . rms: 250 Message accepted for delivery sms: QUIT rms: 221 yahoo.com closing connection

Mail Access Protocols SMTP SMTP access protocol user agent user agent Receiver Sender sender’s mail server receiver’s mail server SMTP: delivery/storage to receiver’s server Mail access protocol: retrieval from server POP: Post Office Protocol [RFC 1939] authorization (agent <-->server) and download IMAP: Internet Mail Access Protocol [RFC 1730] more features (more complex) manipulation of stored messages on server HTTP (webmail) : Hotmail , Yahoo Mail, Gmail, etc.

POP3-Post Office Protocol - Version 3 RFC 1939 / Std 53 To retrieve emails from server (POP3 server) Server port number: 110 Three states Authorization State Transaction State Update State

POP3 Commands Authorization State Update State Transaction State USER username PASS password APOP name digest Transaction State STAT LIST [msgNo] RETR msgNo DELE msgNo NOOP RSET TOP msgNo line UIDL [msgNo] Update State QUIT

POP3 Protocol authorization phase C: list transaction phase, client: S: +OK POP3 server ready C: user USER1 S: +OK C: pass zxcdvf S: +OK user successfully logged on authorization phase client commands: user: declare username pass: password server responses +OK -ERR transaction phase, client: list: list message numbers retr: retrieve message by number dele: delete quit C: list S: 1 498 S: 2 912 S: . C: retr 1 S: <message 1 contents> C: dele 1 C: retr 2 C: dele 2 C: quit S: +OK POP3 server signing off

POP3 (more) and IMAP IMAP Keep all messages in one place: the server More about POP3 Previous example uses “download and delete” mode. “Download” brings the mail into client computer. “Delete” removes the mail from the mail server. User1 cannot re-read e-mail if he/she changes client computer. “Download-and-keep”: leaves the message on the mail server. POP3 is stateless across sessions. IMAP Keep all messages in one place: the server Allows user to organize messages in folders IMAP keeps user state across sessions: names of folders and mappings between message IDs and folder name

Webmail Web-based e-mail programs called webmail. Webmail has several advantages: ability to send and receive e-mail from anywhere using a single application: a web browser. This eliminates the need to configure an e-mail client. Examples of e-mail services which also provide the user a webmail interface are Hotmail, Gmail, AOL and Yahoo.

Webmail (more) The main drawbacks of webmail are: User interactions are subject to network response and There is no offline capability. Webmail generally provides the best experience over broadband, A mail-reader can provide a satisfactory experience over dialup, and messages can be searched and viewed without an internet connection.

Chapter 2 Outline 2.5 DNS 2.1 Principles of app layer protocols clients and servers app requirements 2.2 Web and HTTP 2.3 FTP 2.4 Electronic Mail SMTP, POP3, IMAP, HTTP 2.5 DNS (Internet directory service) 2.6 Socket programming with TCP 2.7 Socket programming with UDP 2.8 Building a Web server 2.9 Content distribution Network Web caching Content distribution networks P2P file sharing

Domain Name System (DNS) Paul V. Mockapetris is the inventor of the Domain Name System. In 1983, he proposed a Domain Name System (DNS) architecture in RFCs 882 and 883. He had recognized the problem in the early Internet (then ARPAnet) of holding name to address translations in a single table on a single host, and instead proposed a distributed and dynamic DNS database: essentially DNS as we have it today. Together with Jon Postel, he is acknowledged as the inventor of DNS.

www.internic.net . www.whois.net Public Information Regarding Internet Domain Name Registration Services www.whois.net domain name lookup, available domain names, domain keyword search, deleted domains

Name, Address, Route The name of a resource indicates what we seek human recognizable Universal Resource Names (URNs) An address indicates where it is IP address, port A route tells how to get there Internet routing Names are for human use to identify objects; Addresses are for computer use to find route.

DNS: Domain Name SystemRFC1034 People: has name(s)/identifier(s): given and family names, national number, passport # address Internet hosts, routers: Host: may have “name(s)”, e.g., www.iust.ac.ir Router: has no name Hosts and Routers: have address (IP address) Domain Name System: map between name(s) and IP addresses: Name translation Address distributed database implemented in hierarchy of many name servers application-layer protocol host and name servers communicate to resolve names (address/name translation)

Why DNS Although DNS is most commonly associated with the Internet, private networks also use DNS because of the following benefits: Convenience: User-friendly names are easier for people to remember than numerical IP addresses. Consistency: IP addresses may change, but the server names can remain constant. Simplicity: Users need to learn only one naming convention to find resources on either the Internet or on an intranet.

DNS: Domain Name System The Domain Name System is a distributed database system that can serve as the foundation for name resolution in a TCP/IP network. DNS is used by most internetworking software, such as web browsers and electronic mail programs, to locate servers and to resolve, or map, a user-friendly name of a computer to its IP address.

DNS Components To understand how DNS maps names to IP addresses, you will need to understand the following five components: 1. Domain Name Space Name Structure, 2. Zone Zone data base file, 3. Name Server (DNS Server) DNS Root Servers, 4. DNS Resolver, 5. Protocol Forward Lookup Query Caching.

1. Domain Name Space The domain name space has a hierarchical structure. The root domain is at the top, and it is represented by a period. Below the root domain, the top-level, or 1st-level domains can be an organizational type, such as com or edu., or a geographic location, such as ir. for IRAN. Second-level domains are registered to individuals or organizations, such as microsoft.com, the Microsoft Corporation domain. Second-level domains can have many sub domains. …and any domain can have hosts.

1. Domain Name Space “ .” Root Domain org. ir. edu. com. Top Level or First Level Domain host subdomain 2nd-level domain top-level webmail . iust . ac . ir . root fully qualified domain name, or FQDN org. edu. com. ir. Second Level Domain (Registered to Individuals or Organizations) ac.ir. iust.ac.ir. ce.iust.ac.ir. Any domain can have hosts. A host is a specific computer within a domain. Iran University of Science & Technology ‘s Domain: iust.ac.ir.

1. Domain Name Space: Name Structure Max: 63 Characters Root Host name . … . Subdomain-name . 2nd-level –name. 1st-level-name . Max: 255 Characters ICANN: Internet Corporation For Assigned Names and Numbers) http://icann.org Levelm -name authority appoints levelm-1 -names. Example: ir is controlled by iranic.com iust.ir is controlled by Iran University of science and Technology.

Names are Valuable new

There are more than 200 top-level domains. Three types of top-level domains: Organizational: 3-character code indicates the function of the organization Used primarily within the US Examples: gov, mil, edu, org, com, net Geographical: 2-character country or region code Examples: ir, uk, jp, de Reverse domains: A special domain (in-addr.arpa) used for IP address-to-name mapping and gateway locating. Infrastructure Domain (Address and Routing Parameter Area domain) .arpa There are more than 200 top-level domains. .arpa is an Internet top-level domain (TLD) used exclusively for Internet-infrastructure purposes.

Examples: Top Level Domains com: commercial edu: educational institute ac: Academic, research institute int: international organization gov: US federal goverment org: US armed forces net: network providers biz: business name: people’s name pro: professions tv: TV stations and networks … ir: Iran nl: Netherlands jp: Japan uk: England … For the most up-to-date information about new top level domains, consult: http://www.icann.org/tlds

Registrar A few words about a Registrar: Organization where you register a domain name Verifies uniqueness of name Enters domain name into various databases: whois & DNS

List of registrars from internic.net:

Whois databases Input: domain name or company name Output: registrar, whois server, dns server Some useful whois sites: www.internic.net For com, net and org top-level domains www.allwhois.com For country-code top-level domains, e.g., jp, fr Two steps First find target’s registrar Then whois target at registrar

Internic Whois: Target “kazaa”

Whois: next step Do whois at registrar, eg, register.com Input: domain name, IP address, net administrator name Output: Names of people (administrator, billing contact) Telephone numbers E-mail addresses Name servers and IP addresses

Whois at kazaa’s registrar

Reconnaissance: IP Ranges ARIN: American Registry for Internet Numbers Maintains whois database that includes IP address ranges in US RIPE: Europe, Middle East, Central Asia APNIC: Asia, Pasific

RIPE NCC

Local Internet Registrar for Iran

Reverse Domains A special domain to support gateway location and Internet address to host mapping. The domain begins at IN-ADDR.ARPA and has a substructure which follows the Internet addressing structure. Domain names in the IN-ADDR.ARPA domain are defined to have up to four labels in addition to the IN-ADDR.ARPA suffix. Example: Data for Internet address 10.2.0.52 is located at domain name 52.0.2.10.IN-ADDR.ARPA. Example: 10.IN-ADDR.ARPA can be a zone containing data for the ARPANET.

Gateway Locating A program (name server) which wanted to locate gateways on net 10 would originate a query of the form QTYPE=PTR, QCLASS=IN, QNAME=10.IN-ADDR.ARPA. It would receive two RRs in response: 10.IN-ADDR.ARPA. PTR MILNET-GW.ISI.EDU. 10.IN-ADDR.ARPA. PTR GW.LCS.MIT.EDU. The program could then originate QTYPE=A, QCLASS=IN queries for MILNET- GW.ISI.EDU. and GW.LCS.MIT.EDU. to discover the Internet addresses of these gateways.

6.0.0.10.IN-ADDR.ARPA. PTR MULTICS.MIT.EDU. Address to Host Name A resolver which wanted to find the host name corresponding to Internet host address 10.0.0.6 would pursue a query of the form QTYPE=PTR, QCLASS=IN, QNAME=6.0.0.10.IN-ADDR.ARPA, and would receive: 6.0.0.10.IN-ADDR.ARPA. PTR MULTICS.MIT.EDU.

2. Zones & Authoritative DNS Server For administrative purposes, domains can be organized into zones. As you can see in this example, a zone is a discrete and contiguous area of the domain name space. One reason to divide a name space into zones is to delegate authority for different portions of it. One very large domain could be difficult to administer. The name-to-IP address data for computers located in a zone, is stored in a zone database file on a DNS name server. The zone DNS server is an authoritative server. ir. Name Server Zone Database File Name Server ac.ir. Zone Database File Zone Database File Name Server

2. Zone Owner’s Responsibilities Authoritatively maintain the zone’s data. Arrange for replicated name servers for the zone Typically, zone data is maintained in a primary file and loaded into a primary server Replicated servers use TCP-based zone transfers specified in DNS protocol to refresh their data. Therefore, either servers can answer name resolution queries for that zone. A name server authoritative for a zone does not have to be in that zone. A name server can handle any number of zones, which don’t have to be contiguous.

RR format: (Name, TTL, Class, Type, Value) 2. Zone Database File A zone database storing resource records (RR) of the zone for which a server is authoritative. RR format: (Name, TTL, Class, Type, Value) Type=A name is hostname value is IP address Type=CNAME name is alias name for some “canonical” (the real) name www.ibm.com is really servereast.backup2.ibm.com value is canonical name Type=NS name is domain (e.g. iust.ac.ir) value is IP address of authoritative name server for this domain Type=MX value is name of mail-server associated with name

2. Zone Database File: Type Description Example Resource Record Format Description Type Number www 1D IN A 10.1.1.1 Network address A 1 @ 1D IN NS na.av.com Authoritative name server NS 2 Mail destination; now replaced by MX MD 3 Mail forwarder; now replaced by MX MF 4 Canonical alias name CNAME 5 Start of zone authority SOA 6 Mailbox domain name MB 7 Mailbox member MG 8 Mail rename domain MR 9 Null resource record NULL 10 Well-Known service WKS 11 RR format: (Name, TTL, Class, Type, Value)

2. Zone Database File: Type Description Example Record Format Description Type Number Pointer to a domain name PTR 12 www 1D IN HINFO Solaris Host information HINFO 13 Mailbox information MINFO 14 Mail exchange MX 15 System 1 IN TXT “This is a new system” Text strings TXT 16 Responsible person RP 17 AFS-type services AFSDB 18 X.25 address X.25 19 ISDN address ISDN 20 Route through RT 21 RR format: (Name, TTL, Class, Type, Value)

2. Zone Database File: Example Host Information Name TTL Class Type Value mail.iust.ac.ir 1 D IN HINFO Sun Unix mail.iust.ac.ir 1 D IN A 130.37.16.112 mail.iust.ac.ir 1 D IN A 192.31.231.165 mail.iust.ac.ir 1 D IN MX 1 mail.iust.ac.ir mail.iust.ac.ir 1 D IN MX 2 ms.iust.ac.ir post.iust.ac.ir. 1 D IN CNAME mail.iust.ac.ir ftp.iust.ac.ir. 1 D IN CNAME file.iust.ac.ir Primary mail server Secondary Canonical Names Real names

An Example Max. age of cached data in seconds * Start of authority (SOA) record. Means: “This name server is authoritative for the zone Mylab.com” * PC4.mylab.com is the name server * hostmaster@mylab.com is the email address of the person in charge Name server (NS) record. One entry for each authoritative name server Address (A) records. One entry for each hostaddress

3. DNS Name Servers Why not centralize DNS? single point of failure traffic volume distant centralized database maintenance doesn’t scale! no server has all name-to-IP address mappings. does scale! A DNS server is a computer that runs a DNS server program, such as Berkeley Internet NAME Domain (BIND). DNS Servers contain DNS database files. They resolve name resolution queries issued by DNS clients.

BIND Software – Last Version About the BIND 10 The architecture of BIND 10 concentrates on these technical aspects: modularity, customizability, clusterization, integration, resilience, and runtime control. Source: Internet System Consortium, http://www.isc.org

3. DNS Name Servers ir. One name server holds the primary zone database file for a zone. That name server is said to "have authority" for making changes to the zone file. A copy of the zone file may be stored on other servers to balance network load. These copies are referred to as secondary zone database files. The secondary zone database files are updated regularly from the primary zone database file by a process called zone transfer. Primary Zone Database File Name Server Secondary Zone Database File ac.ir.

13 root name servers worldwide 3. DNS Root Name Servers (a) NSI Herndon, VA (c) PSInet Herndon, VA (d) U Maryland College Park, MD (g) DISA Vienna, VA (h) ARL Aberdeen, MD (j) NSI (TBD) Herndon, VA (i) NORDUnet Stockholm, Sweden (k) RIPE London, UK (m) WIDE Tokyo, Japan (e) NASA Mt View, CA (f) Internet Software C. Palo Alto, CA (b) USC-ISI Marina del Rey, CA (l) ICANN Marina del Rey, CA 13 root name servers worldwide

3. Root Name Server in Action Contacted by local name server that can not resolve name Root name server: Either, contacts authoritative name server gets mapping returns mapping to local name server Or, introduces authoritative/top level name server to local name server.

13 Root Name Servers F gets 270,000,000+ hits per day. Other root servers are comparable. Top Level Domain servers answer 5,000,000,000 per day.

3. DNS Root Name Servers Root Servers A B C D E F G Server Operator Locations IP Addr Home ASN A VeriSign Global Registry Services Dulles VA 198.41.0.4 19836 B Information Sciences Institute Marina Del Rey CA IPv4: 192.228.79.201 IPv6: 2001:478:65::53 tba C Cogent Communications Herndon VA; Los Angeles; New York City; Chicago 192.33.4.12 2149 D University of Maryland College Park MD 128.8.10.90 27 E NASA Ames Research Center Mountain View CA 192.203.230.10 297 F Internet Systems Consortium, Inc. Ottawa; Palo Alto; San Jose CA; New York City; San Francisco; Madrid; Hong Kong; Los Angeles; Rome; Auckland; Sao Paulo; Beijing; Seoul; Moscow; Taipei; Dubai; Paris; Singapore; Brisbane; Toronto; Monterrey; Lisbon; Johannesburg;Tel Aviv;Jakarta; Munich; IPv4: 192.5.5.241 IPv6: 2001:500::1035 3557 G U.S. DOD Network Information Center Vienna VA 192.112.36.4 568

3. DNS Root Name Servers Root Servers H I J K L M Server Operator Locations IP Addr Home ASN H U.S. Army Research Lab Aberdeen MD IPv4: 128.63.2.53 IPv6:2001:500:1::803f:235 13 I Autonomica/NORDUnet Stockholm; Helsinki; Milan; London; Geneva; Amsterdam; Oslo; Bangkok; Hong Kong; Brussels; Frankfurt; Bucharest; Ankara; Chicago; Washington DC; Tokyo; Kuala Lumpur 192.36.148.17 29216 J VeriSign Global Registry Services Dulles VA (2 locations); Mountain View CA; Seattle WA; Amsterdam; Atlanta GA; Los Angeles CA; Miami; Stockholm; London; Tokyo; Seoul; Singapore; Sterling VA (2 locations, standby) 192.58.128.30 26415 K Reseaux IP Europeens - Network Coordination Centre London (UK); Amsterdam (NL); Frankfurt (DE); Athens (GR); Doha (QA); Milan (IT) IPv4: 193.0.14.129 IPv6: 2001:7fd::1 25152 L Internet Corporation for Assigned Names and Numbers Los Angeles 198.32.64.12 20144 M WIDE Project Tokyo; Seoul (KR); Paris (FR) 202.12.27.33 IPv6: 2001:dc3::35 7500

Autonomous System On the Internet, an Autonomous System (AS) is the unit of router policy, Either a single network Or a group of network that is controlled by a common network administrator (or group of administrators) on behalf of a single administrative entity (such as a university, a business enterprise, or a business division). An autonomous system is also sometimes referred to as a routing domain. An autonomous system is assigned a globally unique number, sometimes called an Autonomous System Number (ASN).

TLD and Authoritative Servers Top-level domain (TLD) servers: responsible for com, org, net, edu, etc, and all top-level country domains ir, uk, fr, ca. Network Solutions maintains servers for com TLD Educause for edu TLD Authoritative DNS servers: organization’s DNS servers, providing authoritative hostname to IP mappings for organization’s servers (e.g., Web, mail). can be maintained by organization or service provider

Distributed Hierarchical DNS Servers Portion of Hierarchy of DNS Servers Root DNS Servers com DNS Servers (Top-Level-Domain) org DNS Servers (Top-Level-Domain) ir DNS Servers (Top-Level Domain) Yahoo.com DNS Servers amazon.com DNS Servers dns.org DNS Servers ac.ir DNS Servers iust.ac.ir DNS Server(s)

Local Name Server Does not strictly belong to hierarchy Each ISP (residential ISP, company, university) has one. also called “default name server” When host makes DNS query, query is sent to its local DNS server acts as proxy, forwards query into hierarchy

4. Resolver A DNS resolver is a service that uses the DNS protocol to query for information from DNS servers. Function of DNS resolver is performed by the DNS client service. DNS service uses UDP in transport layer on port 53.

5. DNS Protocol: Forward Lookup Query (recursive) a:answer r:referral q:query root name server q Root name server: may not know authoritative name server may know intermediate name server: who to contact to find authoritative name server 2 6 a 7 3 q a q a q a local name server dns.iust.ac.ir intermediate name server (TDL) dns.org 4 5 1 8 authoritative name server dns.icann.org requesting host www.icann.org

5. DNS Protocol: Forward Lookup Query (iterative) a:answer r:referral q:query root name server recursive query: puts burden of name resolution on contacted name server heavy load? iterated query: contacted server replies with name of server to contact “I don’t know this name, but ask this server” q iterated query 2 3 r q 4 a 7 q a q a intermediate name server (TDL) dns.org local name server dns.iust.ac.ir 5 6 1 8 authoritative name server dns.icann.org requesting host www.icann.org

5. DNS Protocol: Forward Lookup Query a:answer r:referral q:query root name server q 2 iterated query r 3 q 4 r 5 q a 7 q a intermediate name server (TDL) dns.org local name server dns.iust.ac.ir 6 1 8 authoritative name server dns.icann.org requesting host www.icann.org

5. DNS Protocol: Forward Lookup Query1 IP Address of www.icann.org is 142.12.01.23 Root Name Server org top level DNS server “ .” org. ir. edu. com. ac.ir. iust.ac.ir. Local Name Server www.icann.org. Client The query contains the FQDN of the requested computer. A forward lookup query is a request to map a name to an IP address. Forward Lookup Query: What is IP Address of www.icann.org

5. DNS Protocol: Forward Lookup Query2 Root server 3 4 Application DNS cache Top-level domain server 5 1 10 DNS query 2 Local DNS server 6 Second-level domain server (authoritative) 7 DNS resolver DNS response 9 8 Caching based on a time-to-live (TTL) assigned by the DNS server responsible for the host name to reduce latency in DNS translation.

5. DNS Protocol: Caching and Updating Records once (any) name server learns mapping, it caches mapping cache entries timeout (disappear) after some time Top Level Domain servers typically cached in local name servers. Thus root name servers not often visited. update/notify mechanisms under design by IETF RFC 2136 http://www.ietf.org/html.charters/dnsind-charter.html

5. DNS Protocol Messages msg header DNS protocol : query and reply messages, both with same message format identification flags number of questions number of answer RRs number of authority RRs number of additional RRs questions (variable number of questions) answers (variable number of resource records) authority additional information msg header identification: 16 bit # for query, reply to query uses same # flags: query or reply recursion desired recursion available reply is authoritative 12bytes

5. DNS Protocol Messages Name, Type fields for a query RRs in response identification flags number of questions number of answer RRs number of authority RRs number of additional RRs questions (variable number of questions) answers (variable number of resource records) authority additional information Name, Type fields for a query 12bytes RRs in response to query records for authoritative servers additional “helpful” info that may be used Query and reply messages sent over UDP on port 53.

Inserting records into DNS Example: new startup “iust company” Register name iustcompany.com at DNS registrar (e.g., Network Solutions) Provides names, IP addresses of authoritative name server (primary and secondary) Registrar inserts two RRs into com TLD server: (iustcompany.com, dns1.iustcompany.com, NS) (dns1.iustcompany.com, 212.212.212.1, A) Create authoritative server Type A record for www.iustcompany.com; Type MX record for networkutopia.com

DNS Other Important Services Host Aliasing Complicated hostname (computer.enginerring-student.1990.iust.ac.ir) can have one or more nickname (alias names) such as student90.iust.ir and computer90.ir computer.enginerring-student.1990.iust.ac.ir is said to be canonical hostname Mail Server Aliasing Mail server can also have aliased hostnames. MX records permits the mail server and web server of a company have identical aliased hostname. Load Distribution DNS is used to perform load distribution among replicated servers. For replicated web servers, a set of IP addresses is associated with one canonical hostname. Name server rotates IP add. within each reply to dns query.

Problems of DNS Domain names may not be the best way to name other resources, e.g. files Relatively static resource types make it hard to introduce new services or handle mobility Although theoretically you can update the values of the records, it is rarely enabled Simple query model make it hard to implement advanced query Early binding (separation of DNS query from application query) does not work well in mobile, dynamic environments e.g., load balancing, locate the nearest printer

Chapter 2 Outline 2.6 Socket programming with TCP 2.1 Principles of app layer protocols clients and servers app requirements 2.2 Web and HTTP 2.3 FTP 2.4 Electronic Mail SMTP, POP3, IMAP 2.5 DNS 2.6 Socket programming with TCP 2.7 Socket programming with UDP 2.8 Building a Web server 2.9 Content distribution Network Web caching Content distribution networks P2P file sharing

Server Types Servers Iterative server Concurrent server Connectionless Iterative Concurrent Connection-Oriented Iterative server it can handle only one request at a time other requests have to wait Concurrent server handles several requests concurrently

Iterative Server

Concurrent Server

Using Ports to Identify Services Server host 128.2.194.242 Service request for 128.2.194.242:80 (i.e., the Web server) Client host Web server (port 80) Client OS Echo server (port 7) Client host Service request for 128.2.194.242:7 (i.e., the echo server) Web server (port 80) Client OS Echo server (port 7)

Network Application-Communication Network Applications (process) communicates through a network. API is a programming means, either a library or part of OS. API provides the communication functionality for the Network Applications (Processes). Application Program Interface (API) Communication Software & Hardware Platform (OS + Hardware) Application Software (Network Application) OS API comes as a library

API and Networking Layers Application-layer protocols: One “piece” of the application. Define messages exchanged by apps and actions taken. the application App. Software Application Program Interface (API) Communication Software & Hardware Platform (OS + Hardware) Application Software Controlled by App. Soft. application (API) transport network link physical Controlled by OS

Socket API Sockets: doors between process and host process sends/receives messages to/from its socket socket analogous to door sending process pushes message out door sending process assumes transport infrastructure on other side of door which brings message to socket at receiving process host or server host or server process process socket socket transport network link physical transport network link physical Internet

API Technologies-Sockets Sockets (earliest technology) Low level functions for the communication Socket, ServerSocket, ConnectionSocket, ClientSocket, DatagramSocket, Send, Read, Write, Close, Accept

API Technologies-RPC Remote Procedure Call (RPC) Client code invokes a procedure on a remote server. RPCs support a wire format common to all platforms. Client and server translate from their binary format into the wire format. Each RPC involves four translation: Client to wire format, Wire to server format, Server back to wire (for the return value), Wire format back to client format.

API Technologies-Components Components: Components Look and act like objects, CORBA (common object request Broker Architecture) COM, COM+ , DCOM (Distributed Component Object Model)

Socket Programming Goal: learn how to build client/server application that communicate using sockets Socket API introduced in BSD4.1 UNIX, 1981 explicitly created, used, released by applications client/server paradigm two types of transport service via socket API: unreliable datagram reliable, byte stream-oriented Socket a host-local, application-created, OS-controlled interface a “door” into which application process can both send and receive messages to/from another application process

Socket Programming Using TCP Socket: a door between application process and end-end-transport protocol (UCP or TCP) TCP service: reliable transfer of bytes from one process to another controlled by application developer controlled by application developer process process socket socket TCP with buffers, variables controlled by operating system controlled by operating system TCP with buffers, variables Network client or server client or server

Socket Programming with TCP Client must contact server server process must first be running server must have created socket (door) that welcomes client’s contact Client contacts server by: creating client-local TCP socket specifying IP address, port number of server process When client creates socket: client TCP establishes connection to server TCP When contacted by client, server TCP creates new socket for server process to communicate with client allows server to talk with multiple clients source port numbers used to distinguish clients (more in Chap 3) TCP provides reliable, in-order transfer of bytes (“pipe”) between client and server application viewpoint

Sockets bytes Three-way handshake send read write read Client process Server process Server IP Address & Port Number1 Welcoming socket Three-way handshake send read write read bytes Client socket Connection socket Server IP Address & Port Number2 Client IP Address & Port Number

Example: Socket Programming with TCP Example client-server app. in Java: Client reads line from standard input (keyboard) (inFromUser stream) sends to server via socket (outToServer stream) Server reads line from socket converts line to uppercase, sends back to client reads (inFromServer stream), prints modified line from socket on its standard output (monitor)

Example: 3 streams and 1 socket inFormUser Input Stream Client Process inFormServer outTOServer Input Stream Output Stream ClientSocket From transport layer To transport layer TCP socket

Streams A stream is a sequence of characters that flow into or out of a process. An input stream is attached to some input source for the process, eg, keyboard or socket. An output stream is attached to an output source, eg, monitor or socket.

Client/Server Socket Interaction: TCP wait for incoming connection request connectionSocket = welcomeSocket.accept() create socket, port=x, for incoming request: welcomeSocket = ServerSocket() connect to hostid, port=x clientSocket = Socket() close connectionSocket read reply from clientSocket Server (running on hostid) Client send request using read request from write reply to TCP connection setup

Example: TCPClient.java import java.io.*; import java.net.*; class TCPClient { public static void main(String argv[ ]) throws Exception { String sentence; String modifiedSentence; BufferedReader inFromUser = new BufferedReader(new InputStreamReader(System.in)); Socket clientSocket = new Socket("hostname", 6789); import Statements make Java classes available networking class Creates input stream Creates client socket, TCP connection to server server port no. server name class: is a collection of data and methods that operate on that data

Example: TCPClient.java (cont.) Creates output stream attached to socket DataOutputStream outToServer = new DataOutputStream(clientSocket.getOutputStream()); BufferedReader inFromServer = new BufferedReader(new InputStreamReader(clientSocket.getInputStream())); sentence = inFromUser.readLine(); outToServer.writeBytes(sentence + '\n'); modifiedSentence = inFromServer.readLine(); System.out.println("FROM SERVER: " + modifiedSentence); clientSocket.close(); } Creates input stream attached to socket Sends line to server Reads line from server

Code Descriptions-TCPClient.java import java.io.*; java.io and java.net are java packages. The java.io package contains classes for input (BufferedReader) output (DataOutputStream) streams. import java.net.*; The java.net package provides classes for network support. It contains the Socket (clientSocket object is derived from the Socket class) and ServerSocket classes.

Code Descriptions-TCPClient.java class TCPClient { public static void main(String argv[ ]) throws Exception {......} } First is beginning of a class definition block. Class definition for the class named TCPClient. A class contains variables and methods. The class TCPClient has no class variables and exactly one method, the main() method. Java interpreter executes an application, starts by calling the class's main method. The main method then calls all the other methods required to run the application.

Code Descriptions-TCPClient.java String sentence; String modifiedSentence; These lines declare objects of type String. Object sentence is string typed by user and sent to server. Object modifiedSentence is the string obtained from the server and sent to the user's standard output. BufferedReader inFromUser = new BufferedReader (new InputStreamReader(System.in)); The line creates the stream object inFromUser of type BufferedReader. Input stream is initialized with System.in, which attaches the stream to standard input. The command allows the client to read text from its keyboard.

Code Descriptions-TCPClient.java Socket clientSocket = new Socket("hostname", 6789); The line creates object clientSocket of type Socket. performs a DNS look-up on the hostname to obtain the host's IP address. initiates the TCP connection between client and server. String "hostname" replaced with host name of server (for example, "fling.seas.upenn.edu"). The number 6789 is the port number. You can use a different port number; but you must make sure that you use the same port number at the server side of the application.

Code Descriptions-TCPClient.java DataOutputStream outToServer =new DataOutputStream(clientSocket.getOutputStream()); BufferedReader inFromServer =new BufferedReader(new inputStreamReader(clientSocket.getInputStream())); The above two lines create stream objects that are attached to the socket. outToServer stream provides the process output to the socket. inFromServer stream provides the process input from the socket.

Code Descriptions-TCPClient.java sentence = inFromUser.readLine(); The above line places a line typed by the user into the string sentence. The string sentence continues to gather characters until the user ends the line by typing a carriage return. The line passes from standard input through the stream inFromUser into the string sentence.

Code Descriptions-TCPClient.java outToServer.writeBytes(sentence + '\n'); Line sends the string sentence augmented with a carriage return into the outToServer stream. The augmented sentence flows through the client's socket and into the TCP pipe. The client then waits to receive characters from the server. modifiedSentence = inFromServer.readLine(); When characters arrive from the server, they flow through the stream inFromServer and get placed into the string modifiedSentence. Characters continue to accumulate in modifiedSentence until the line ends with a carriage return character.

Code Descriptions-TCPClient.java System.out.println("FROM SERVER " + modifiedSentence); The above line prints to the monitor the string modifiedSentence returned by the server. clientSocket.close(); This last line closes the socket and, hence, closes the TCP connection between the client and the server. It causes TCP in the client to send a TCP message to TCP in the server.

Example: TCPServer.java import java.io.*; import java.net.*; class TCPServer { public static void main(String argv[]) throws Exception { String clientSentence; String capitalizedSentence; ServerSocket welcomeSocket = new ServerSocket(6789); while(true) { Socket connectionSocket = welcomeSocket.accept(); BufferedReader inFromClient = new BufferedReader(new InputStreamReader(connectionSocket.getInputStream())); Create welcoming socket at port 6789 Wait, on welcoming socket for contact by client Create input stream, attached to socket

Example: TCPServer.java (cont.) DataOutputStream outToClient = new DataOutputStream(connectionSocket.getOutputStream()); clientSentence = inFromClient.readLine(); capitalizedSentence = clientSentence.toUpperCase() + '\n'; outToClient.writeBytes(capitalizedSentence); } Create output stream, attached to socket Read in line from socket Write out line to socket End of while loop, loop back and wait for another client connection

Code Descriptions-TCPServer.java TCPServer has many similarities with TCPClient. The first line in TCPServer that is substantially different from what we saw in TCPClient is: ServerSocket welcomeSocket = new ServerSocket(6789); Line creates the object welcomeSocket, which is of type ServerSocket. The WelcomeSocket is a sort of door that waits for a knock from some client. The port number 6789 identifies the process at the server.

Code Descriptions-TCPServer.java Socket connectionSocket = welcomeSocket.accept(); It creates a new socket, called connectionSocket, when some client knocks on welcomeSocket. TCP then establishes a direct virtual pipe between clientSocket at the client and connectionSocket at the server. The client and server can then send bytes to each other over the pipe, and all bytes sent arrive at the other side in order. With connectionSocket established, the server can continue to listen for other requests from other clients for the application using welcomeSocket. This version of the program doesn't actually listen for more connection requests, but it can be modified with threads to do so.

Code Descriptions-TCPServer.java The program then creates several stream objects, analogous to the stream objects created in clientSocket. Now consider: capitalizedSentence = clientSentence.toUpperCase() + '\n'; This command is the heart of the application. It takes the line sent by the client, capitalizes it, and adds a carriage return. It uses the method toUpperCase(). All the other commands in the program are peripheral; they are used for communication with the client.

Code Descriptions-TCPServer.java To test the program pair, you install and compile TCPClient.java in one host and TCPServer.java in another host. Be sure to include the proper host name of the server in TCPClient.java. You then execute TCPServer.class, the compiled server program, in the server. This creates a process in the server that idles until it is contacted by some client. Then you execute TCPClient.class, the compiled client program, in the client. This creates a process in the client and establishes a TCP connection between the client and server processes. Finally, to use the application, you type a sentence followed by a carriage return.

Chapter 2 Outline 2.7 Socket programming with UDP 2.1 Principles of app layer protocols clients and servers app requirements 2.2 Web and HTTP 2.3 FTP 2.4 Electronic Mail SMTP, POP3, IMAP 2.5 DNS 2.6 Socket programming with TCP 2.7 Socket programming with UDP 2.8 Building a Web server 2.9 Content distribution Network Web caching Content distribution networks P2P file sharing

Socket Programming with UDP UDP: no “connection” between client and server, no handshaking, sender explicitly attaches IP address and port of destination to each packet, server must extract IP address, port of sender from received packet. UDP: transmitted data may be received out of order, or lost. application viewpoint UDP provides unreliable transfer of groups of bytes (“datagrams”) between client and server

Client/Server Socket Interaction: UDP close clientSocket Server (running on hostid) read reply from create socket, clientSocket = DatagramSocket() Client Create, address (hostid, port=x, send datagram request using clientSocket port=x, for incoming request: serverSocket = read request from serverSocket write reply to specifying client host address, port number

Example: Java Client (UDP) Output: sends packet (TCP sent “byte stream”) Input: receives packet (TCP received “byte stream”) Process ClientSocket recievepacket sendpacket inFromUser Input Stream UDP datagram packet UDP socket To transport layer From transport layer

Example: UDPClient.java import java.io.*; import java.net.*; class UDPClient { public static void main(String args[]) throws Exception { BufferedReader inFromUser = new BufferedReader(new InputStreamReader(System.in)); DatagramSocket clientSocket = new DatagramSocket(); InetAddress IPAddress = InetAddress.getByName("hostname"); byte[ ] sendData = new byte[1024]; byte[ ] receiveData = new byte[1024]; String sentence = inFromUser.readLine(); sendData = sentence.getBytes(); Create input stream Create client socket Translate hostname to IP address using DNS

Example: Example: UDPClient.java (cont.) Create datagram with data-to-send, length, IP addr, port DatagramPacket sendPacket = new DatagramPacket(sendData, sendData.length, IPAddress, 9876); clientSocket.send(sendPacket); DatagramPacket receivePacket = new DatagramPacket(receiveData, receiveData.length); clientSocket.receive(receivePacket); String modifiedSentence = new String(receivePacket.getData()); System.out.println("FROM SERVER:" + modifiedSentence); clientSocket.close(); } Send datagram to server Read datagram from server

Code Descriptions-UDPClient.java The program UDPClient.java constructs one stream and one socket. The socket is called clientSocket, and it is of type DatagramSocket. UDP uses a different kind of socket than TCP at the client. Here the client uses a DatagramSocket whereas with TCP our client used a Socket. The stream inFromUser is an input stream to the program; it is attached to the standard input, that is, to the keyboard. We had an equivalent stream in our TCP version of the program. When the user types characters on the keyboard, the characters flow into the stream inFromUser. In contrast with TCP, there are no streams (input or output) attached to the socket. Instead of feeding bytes to the stream attached to a Socket object, UDP will push individual packets through the DatagramSocket object.

Code Descriptions-UDPClient.java DatagramSocket clientSocket = new DatagramSocket(); It creates the object clientSocket of type DatagramSocket. In contrast with TCPClient.java, this line does not initiate a TCP connection. In particular, the client host does not contact the server host upon execution of this line. For this reason, the constructor DatagramSocket() does not take the server hostname or port number as arguments. The execution of the above line creates a port for the client process but does not create a connection between the two processes.

Code Descriptions-UDPClient.java InetAddress IPAddress = InetAddress.getByName("hostname"); In order to send bytes to a destination process, we shall need to obtain the address of the process. Part of this address is the IP address of the destination host. The above line invokes a DNS look-up that translates the hostname to an IP address. DNS was also invoked by the TCP version of the client, although it was done there implicitly rather than explicitly. The method getByName() takes as an argument the hostname of the server and returns the IP address of this same server. It places this address in the object IPAddress of type InetAddress.

Code Descriptions-UDPClient.java byte[ ] sendData = new byte[ ]; byte[ ] receiveData = new byte[ ]; The byte arrays sendData and receiveData will hold the data the client sends and receives, respectively. sendData = sentence.getBytes(); The above line essentially performs a type conversion. It takes the string sentence and renames it as sendData, which is an array of bytes.

Code Descriptions-UDPClient.java DatagramPacket sendPacket = new DatagramPacket(sendData, sendData.length, IPAddress, 9876); This line constructs the packet, sendPacket, that the client will pop into the network through its socket. Note that sendPacket is of type DatagramPacket. This packet includes that data that is contained in the packet, sendData, the length of this data, the IP address of the server, and the port number of the application (which we have set to 9876).

Code Descriptions-UDPClient.java clientSocket.send(sendPacket); In the above line, the method send() of the object clientSocket takes the packet just constructed and pops it into the network through clientSocket. Once again, note that UDP sends the line of characters in a manner very different from TCP. TCP simply inserted the line into a stream, which had a logical direct connection to the server; UDP creates a packet that includes the address of the server. After sending the packet, the client then waits to receive a packet from the server.

Code Descriptions-UDPClient.java DatagramPacket receivePacket = new DatagramPacket(receiveData, receiveData.length); In the above line, while waiting for the packet from the server, the client creates a place holder for the packet, receivePacket, an object of type DatagramPacket. clientSocket.receive(receivePacket); The client idles until it receives a packet; when it does receive a packet, it puts the packet in receivePacket.

Code Descriptions-UDPClient.java String modifiedSentence = new String(receivePacket.getData()); It extracts the data from receivePacket and performs a type conversion, converting an array of bytes into the string modifiedSentence. System.out.println("FROM SERVER:" + modifiedSentence); This line, which is also present in TCPClient, prints out the string modifiedSentence at the client's monitor. clientSocket.close(); This last line closes the socket. Because UDP is connectionless, this line does not cause the client to send a transport-layer message to the server (in contrast with TCPClient).

Example: UDPServer.java import java.io.*; import java.net.*; class UDPServer { public static void main(String args[ ]) throws Exception { DatagramSocket serverSocket = new DatagramSocket(9876); byte[ ] receiveData = new byte[1024]; byte[ ] sendData = new byte[1024]; while(true) DatagramPacket receivePacket = new DatagramPacket(receiveData, receiveData.length); serverSocket.receive(receivePacket); Create datagram socket at port 9876 Create space for received datagram Receive datagram

Example: Example: UDPServer.java (cont.) String sentence = new String(receivePacket.getData()); InetAddress IPAddress = receivePacket.getAddress(); int port = receivePacket.getPort(); String capitalizedSentence = sentence.toUpperCase(); sendData = capitalizedSentence.getBytes(); DatagramPacket sendPacket = new DatagramPacket(sendData, sendData.length, IPAddress, port); serverSocket.send(sendPacket); } Get IP addr port #, of sender Create datagram to send to client Write out datagram to socket End of while loop, loop back and wait for another datagram

Building a Simple Web Server handles one HTTP request accepts the request parses header obtains requested file from server’s file system creates HTTP response message: header lines + file sends response to client after creating server, you can request file using a browser (eg IE explorer) see the following slides for details.

Example: WebServer.java import java.io.*; import java.net.*; import java.util.*; class WebServer { public static void main(String argv[ ]) throws Exception { String requestMessageLine; String fileName; ServerSocket listenSocket = new ServerSocket(6789); Socket connectionSocket = listenSocket.accept(); BufferedReader inFromClient = new BufferedReader( new InputStreamReader(connectionSocket.getInputStream())); DataOutputStream outToClient = new DataOutputStream(connectionSocket.getOutputStream()); contain the first line in the HTTP request message contain the file name of the requested file When a request for a connection arrives, the accept( ) method of listenSocket creates a new object, connectionSocket, of type Socket. two streams are created

Example: WebServer.java (cont.) requestMessageLine = inFromClient.readLine(); StringTokenizer tokenizedLine = new StringTokenizer(requestMessageLine); If(tokenizedLine.nextToken().equals("GET")) { fileName = tokenizedLine.nextToken(); if (fileName.startsWith("/") == true ) fileName = fileName.substring(1); File file = new File(fileName); int numOfBytes = (int) file.length(); FileInputStream inFile = new FileInputStream (fileName); byte[ ] fileInBytes = new byte[ ]; inFile.read(fileInBytes); reads the first line of the HTTP request message. tokenizedLine holds the request line with each of the "words" GET, file_name, and HTTP/1.1 placed in a separate placeholder called a token. removes the backslash that may precede the filename. These commands determine the size of the file and construct an array of bytes of that size. The name of the array is fileInBytes. reads from the stream inFile to the byte array fileInBytes

Example: WebServer.java (cont.) sends the mandatory status line: HTTP/1.1 200 Document Follows, followed by a carriage return and a line feed. outToClient.writeBytes( "HTTP/1.0 200 Document Follows\r\n"); if (fileName.endsWith(".jpg")) outToClient.writeBytes("Content-Type:image/jpeg\r\n"); if (fileName.endsWith(".gif")) outToClient.writeBytes("Content-Type:image/gif\r\n"); outToClient.writeBytes("Content-Length: " +numOfBytes + "\r\n"); outToClient.writeBytes("\r\n"); outToClient.write(fileInBytes, 0, numOfBytes); connectionSocket.close(); } else System.out.println("Bad Request Message"); } } ◊ to transfer a GIF image, server prepares the header line Content-Type: image/gif. ◊ to transfer a JPEG image, server prepares the header line Content-Type: image/jpeg. sends a content-length header line and a mandatory blank line sends the requested file, fileInBytes, to the TCP send buffer closing the socket connectionSocket

Code Descriptions-WebServer.java The first half of the program is almost identical to TCPServer.java. As with TCPServer.java, we import the java.io and java.net packages. In addition to these two packages we also import the java.util package, which contains the StringTokenizer class, which is used for parsing HTTP request messages.

Code Descriptions-WebServer.java Looking now at the lines within the classWebServer, we define two string objects: String requestMessageLine; String fileName; The object requestMessageLine is a string that will contain the first line in the HTTP request message. The object fileName is a string that will contain the file name of the requested file.

Code Descriptions-WebServer.java The next set of commands is identical to the corresponding set of commands in TCPServer.java. ServerSocket listenSocket = new ServerSocket(6789); Socket connectionSocket = listenSocket.accept(); Two socket-like objects are created. The first of these objects is listenSocket, which is of type ServerSocket. The object listenSocket is created by the server program before receiving a request for a TCP connection from a client. It listens at port 6789 and waits for a request from some client to establish a TCP connection. When a request for a connection arrives, the accept() method of listenSocket creates a new object, connectionSocket, of type Socket.

Code Descriptions-WebServer.java BufferedReader inFromClient = new BufferedReader(new InputStreamReader (connectionSocket.getInputStream())); DataOutputStream outToClient = new DataOutputStream(connectionSocket. getOutputStream()); Next two streams are created: the BufferedReader inFromClient and the DataOutputStream outToClient. The HTTP request message comes from the network, through connectionSocket and into inFromClient; the HTTP response message goes into outToClient, through connectionSocket and into the network.

Code Descriptions-WebServer.java The remaining portion of the code differs significantly from TCPServer.java. requestMessageLine = inFromClient.readLine(); The above command reads the first line of the HTTP request message. This line is supposed to be of the form: GET file_name HTTP/1.1

Code Descriptions-WebServer.java Our server must now parse the line to extract the filename. StringTokenizer tokenizedLine = new StringTokenizer(requestMessageLine); if (tokenizedLine.nextToken().equals("GET")) { fileName = tokenizedLine.nextToken(); They parse the first line of the request message to obtain the requested filename. The object tokenizedLine can be thought of as the original request line with each of the "words" GET, file_name, and HTTP/1.1 placed in a separate placeholder called a token.

Code Descriptions-WebServer.java The server knows from the HTTP RFC that the file name for the requested file is contained in the token that follows the token containing "GET." This file name is put in a string called fileName. if (fileName.startsWith("/") == true ) fileName = fileName.substring(1); The purpose of the above if statement in the above code is to remove the backslash that may precede the filename.

Code Descriptions-WebServer.java FileInputStream inFile = new FileInputStream (fileName); The above command attaches a stream, inFile, to the file fileName. byte[] fileInBytes = new byte[]; inFile.read(fileInBytes); These commands determine the size of the file and construct an array of bytes of that size. The name of the array is fileInBytes. The last command reads from the stream inFile to the byte array fileInBytes. The program must convert to bytes because the output stream outToClient may only be fed with bytes.

Code Descriptions-WebServer.java Now we are ready to construct the HTTP response message. To this end we must first send the HTTP response header lines into the DataOutputStream outToClient: outToClient.writeBytes("HTTP/1.0 200 Document Follows\r\n"); The above and next two commands are particularly interesting. These commands prepare the header lines for the HTTP response message and send the header lines to the TCP send buffer. The above command sends the mandatory status line: HTTP/1.1 200 Document Follows, followed by a carriage return and a line feed.

Code Descriptions-WebServer.java if (fileName.endsWith(".jpg")) outToClient.writeBytes("Content-Type: image/jpeg\r\n"); if (fileName.endsWith(".gif")) outToClient.writeBytes("Content-Type: image/gif\r\n"); These tow command lines prepare a single content-type header line. If the server is to transfer a GIF image, then the server prepares the header line Content-Type: image/gif. If the server is to transfer a JPEG image, then the server prepares the header line Content-Type: image/jpeg. (In this simple Web server, no content line is sent if the object is neither a GIF nor a JPEG image.)

Code Descriptions-WebServer.java outToClient.writeBytes("Content-Length: " + numOfBytes + "\r\n"); outToClient.writeBytes("\r\n"); The server then prepares and sends a content-length header line and a mandatory blank line to precede the object itself that is to be sent. We now must send the file FileName into the DataOutputStream outToClient.

Code Descriptions-WebServer.java We can now send the requested file: outToClient.write(fileInBytes, 0, numOfBytes); The above command sends the requested file, fileInBytes, to the TCP send buffer. TCP will concatenate the file, fileInBytes, to the header lines just created, segment the concatenation if necessary, and send the TCP segments to the client. connectionSocket.close(); After serving one request for one file, the server performs some housekeeping by closing the socket connectionSocket.

Code Descriptions-WebServer.java To test this Web server, install it on a host. put some files in the host. Then use a browser running on any machine to request a file from the server. When you request a file, you will need to use the port number that you include in the server code (for example, 6789). So if your server is located at somehost.somewhere.edu, the file is somefile.html, and the port number is 6789, then the browser should request the following: http://somehost.somewhere.edu:6789/somefile.html

Socket Programming: References C-language tutorial (audio/slides): “Unix Network Programming” (J. Kurose), http://manic.cs.umass.edu/~amldemo/courseware/intro. Java-tutorials: “All About Sockets” (Sun tutorial), http://www.javaworld.com/javaworld/jw-12-1996/jw-12-sockets.html “Socket Programming in Java: a tutorial,” http://www.javaworld.com/javaworld/jw-12-1996/jw-12-sockets.html

Chapter 2 Outline 2.1 Principles of app layer protocols clients and servers app requirements 2.2 Web and HTTP 2.3 FTP 2.4 Electronic Mail SMTP, POP3, IMAP 2.5 DNS 2.6 Socket programming with TCP 2.7 Socket programming with UDP 2.8 Building a Web server 2.9 Overlay Networks P2P Networks Unstructured Structured Content Networks Network Caches Content Distribution Networks (CDNs) Server Farms

Overlay networks A network built on top of one or more existing networks. Generally defined as “a routing (relay) system among involved nodes” Relay nodes may become bottlenecks. Independent from the underlying (physical) network. Applications neglect the network topology Adds additional layers of indirection/virtualization. Our system can be classified to a kind of overlay network. Overlay network is a virtual network constructed on application layer in distributed computing environment. It is generally defined as a routing system among involved nodes. All communications on overlay network are independent from the physical network. So we cannot predict the communication cost and design applications for effective communication on actual network topology. On the other hand, only the name space and the method for direct communication between any pair of nodes are provided in our system. Because we eliminate the relay node, high throughput and system size scalability are available. Also, the application programming, which is conscious of network topology, is available for effective communication among participating nodes. In our system, we provide features to support not only simple applications but also a wider framework to build some applications.

Overlay networks over TCP/IP Network Edge between peer X and Y if there is a TCP connection. All active nodes and edges is overlay network. An edge is not a physical link. A given node (peer) will typically be connected with <10 overlay neighbors. Y X Overlay Network TCP/IP (Underlying Network) X Y

Overlay Networks Overlay Network A node is a destination/source peer and a router (relay). Overlay Network Y X X Y TCP/IP – Underlying (Physical Network)

Abstract of an Overlay Network Architecture

Overlay Networks Examples P2P Structured CAN, Chord, Pastry Unstructured Napster, Gnutella, BitTorrent, KaZaA Content Networks Network Caches Content Distribution Networks (CDNs) Server Farms

Examples of p2p Applications File-sharing applications Napster, KazaA, Gnutella, BitTorrent, … E-Commerce Systems eBay, … Parallel Processing Distributed computing (grid?), SETI (Internet-connected computers used for Search for Extraterrestrial Intelligence), … Collaboration Distributed (Internet) games, Instant messaging, Shared Application, … Distributed databases Mariposa, Litwin, Yokota, Plaxton, … Mobile Ad hoc networks (Unstructured networks) Application-level multicast …

Chapter 2 Outline 2.1 Principles of app layer protocols clients and servers app requirements 2.2 Web and HTTP 2.3 FTP 2.4 Electronic Mail SMTP, POP3, IMAP 2.5 DNS 2.6 Socket programming with TCP 2.7 Socket programming with UDP 2.8 Building a Web server 2.9 Overlay Networks P2P Networks Content Networks Network Caches Content Distribution Networks (CDNs) Server Farms

P2P File Sharing User1 chooses one of the peers, User2. Example User1 runs P2P client application on his notebook computer. Intermittently connects to Internet; gets new IP address for each connection. Asks for “network e-book” Application displays other peers that have copy of network e-book. User1 chooses one of the peers, User2. File is copied from User2’s PC to User1’s notebook: HTTP While User1 downloads, other users uploading from User1. User1’s peer is both a Web client and a transient Web server. All peers are servers = highly scalable!

Example: University of Washington (2002) UW is an HTTP content provider Exported 16.65 TB - Imported 3.44 TB Bandwidth consumption (in + out) 0.2% Akamai (overnet. eDonkey) [content delivery networks] 6.04% Gnutella (p2p system) 14.3% WWW 36.9% Kazaa (p2p system) 43% other TCP protocols, e.g. mail, streaming video

File Distribution: Server-Client vs P2P Question : How much time to distribute file from one server to N peers? uS: server upload bandwidth ui: peer i upload bandwidth Server u2 di: peer i download bandwidth u1 d1 d2 uS File, size F Network (with abundant bandwidth) dN uN

Throughput: Internet scenario End-to-end throughput constrained by bottleneck link Per-connection end-end Throughput: min(dc, us, R/10) In practice: dc or us is often bottleneck: R/10 >> dc R/10 >> uc us us us R dc dc dc 10 connections (fairly) share backbone bottleneck link R bits/sec

File distribution time: server-client Server sequentially sends N copies: NF/us time Server sends 1 copy for N clients in parallel: F/(us/N)=NF/us time Client i takes F/di time to download = Tcs = max { NF/us , F/min(di) } i Time to distribute F to N clients using client/server approach = increases linearly in N (for large N)

File distribution time: P2P server must send one copy: F/uS time client i takes F/di time to download NF bits must be downloaded (aggregate) fastest possible upload rate: uS + Sui TP2P = max { F/uS , F/min(di) , NF/(uS + Sui) } i

Server-client vs. P2P: example Client upload rate = ui , F/ui = 1 hour, uS = 10ui , dmin ≥ uS NF/us =1 h TP2P = max { F/uS , F/min(di) , NF/(uS + Sui) } =10F/20ui = 0.5 h

Properties of P2P Systems No central control, no central database No hierarchy Every node is both a client and a server The communication between peers is symmetric No global view of the system Scalability Availability for any peer Peers are autonomous System globally unreliable Robustness and security issues

Key Issues for P2P Systems Join/leave How do nodes join/leave? Who is allowed? Publish How to advertise a file? Search and retrieval How to find content? How are metadata indexes built, stored, distributed? Content Distribution Where is content stored? How is it downloaded and retrieved?

Types Publish and Search Basic Strategies: Hybrid or Centralized (BitTorrent, Napster) Pure or Decentralized (Gnutella, Freenet) Flood the query Super Peers or Hierarchical (KaZaA, DNS) Redundant Super Peers (DNS) Rout the query

P2P Information Management How Much P2P is Involved? P2P User Interaction P2P Application P2P Information Management eBay yes no Napster Gnutella, Freenet

Hybrid or Centralized P2P In a hybrid P2P system, some operations are performed at a central server. For other operations, the peers have equal status. Example: Napster file download Central server contains an index of the files located at all the peers currently available. Peer 1 asks server where a file is located. Server returns address of Peer 2 who has the file. Peer 1 asks Peer 2 (directly) for the file.

Hybrid System 1 2 1 5 5 2 5 3 3 4 Give me 4 5 Who has 1 5 4

Napster as a Centralized System Original “Napster” design Peers to notify server when they join and leave: (IP Add, Port Num) Peers to notify server if their available files change: Contents Peers to run a “server” program that listens on a port for download requests. 2) User1 queries for “network e-book”, 3) User1 requests file from User2. User2 location server 1 ° 1 ° 1 3 1 peers ° 2 1 2 3 User1

Centralized model (Napster) Peer Z IP: a.b.c.d location server INTERNET register  Document x! x  OK: Peer Z IP = a.b.c.d Document x?

Centralized Model (BitTorrent) Released in the summer of 2001 Uses basic ideas from game theory to largely eliminate the free-rider problem "free riders" are those who consume more than their fair share. All previous systems could not deal with this problem well.

Basic Idea Chop file into many pieces (chunks). Replicate DIFFERENT pieces on different peers as soon as possible. As soon as a peer has a complete piece, it can trade it with other peers. Hopefully, we will be able to assemble the entire file at the end.

Basic Components Seed Leacher A Torrent file A Tracker Peer that has the entire file Leacher Peer that has an incomplete copy of the file A Torrent file Passive component Files are typically fragmented into 256KB pieces (chunks) The torrent file lists SHA1 hashes of all the pieces to allow peers to verify integrity Typically hosted on a web server A Tracker Active component Allows peers to find each other Returns a random list of peers

P2P File distribution: BitTorrent torrent: group of peers exchanging chunks of a file tracker: tracks peers participating in torrent The peer obtain list of peers trading chunks The peer

BitTorrent - 1 File divided into 256KB chunks. Peer joining torrent: has no chunks, but will accumulate them over time registers with tracker to get list of peers, connects to subset of peers (“neighbors”). While downloading, peer uploads chunks to other peers. Peers may come and go. Once peer has entire file, it may (selfishly) leave or (altruistically) remain.

BitTorrent - 2 Pulling Chunks At any given time, different peers have different subsets of file chunks. Periodically, a peer asks each neighbor for list of chunks that they have. The peer sends requests for its missing chunks. The rarest first! Sending Chunks: tit-for-tat A peer sends chunks to four neighbors currently sending it chunks at the highest rate re-evaluate top 4 every 10 secs every 30 secs: randomly select another peer, starts sending chunks newly chosen peer may join top 4 “optimistically unchoke”

Choking One of BitTorrent’s most powerful idea is the choking mechanism. It ensures that nodes cooperate and eliminates the free-rider problem. Cooperation involves uploaded pieces that you have to your peer. Choking is a temporary refusal to upload; downloading occurs as normal. Connection is kept open so that setup costs are not borne again and again Based on game-theoretic concepts Tit-for-tat strategy in Repeated Games

BitTorrent: Tit-for-tat (1) A “optimistically unchokes” B. (2) A becomes one of B’s top-four providers; B reciprocates. (3) B becomes one of A’s top-four providers. B A With higher upload rate, can find better trading partners & get file faster!

Hybrid: Ads and DisAds Advantages: Disadvantages Search is generally efficient Disadvantages High cost on centralized index Performance & scalability bottleneck Vulnerable! Highly visible target

Pure P2P-1 All peers have equal roles in all responsibilities Example: Gnutella In order to find an item, Peer 1 asks his neighbors if they have it. They ask their neighbors, and so on A message has a “time to live” which determines how long it will be sent along. Why is this needed? Different search strategies will be discussed later

Pure P2P-2 What happens when a peer (say, Peer 34) is identified, who has the file that Peer 1 wants? 2 Different options are in use in different systems Peer 34 directly sends file to Peer 1 Peer 34 sends the file back in the same route that it was requested Advantages/disadvantages of each option?

Pure P2P ? ? ? ? ?

Pure P2P: Option 1

Pure P2P: Option 2

Gnutella: protocol Query message HTTP sent over existing TCP connections Gnutella on ports 6346 and 6347 peers forward Query message QueryHit sent over reverse path Scalability: limited scopeflooding QueryHit Query File Transfer: http Query QueryHit Query QueryHit Query Query

Gnutella Protocol Messages Broadcast Messages Ping: initiating message (“I’m here”) Query: search pattern and TTL (time-to-live) Back-Propagated Messages Pong: reply to a ping, contains information about the peer Query response: contains information about the computer that has the needed file Node-to-Node Messages GET: return the requested file PUSH: push the file to me

Peer Joining in Gnutella Joining peer X must find some other peer in Gnutella network; use list of candidate peers. X sequentially attempts to make TCP connection with peers on list until connection setup with Y. X sends Ping message to Y; Y forwarded Ping message. All peers receiving Ping message respond with Pong message. X receives many Pong messages. It can then setup additional TCP connections. Peer living!

Topology of Gnutella Backbone Backbone + Outskirts

Pure: Ads and DisAds Advantages: Disadvantages: Easy to start No maintenance needed Hard to shut down privacy Disadvantages: Inefficient search Lot of network traffic Bottlenecks (limited peers)

Super Peers Make use of heterogeneity Super-peers index clients’ files Powerful peers serve as super peers Weaker peers act as clients Example: KaZaA Super-peers index clients’ files Requires updates on join/leave/update Queries handled at super-peer level. Query evaluation between super-peers similar to query evaluation between peers in a pure system.

Super Peers

Decentralized Directory (KaZaA) Each peer is either a group leader or assigned to a group leader. Group leader tracks the content in all its children. Each peer queries its group leader. Group leader may query other group leaders. KazaA on port 1214 ordinary peer (ordinary node) group-leader peer (super node) neighboring relationships in overlay network

KazaA Query Each file has a hash and a descriptor. Client sends keyword query to its group leader. Group leader responds with matches; For each match: metadata, hash, IP address If group leader forwards query to other group leaders, they respond with matches. Client then selects files for downloading; HTTP requests using hash as identifier sent to peers holding desired file.

Super Peers: Ads and DisAds Advantages: All the advantages of pure Takes advantage of different peer capabilities Disadvantages: Super nodes might become bottlenecks for clients Clients are disconnected when super peer leaves Inefficient search (but better than Pure P2P) Maintenance costs for super peers

Redundant Super Peers Like super peers, but each super-peer is k-redundant, i.e., there are k nodes sharing the super peer load all k nodes have the same indexes of all the clients clients ask super-peer queries in a round robin fashion

Super Peers

Redundant Super Peers

Redundant Super Peers: Ads and DisAds Advantages All advantages of super-peers load of queries is shared among peers less points of fail Disadvantages additional maintenance: nodes must notify all redundant super-peers upon joining, leaving, updating files Additional connections between super-peers required

Non-structured system Two phases (like Napster) Localization + exchange No server Open source gnutella.wego.com Distributed search The query is flooded Loop avoidance Limited TTL (not all nodes are visited)

Structured systems (DHTs) Based on Distributed Hash Tables (DHTs) No flooding Exact matches Overhead Gnutella-like  O(n) DHT  O(log n) Examples CAN, Pastry, Chord, Kademlia, Tapestry, P-Grid, ...

Content-Addressable Networks (CAN) Provides a large scale distributed hash table Keys are mapped into values CAN defines a d-dimensional virtual space No relationship with the physical space Query  O(n1/d) Neighbors  O(d) The virtual space is completely distributed among the peers Each peer is responsible for one share of the space The peer that is responsible for region R is also responsible for the values inside R Documents must be uniquely identified

Example

Example 1

Example 1 2

Example 1 2 3 3

Example 2 4 1 3

Example 4 2 4 1 5 3

Example 4 2 1 6 5 3

Example 2 3 6 4 5 1 7

Association ID  node Hash 2 3 6 4 5 1 7 Ex: Node 3 holds this document

Association ID  node 2 3 6 4 5 1 7

Application-layer multicast Native multicast  not yet completely deployed ALM  easier/faster to implement Scalability  states at end-systems High-level support

Application-layer multicast source

Many works and projects Gnutella and Napster @sourceforge.net

JXTA (Sun) Open platform for p2p cooperation Interoperability Any system/peer/application Platform independency Languages (C, Java, etc) Systems platforms (Unix, Windows, etc) Networking platforms (802.11, Bluetooth, TCP/IP, etc) Ubiquity Sensors, PDAs, routers, desktops, laptops, storage systems

JXTA (Sun) Objectives Projects Find peers and resources Share files with anyone across the network Create a particular group of peers across different networks Communicate securely with peers across public networks Projects Applications (24 projects) Core (13 projects) Demos (3 projects) Forge (15 projects) Other (12 projects) Services (24 projects)

JXTA (Sun) Security Peer (Desktop, cell phone, PDA, etc.) Peer Groups applications Peer (Desktop, cell phone, PDA, etc.) Security Peer Groups Peer Pipes Peer Monitoring JXTA Community Services Sun JXTA Services JXTA Shell Peer Commands JXTA Community Applications JXTA services JXTA core

JXTA Protocols Transport Transport Peer Membership Protocol PeerGroup PeerGroup Peer Membership Protocol Peer Membership Protocol Peer Peer Peer Information Protocol Peer Information Protocol Peer Binding Protocol Peer Binding Protocol Set of six protocols specifically designed for ad hoc, pervasive and multi-hop peer-to-peer network computing Using the protocols peers can cooperate to form self organized and self-configured peer groups independently of their positions in the network Peer Discovery Protocol Peer Discovery Protocol Resolver Peer Resolver Protocol Resolver Peer Resolver Protocol Endpoint Peer Endpoint Routing Protocol Endpoint Peer Endpoint Routing Protocol Transport Transport

JXTA Applications JXTA Content Manager Service (CMS) InstantP2P Allows sharing and retrieving of content InstantP2P Peer-to-peer messenger service Group chat File sharing Net Services Standards such as WSDL (Web Service Description Language ), ebXML (Electronic Business XML), SOAP(Simple Object Access Protocol), UPnP (UPnP is a protocol for data transmission), etc.

Final remarks P2P implies a very large spectrum of areas High interest in both academicals/industrials Much has already been done, but no conclusions are definitive IPv6 and P2P NAT, firewalls, IPv6 as an overlay Many open issues Trust, security, scalability, QoS, etc.

Chapter 2 Outline 2.1 Principles of app layer protocols clients and servers app requirements 2.2 Web and HTTP 2.3 FTP 2.4 Electronic Mail SMTP, POP3, IMAP 2.5 DNS 2.6 Socket programming with TCP 2.7 Socket programming with UDP 2.8 Building a Web server 2.9 Overlay Networks P2P Networks Content Networks Network Caches Content Distribution Networks (CDNs) Server Farms

Content in Network networks browsers browsers browsers server

The Problems Problems: Congestion in routers Overloading servers networks Problems: Congestion in routers Overloading servers High latency for users browsers browsers routers server browsers server

Content Network: Ideas browsers Ideas: Network Caches Content Distribution Networks (mirror) Server Farms. Replicated Content (mirror) content sink (cache) router server content source

Servers: Caching, and Mirror (content distribution strategies) The conventional Technologies are: Proxy cache server, and Mirror server They are cost effective technologies which enhance the performance, namely: Reducing the latency of access. Reducing the network bandwidth required for document transfers. Reducing the demand on servers with very popular documents. Improving the security of electronic file transfer.

Caching: idea (information distribution strategies) Basic idea in caching: Store the frequently retrieved documents into local files or proxies for future use. Caching on client disk: Browsers maintain small caches of previously-viewed pages on the user’s local disk. ( example: Documents and Settings\WELCOME\Local Settings\Temporary Internet Files ) Network caching: Caching system is located on machine in the path from multiple clients to multiple servers.

Aadvantages of web caching It improves access speed and cuts down on network traffic. increase availability in the web by replicating documents among many servers (cache). Also: It reduces the server load. Data can be accessed when remote server is down (enhanced robustness). It allows analysis of organization usage patterns.

Disadvantages of web caching Data not updated automatically. Cache miss can cause increase in latency (extra proxy processing). Bottleneck effect: limited number of clients per proxy. A single proxy is a single point of failure. Information providers can not monitor number of visits per site.

More about Web Caching Cache acts as both client and server. Cache can do up-to-date check using If-modified-since HTTP header Issue: should cache take risk and deliver cached object without checking? Heuristics are used. Typically cache is installed by ISP (university, company, residential ISP)

Caching: proxy (information distribution strategies) Caching proxy servers can be located near a large community of users such as: on a campus network, at an intranet server, or at an ISP server. origin servers origin servers HTML, FTP, Search, Database LAN router Internet router Internet institutional network institutional Cache(Proxy server) Proxy Architecture

Web Caches (Proxy web-Server) Goal: satisfy client request without involving origin server. origin server user sets browser: Web accesses via cache (proxy) browser sends all HTTP requests to cache (proxy) object in cache: cache returns object else cache requests object from origin server, then returns object to client Proxy server HTTP request HTTP request client HTTP response HTTP response HTTP request HTTP response client origin server

Content Distribution Model λ :Arrival (request) Service Providing Entity Departure (response) λ :Arrival rate (request) Departure rate (response) Service Providing Entity in service waiting S Offered load Throughput Service Providing Entity examples: Server LAN Access Link …

Model Parameters S Offered load Throughput λ :Arrival rate (request) Departure rate (response) Service Providing Entity in service waiting S Offered load Throughput λ : Average Arrival rate [requests/sec] S: Average Service time [sec/response]; Example1: Server λ : Average Arrival rate [file request/sec] S: Average Service time [sec/file response]; Example2: Link, LAN λ : Average Arrival rate [bit/sec] S: Average Service time [sec/file] = L/R; L: file length, R: Link Bandwidth

Model Equations (M/M/1) Utilization of Service Providing Entity = U = λ x S ; ( λ ≤ 1/S ) delay (Response Time)=Service Time + Waiting Time; delay = S / (1-U) ; Waiting Time (Queuing Delay) = delay – S

Equations for Link, LAN λ = a [files/sec]; rate of files entering into the link, LAN S = L/R [sec/file]; the time needed to pass a file through the Link, LAN U = λxS = La/R [ ]; Link, LAN Utilization I = U [ ]; Link, LAN Traffic Intensity delay = S/(1-U) = L/[R(1-U)] [secs/file]

Link Response Time λ’ λ Uuplink = Lreq λ /Ruplink = Iuplink LAN router Internet router Files from origin servers Rdownlink Request from clients λ λ’ Uuplink = Lreq λ /Ruplink = Iuplink delayup = Sup/(1-Uuplink) [sec/req] Udownlink = Lres λ’ /Rdownlink = Idownlink delaydown = Sdown/(1-Udownlink) [sec/res] delay = delayup + delaydown ≈ delaydown It is because: Lres >> Lreq

Caching Example1 (1) Assumptions Average request rate from institution’s browser to origin serves: λ = 15reqs/sec Average request massage size ≈ 0 bit Average object size = L= 100,000 bits Internet delay ≈ 2 sec (average) Time spent in the Internet origin servers public Internet Rlink =1.6 Mbps (up and down) LAN Rlan =10 Mbps

Caching Example1 (2) delay is calculated for down stream only: LAN Service time= Slan = L/Rlan = 10 msec LAN Utilization = La/Rlan = 0.15 = 15% LAN delay =S/(1-U) = 10/0.85 =11.8 msec Link Service time Slink = L/Rlink = 62.5 msec Link Utilization (Traffic Intensity) = La/Rlink = 93.75% Link delay = 62.5/0.0625 = 1000 msec delay = Internet delay + access-link delay + LAN delay = 2000 + 1000 + 11.8 =3011.8 msecs qrouter = 1000 – 62.5 = 937.5 msec origin servers public Internet LAN Rlink =1.6 Mbps access-link Rlan =10 Mbps 937.5 msecs 62.5 msecs 11.8 msecs

Caching Example2 Upgrade the link: LAN Service time= Slan = L/Rlan = 10 msec LAN Utilization = La/Rlan = 0.15 = 15% LAN delay = S/(1-U) = 10/0.85 = 11.8 msec Link Service time Slink = L/Rlink = 10 msec Link Utilization (Traffic Intensity) = La/Rlink = 15% Link delay = 11.8 msec delay = Internet delay + access-link delay + LAN delay = 2000 + 11.8 + 11.8 =2023.6 msecs qrouter = 11.8 – 10 = 1.8 msec origin servers public Internet LAN Rlink =10 Mbps access-link Rlan =10 Mbps 1.8 msecs 10 msecs 11.8 msecs Often a costly upgrade

Caching Example3 Install cache Consequence Network cache origin servers Install cache suppose hit rate is 0.4 Consequence 40% requests will be satisfied almost immediately 60% requests satisfied by origin server utilization of access link (reduced to 60%) = 0.6 *93.75= 56.25% Link delay = 62.5/0.4375 = 143 msec delay = 0.4* 11.8 + 0.6 x (2000 + 143 + 11.8) = 1297.6 msec public Internet 80.5 msecs Rlink =1.6 Mbps access-link 62.5 msecs LAN 11.8 msecs Network cache Rlan =10 Mbps

Content Distribution Metric Cost × Performance analysis is the key metric to the effectiveness of an information (content) distributed system.

Note: Cache miss Flows Cache miss flows and utilization: origin servers Cache miss flows and utilization: There to sequential flows: Flow of missed files from origin servers to cache server Flow of the missed files from cache server to the clients. Both above mentioned flows do not appear into LAN at the same time. At any given time there is one of them in the LAN. So, LAN utilization = La/Rlan = 0.15 = 15% public Internet LAN Network cache Rlan =10 Mbps

When are DNS Queries Unnecessary? Client is configured to use a proxy Client sends all HTTP requests through a proxy. Then, the proxy takes care of issuing the DNS request. Requested Web resource is locally cached Client has locally cached http://www.irib.ir/1387/leadstory.html No need to request the resource, so no need to query. Client recently queried (DNS) for a host name User recently visited http://www.irib.ir/ So, the client may be locally caching the resulting IP address.

Content Distribution Network: idea The content (information) providers are the CDN customers. Content replication (mirror) CDN company installs hundreds of CDN servers throughout Internet. in lower-tier ISPs, close to users. CDN replicates its customers’ content in CDN servers. When provider updates content, CDN updates servers. origin server in North America CDN distribution node CDN server in S. America in Europe in Asia

CDN: Architectural Layout Request Routing(RR) 1 Origin Server 2a Distribution Node RR sends client’s OS redirects client 3 Client sends request 2b Client requests request Mirror Server 5 6 Client 4 Origin server informs RR of Content Availability. Content Pushed to Distribution System. Client Requests Content, Origin server redirects client to RR. Client sends query to RR. RR finds the most suitable Mirror Server. Mirror server receives client request and provides services.

More about CDNs routing requests not just Web pages CDN creates a “map”, indicating distances from leaf ISPs and CDN nodes when query arrives at authoritative DNS server: server determines ISP from which query originates uses “map” to determine best CDN server not just Web pages streaming stored audio/video streaming real-time audio/video CDN nodes create application-layer overlay network

Content networks: Server Farms Simple solution to the content distribution problem: deploy a large group of servers www.x.com (Copy 1) Request from grad.ut.edu Virtual IP addr. Virtual IP addr. Request from iust.ac.ir Request from grad.ut.edu www.x.com (Copy n) Request from iust.ac.ir L4-L7 Switch (Load balancer) Virtual IP addr. Arbitrate client requests to servers using an “intelligent” L4-L7 switch Widely used today

Our study of network apps now complete! Chapter 2: Summary1 Our study of network apps now complete! application service requirements: reliability, bandwidth, delay. client-server paradigm. Internet transport service model. connection-oriented, reliable: TCP unreliable, datagrams: UDP specific protocols: HTTP FTP SMTP, POP, IMAP DNS socket programming content distribution P2P caches, CDNs, Server Farms.

Most importantly: learned about protocols Chapter 2: Summary2 Most importantly: learned about protocols typical request/reply message exchange: client requests info or service server responds with data, status code message formats: headers: fields giving information about data data: information being communicated control vs data messages in-band, out-of-band centralized vs. decentralized stateless vs. stateful reliable vs. unreliable msg transfer “complexity at network edge” security: authentication

References & Links Complimentary Hyperlinks References and Hyperlinks This part provides hyperlinks to interesting (and hopefully useful) computer-networking resources. Most of these resources provide complimentary information to the material in chapter 2. If you're asked to write a paper pertaining to a specialized topic in computer networking, these resources should serve as a good starting point for your research. References and Hyperlinks

Complimentary Hyperlinks1 Unix Network Programming http://www-aml.cs.umass.edu/~jsmanic/unetpgm Online tutorial on Unix socket programming, using slides and audio, from author James Kurose. Tutorial on Java sockets http://www.javaworld.com/javaworld/jw-12-1996/jw-12-sockets.html This tutorial presents an introduction to sockets programming and shows how to write client/server applications in Java.

Complimentary Hyperlinks2 The World Wide Web Consortium (W3C) http://www.w3.org/ The W3C was founded in 1994 to develop common protocols for the evolution of the World Wide Web. This is an outstanding site with fascinating information on emerging Web technologies, protocols, and standards. Web Protocols and Practice http://www.aw.com/catalog/academic/product/1,4096,0201710889,00.html Web Protocols and Practice: HTTP/1.1, Networking Protocols, Caching, and Traffic Measurement, a comprehensive and readable book by Balachander Krishnamurthy and Jennifer Rexford on Web protocols and related topics. Cookie Central http://www.cookiecentral.com/ Everything you ever wanted to know about cookies, including how they infringe on users' privacy. The IMAP Connection http://www.imap.org/ Everything you always wanted to know about IMAP, including history, direct access to relevant RFCs, and product information.

Complimentary Hyperlinks3 ISC BIND http://www.isc.org/products/BIND DNS name servers use the Berkeley Internet Name Domain (BIND) software. You can learn about it (and download it) here. Exploring Name Servers http://www.unhooked.net/cgi-bin/host nslookup, host, and dig are client programs available for exploring the contents of name servers in the Internet. Several sites, including the one listed above, allow you to access these programs through a Web browser. All of these programs mimic DNS clients. They send a DNS query message to a name server (which can often be supplied by the user), and they receive a corresponding DNS response. They then extract information (e.g., IP addresses, whether the response is authoritative, etc.) and present the information to the user.

Complimentary Hyperlinks 4 Online Java Tutorial http://java.sun.com/docs/books/tutorial Do you what to learn Java before doing the programming exercises? This is an excellent place to start. Akamai Homepage http://www.akamai.com/ Homepage for leading Content Distribution Network (CDN) company. Developers Site for Gnutella http://www.gnutelladev.com/ Includes links to source code as well as links to documentation of the protocol.

References and Hyperlinks1 A note on Internet Request for Comments (RFCs): Copies of Internet RFCs are maintained at multiple sites. The RFC URLs below all point into the RFC archive at the Information Sciences Institute (ISI), maintained the the RFC Editor of the Internet Society (the body that oversees the RFCs). Other RFC sites include http://www.faqs.org/rfc, http://www.pasteur.fr/other/computer/RFC (located in France), and http://www.csl.sony.co.jp/rfc/ (located in Japan).  Internet RFCs can be updated or obsoleted by later RFCs. We encourage you to check the sites listed above for the most up-to-date information. The RFC search facility at ISI, http://www.rfc-editor.org/rfcsearch.html, will allow you to search for an RFC and show updates to that RFC.  [Abitz 1993] P. Albitz and C. Liu, DNS and BIND, O'Reilly & Associates, Petaluma, CA, 1993.  [Akamai 2000] Akamai homepage, http://www.akamai.com

References and Hyperlinks2 [Berners-Lee 1994] T. Berners-Lee, R. Cailliau, A. Luotonen, H. Frystyk Nielsen, and A. Secret, "The World-Wide Web," Commun. ACM 37, 8 (Aug. 1994), Pages 76-82  [BIND 2000] Internet Software Consortium page on BIND, http://www.isc.org/bind.html [Claffy 1998] K. Claffy, G. Miller, and K. Thompson, "The Nature of the Beast: Recent Traffic Measurements from an Internet Backbone," Proceedings of Inet '98 , (Geneva, Switzerland, July 1998), http://www.caida.org/outreach/resources/papers/Inet98/ [Cookie Central 2000] Cookie Central homepage, http://www.cookiecentral.com [Cusumano 1998] M.A. Cusumano and D.B. Toffle, Competing on Internet Time: Lessons from Netscape and its Battle with Microsoft, Free Press, 1998  [DNSNet] DNSNet page on DNS resources, http://www.dns.net/dnsrd/docs/

References and Hyperlinks3 [Frost 1994] J. Frost, "BSD Sockets: A Quick and Dirty Primer," http://world.std.com/~jimf/papers/sockets/sockets.html [Gauthier 1999] L. Gauthier, C. Diot, and J. Kurose, "End-to-end Transmission Control Mechanisms for Multiparty Interactive Applications on the Internet," Proceedings of IEEE Infocom '99, (New York, NY, Apr. 1999). ftp://ftp.sprintlabs.com/diot/infocom99-mimiaze.zip [Gay 1997] V. Gay and B. Dervella, "MHEGAM--A Multimedia Messaging System," IEEE Multimedia Magazine, Oct.-Dec. 1997, pp. 22-29. 

References and Hyperlinks4 [Heidemann 1997] J. Heidemann, K. Obraczka, and J. Touch, "Modeling the Performance of HTTP over Several Transport Protocols," IEEE/ACM Transactions on Networking, Vol. 5, No. 5 (Oct. 1997), pp. 616-630.  [Hess 1998] C. Hess, D. Lin and K. Nahrstedt, "VistaMail: An Integrated Multimedia Mailing System," IEEE Multimedia Magazine, Oct.-Dec. 1998, pp. 13-23.  [Huffaker 1998] B. Huffaker, J. Jung, D. Wessels, and K. Claffy, "Visualization of the Growth and Topology of the NLANR Caching Hierarchy," The 3rd Web Caching Workshop, June 1998, England. http://www.caida.org/tools/visualization/plankton/Paper/plankton.xml

References and Hyperlinks5 [IMAP 1999] The IMAP Connection, http://www.imap.org/ [ITU 1997] International Telecommunications Union, "Recommendation E.164/ I.331--The international public telecommunication numbering plan," May 1997. http://www.itu.int/itudoc/itu-t/rec/e/s_e164.html [Kurose 1996] J. F. Kurose, Unix Network Programming. http://manic.cs.umass.edu/~amldemo/courseware/intro.html [Luotonen 1998] A. Luotonen, Web Proxy Servers, Prentice Hall, Englewood Cliffs, New Jersey, 1998.  [McKusik 1996] Marshall Kirk McKusick, Keith Bostic, Michael Karels, and John Quarterman, The Design and Implementation of the 4.4BSD Operating System, Addison-Wesley, Reading, MA, 1996.  [Netcraft 2000] The Netcraft Web Server Survey, Netcraft Web Site, http://www.netcraft.com/survey/

References and Hyperlinks6 [Netscape Cookie 1999] Netscape, "Persistent Client State http Cookies. http://home.netscape.com/newsref/std/cookie_spec.html [Nielsen 1997] H. F. Nielsen, J. Gettys, A. Baird-Smith, E. Prud'hommeaux, H. W. Lie, and C. Lilley, "Network Performance Effects of HTTP/1.1, CSS1, and PNG," W3C Document, 1997 (also appears in Proceedings of ACM SIGCOMM '97, Cannes, France, pp. 155-166 ). http://www.acm.org/sigcomm/sigcomm97/papers/p102.html [NLANR 1999] A Distributed Testbed for National Information Provisioning, http://ircache.nlanr.net/ [Padhye 1999] J. Padhye and J. Kurose, "An Empirical Study of Client Interactions with a Continuous-Media Courseware Server," IEEE Internet Computing, Apr. 1999. ftp://gaia.cs.umass.edu/pub/Padh97:Empirical.ps.gz [Ramjee 1994] R. Ramjee, J. Kurose, D. Towsley, and H. Schulzrinne, "Adaptive Playout Mechanisms for Packetized Audio Applications in Wide-Area Networks," Proceeding IEEE Infocom 94. ftp://gaia.cs.umass.edu/pib/Ramj94:Adaptive.ps.Z [RFC 821] J. Postel, "Simple Mail Transfer Protocol," RFC 821, Aug. 1982. http://www.rfc-editor.org/rfc/rfc821.txt

References and Hyperlinks7 [RFC 854] J. Postel and J. Reynolds, "TELNET Protocol Specification," RFC 854. May 1993. http://www.rfc-editor.org/rfc/rfc854.txt [RFC 959] J. Postel and J. Reynolds, "File Transfer Protocol (FTP)," RFC 959, Oct. 1985. http://www.rfc-editor.org/rfc/rfc959.txt [RFC 1034] P. V. Mockapetris, "Domain Names--Concepts and Facilities," RFC 1034, Nov. 1987. http://www.rfc-editor.org/rfc/rfc1034.txt [RFC 1035] P. Mockapetris, "Domain Names--Implementation and Specification," RFC 1035, Nov. 1987. http://www.rfc-editor.org/rfc/rfc1035.txt [RFC 1945] T. Berners-Lee, R. Fielding, H. Frystyk, "Hypertext Transfer Protocol--HTTP/1.0," RFC 1945, May 1996 http://www.rfc-editor.org/rfc/rfc1945.txt [RFC 2045] N. Freed, N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies," RFC 2045, Nov. 1996. http://www.rfc-editor.org/rfc/rfc2045.txt

References and Hyperlinks8 [RFC 2046] N. Freed, N. Borenstein,, "Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types," RFC 2046, Nov. 1996. http://www.rfc-editor.org/rfc/rfc2046.txt [RFC 2068] R. Fielding, J. Gettys, J. Mogul, H. Frystyk, and T. Berners-Lee, "Hypertext Transfer Protocol--HTTP/1.1," RFC 2068, Jan. 1997. http://www.rfc-editor.org/rfc/rfc2068.txt. Obsoleted by RFC 2616.  [RFC 2186] K. Claffy and D. Wessels, "Internet Caching Protocol (ICP), version 2," RFC 2186, Sept. 1997. http://www.rfc-editor.org/rfc/rfc2186.txt [RFC 2616] R. Fielding, J. Gettys, J. Mogul, H. Frystyk, L. Masinter, P. Leach, T. Berners-Lee, R. Feilding, "Hypertext Transfer Protocol--HTTP/1.1," RFC 2616, June 1999. http://www.rfc-editor.org/rfc/rfc2616.txt [Ross 1997] K. W. Ross, "Hash-Routing for Collections of Shared Web Caches," IEEE Network Magazine, Vol. 11 (Nov.-Dec. 1997), pp. 37-45.  [Ross 1998] K. W. Ross, Distribution of Stored Information in the Web, An Online Tutorial, http://www.eurecom.fr/~ross/CacheTutorial/DistTutorial.html, 1998. 

References and Hyperlinks9 [Schurmann 1996] G. Schurmann, "Multimedia Mail," ACM Multimedia Systems, Oct. 1996, pp. 281-295.  [Squid 2000] Squid Web Proxy Cache, http://www.squid-cache.org/ [Stevens 1997] W.R. Stevens, Unix Network Programming, Volume 1: Networking APIs-Sockets and XTI, 2nd edition, Prentice-Hall, Englewood Cliffs, NJ, 1997.  [Turner 1999] D. A. Turner and K. W. Ross, "Continuous-Media Internet E-Mail: Infrastructure Inadequacies and Solutions," http://www.eurecom.fr/~turner/cmail1.html [Valloppillil 1997] V. Valloppillil and K. W. Ross, "Cache Array Routing Protocol," Internet Draft, <draft-vinod-carp-v1-03.txt>, June 1997.  [Yeager 1996] N. J. Yeager and R. E. McGrath, Web Server Technology, Morgan Kaufmann Publishers, San Francisco, 1996. © 2000-2001 by Addison Wesley Longman A division of Pearson Education

P2P-Working groups et al. A generic site on p2p from O'Reilly www.openp2p.com P2P working group www.peer-to-peerwg.org/ Internet2 p2p working group p2p.internet2.edu Peer-to-peer development (p2p-hackers) zgp.org/mailman/listinfo/p2p-hackers Interesting meeting www.codecon.org

P2P-References-1 Distributed Computing File sharing Distributed (www.distributed.net) SETI@home (www.seti.org) Genome@home (gah.stanford.edu) Folding@home (www.stanford.edu/group/pandegroup/folding) Global Grid Forum (www.globalgridforum.org) Globus Project (www.globus.org) File sharing Napster (www.napster.com) Gnutella (gnutella.wego.co) Kazaa (www.kazaa.com)

P2P-References-2 Distributed hash tables Ad hoc networking CAN (www.acm.org/sigs/sigcomm/sigcomm2001/p13-ratnasamy.pdf) Pastry (research.microsoft.com/~antr/Pastry) Chord (www.pdos.lcs.mit.edu/chord) Tapestry (www.cs.berkeley.edu/~ravenben/tapestry) Freenet (freenet.sourceforge.net) Kademlia (kademlia.scs.cs.nyu.edu) Ad hoc networking AODV (www.ietf.org/internet-drafts/draft-ietf-manet-aodv-13.txt) OLSR (www.ietf.org/internet-drafts/draft-ietf-manet-olsr-10.txt) Tribe (rp.lip6.fr/site_rp/_publications/350-79Viana.ps.gz)

P2P-References-3 Platforms Collaboration IPv6 as a p2p overlay JXTA (www.jxta.org) .NET (www.microsoft.com/net) Collaboration Groove (www.groove.net) Endeavors (www.endeavors.com) IPv6 as a p2p overlay Working Groups p2p.internet2.edu www.openp2p.com

Socket Programming in C Appendix Socket Programming in C

Socket programming with TCP Client must contact server server process must first be running server must have created socket (door) that welcomes client’s contact Client contacts server by: creating client-local TCP socket specifying IP address, port number of server process When client creates socket: client TCP establishes connection to server TCP When contacted by client, server TCP creates new socket for server process to communicate with client allows server to talk with multiple clients source port numbers used to distinguish clients (more in Chap 3) TCP provides reliable, in-order transfer of bytes (“pipe”) between client and server application viewpoint

Example: C client (TCP) /* client.c */ void main(int argc, char *argv[]) { struct sockaddr_in sad; /* structure to hold an IP address */ int clientSocket; /* socket descriptor */ struct hostent *ptrh; /* pointer to a host table entry */ char Sentence[128]; char modifiedSentence[128]; host = argv[1]; port = atoi(argv[2]); clientSocket = socket(PF_INET, SOCK_STREAM, 0); memset((char *)&sad,0,sizeof(sad)); /* clear sockaddr structure */ sad.sin_family = AF_INET; /* set family to Internet */ sad.sin_port = htons((u_short)port); ptrh = gethostbyname(host); /* Convert host name to IP address */ memcpy(&sad.sin_addr, ptrh->h_addr, ptrh->h_length); connect(clientSocket, (struct sockaddr *)&sad, sizeof(sad)); Create client socket, connect to server

Example: C client (TCP), cont. Get input stream from user gets(Sentence); n=write(clientSocket, Sentence, strlen(Sentence)+1); n=read(clientSocket, modifiedSentence, sizeof(modifiedSentence)); printf("FROM SERVER: %s\n”,modifiedSentence); close(clientSocket); } Send line to server Read line from server Close connection

Example: C server (TCP) /* server.c */ void main(int argc, char *argv[]) { struct sockaddr_in sad; /* structure to hold an IP address */ struct sockaddr_in cad; int welcomeSocket, connectionSocket; /* socket descriptor */ struct hostent *ptrh; /* pointer to a host table entry */ char clientSentence[128]; char capitalizedSentence[128]; port = atoi(argv[1]); welcomeSocket = socket(PF_INET, SOCK_STREAM, 0); memset((char *)&sad,0,sizeof(sad)); /* clear sockaddr structure */ sad.sin_family = AF_INET; /* set family to Internet */ sad.sin_addr.s_addr = INADDR_ANY; /* set the local IP address */ sad.sin_port = htons((u_short)port);/* set the port number */ bind(welcomeSocket, (struct sockaddr *)&sad, sizeof(sad)); Create welcoming socket at port & Bind a local address

Example: C server (TCP), cont /* Specify the maximum number of clients that can be queued */ listen(welcomeSocket, 10) while(1) { connectionSocket=accept(welcomeSocket, (struct sockaddr *)&cad, &alen); n=read(connectionSocket, clientSentence, sizeof(clientSentence)); /* capitalize Sentence and store the result in capitalizedSentence*/ n=write(connectionSocket, capitalizedSentence, strlen(capitalizedSentence)+1); close(connectionSocket); } Wait, on welcoming socket for contact by a client Write out the result to socket End of while loop, loop back and wait for another client connection

Socket programming with UDP UDP: no “connection” between client and server no handshaking sender explicitly attaches IP address and port of destination to each packet server must extract IP address, port of sender from received packet UDP: transmitted data may be received out of order, or lost application viewpoint UDP provides unreliable transfer of groups of bytes (“datagrams”) between client and server

Client/server socket interaction: UDP Server (running on hostid) create socket, clientSocket = DatagramSocket() Client Create, address (hostid, port=x, send datagram request using clientSocket create socket, port=x, for incoming request: serverSocket = DatagramSocket() read request from serverSocket close clientSocket read reply from clientSocket write reply to serverSocket specifying client host address, port number

Example: C client (UDP) /* client.c */ void main(int argc, char *argv[]) { struct sockaddr_in sad; /* structure to hold an IP address */ int clientSocket; /* socket descriptor */ struct hostent *ptrh; /* pointer to a host table entry */ char Sentence[128]; char modifiedSentence[128]; host = argv[1]; port = atoi(argv[2]); clientSocket = socket(PF_INET, SOCK_DGRAM, 0); /* determine the server's address */ memset((char *)&sad,0,sizeof(sad)); /* clear sockaddr structure */ sad.sin_family = AF_INET; /* set family to Internet */ sad.sin_port = htons((u_short)port); ptrh = gethostbyname(host); /* Convert host name to IP address */ memcpy(&sad.sin_addr, ptrh->h_addr, ptrh->h_length); Create client socket, NO connection to server

Example: C client (UDP), cont. Get input stream from user gets(Sentence); addr_len =sizeof(struct sockaddr); n=sendto(clientSocket, Sentence, strlen(Sentence)+1, (struct sockaddr *) &sad, addr_len); n=recvfrom(clientSocket, modifiedSentence, sizeof(modifiedSentence). (struct sockaddr *) &sad, &addr_len); printf("FROM SERVER: %s\n”,modifiedSentence); close(clientSocket); } Send line to server Read line from server Close connection

Example: C server (UDP) /* server.c */ void main(int argc, char *argv[]) { struct sockaddr_in sad; /* structure to hold an IP address */ struct sockaddr_in cad; int serverSocket; /* socket descriptor */ struct hostent *ptrh; /* pointer to a host table entry */ char clientSentence[128]; char capitalizedSentence[128]; port = atoi(argv[1]); serverSocket = socket(PF_INET, SOCK_DGRAM, 0); memset((char *)&sad,0,sizeof(sad)); /* clear sockaddr structure */ sad.sin_family = AF_INET; /* set family to Internet */ sad.sin_addr.s_addr = INADDR_ANY; /* set the local IP address */ sad.sin_port = htons((u_short)port);/* set the port number */ bind(serverSocket, (struct sockaddr *)&sad, sizeof(sad)); Create welcoming socket at port & Bind a local address

Example: C server (UDP), cont while(1) { n=recvfrom(serverSocket, clientSentence, sizeof(clientSentence), 0 (struct sockaddr *) &cad, &addr_len ); /* capitalize Sentence and store the result in capitalizedSentence*/ n=sendto(connectionSocket, capitalizedSentence, strlen(capitalizedSentence)+1,0 (struct sockaddr *) &cad, &addr_len); close(connectionSocket); } Receive messages from clients Write out the result to socket End of while loop, loop back and wait for another client connection