Chapter 2 The Application Layer. Goals of this Chapter To understand common application protocols work –Web (http) – (smtp) –FTP –DNS –P2P –DHT (distributed.

Chapter 2 The Application Layer

Goals of this Chapter To understand common application protocols work –Web (http) –Email (smtp) –FTP –DNS –P2P –DHT (distributed hash table) To understand how the design alternatives for application layer networking protocols –A network application runs on many hosts, it is a distributed application –This chapter discusses several designs of distributed applications

Road Map Application networking basics Web Email FTP DNS P2P DHP

Road Map Application networking basics Web Email FTP DNS P2P DHT

Creating a network app write programs that –run on (different) end systems –communicate over network –e.g., web server software communicates with browser software No need to write software for network-core devices –Network-core devices do not run user applications –applications on end systems allows for rapid app development, propagation application transport network data link physical application transport network data link physical application transport network data link physical

An app-layer networking protocol defines Types of messages exchanged, –e.g., request, response Message syntax: –what fields in messages & how fields are delineated Message semantics –meaning of information in fields Rules for when and how processes send & respond to messages Public-domain protocols: defined in RFCs allows for interoperability e.g., HTTP, SMTP Proprietary protocols: e.g., Skype

Which application gets a newly arriving packet? Web server SSH server SSH client Web browser Skype IM Operating System Transport Network Link Layer Physical Layer Applications Dest IP: 74.125.115.99 IP: 74.125.115.99

IP: 128.174.13.63 Transport layer multiplexing: TCP TCP port 80 Web server app socket Operating System Network Link layer Operating System Web browser app TCP port 23421 Network Link layer I would like to communicate with 74.125.115.99 port 80 I would like to accept communication on port 80 Dest IP: 74.125.115.99 Source IP: 128.174.13.63 Dest port: 80 Source port: 23421 IP: 74.125.115.99 An application is identified by the hosts IP addresses, transport protocol, and ports A TCP connection is identified by the pair of IPs, the pair of ports, and the transport protocol

IP: 128.174.13.63 Transport layer multiplexing: TCP TCP port 80 Web server app socket Operating System Network Link layer Operating System Web browser app TCP port 23421 Network Link layer I would like to communicate with 74.125.115.99 port 80 IP: 74.125.115.99 An application is identified by the hosts IP addresses, transport protocol, and ports A TCP connection is identified by the pair of IPs, the pair of ports, and the transport protocol Dest IP: 128.174.13.63 Source IP: 74.125.115.99 Dest port: 23421 Source port:80 socket

IP: 128.174.13.63 Transport layer multiplexing: TCP TCP port 80 Web server app socket Operating System Network Link layer Operating System Web browser app TCP port 23421 Network Link layer IP: 74.125.115.99 socket An application is identified by the hosts IP addresses, transport protocol, and ports A TCP connection is identified by the pair of IPs, the pair of ports, and the transport protocol socket Dest IP: 74.125.115.99 Source IP: 128.174.13.63 Dest port: 80 Source port: 23421

IP: 128.174.13.63 Transport layer multiplexing: TCP TCP port 80 Web server app socket Operating System Network Link layer socket Operating System Web browser app TCP port 23421 Network Link layer I would like to communicate with 74.125.115.99 port 80 I would like to accept communication on port 80 Dest IP: 74.125.115.99 Source IP: 128.174.13.63 Dest port: 80 Source port: 23421 IP: 74.125.115.99 socket An application is identified by the hosts IP addresses, transport protocol, and ports A TCP connection is identified by the pair of IPs, the pair of ports, and the transport protocol

IP: 128.174.13.63 Transport layer multiplexing: TCP TCP port 80 Web server app socket Operating System Network Link layer Web browser app TCP port 23421 socket Network Link layer Operating System I would like to send data: XXSFGFEWRV Dest IP: 74.125.115.99 Source IP: 128.174.13.63 Protocol: TCP Dest port: 80 Source port: 23421 Data: XXSFGFEWRV Dest IP: 74.125.115.99 Source IP: 128.174.13.63 Protocol: TCP Dest port: 80 Source port: 23421 Data: XXSFGFEWRV Dest IP: 74.125.115.99 Source IP: 128.174.13.63 Protocol: TCP Dest port: 80 Source port: 23421 Data: XXSFGFEWRV Network TCP port 80 Dest IP: 74.125.115.99 Source IP: 128.174.13.63 Protocol: TCP Dest port: 80 Source port: 23421 Data: XXSFGFEWRV IP: 74.125.115.99 socket An application is identified by the hosts IP address, transport protocols, and port A TCP connection is identified by the pair of IPs, the pair of ports, and the transport protocol

IP: 128.174.13.63 Transport layer multiplexing: UDP IM server app socket Operating System Network Link layer socket Operating System IM client app UDP port 23421 Network Link layer I would like to send/receive data over UDP port 23421 I would like to receive any data on UDP port 1401 IP: 74.125.115.99 UDP port 1401 An application is identified by the hosts IP address, transport protocols, and port

IP: 128.174.13.63 Transport layer multiplexing: UDP IM Server app socket Operating System Network Link layer socket Operating System IM client app UDP port 23421 Network Link layer Send data to 74.125.115.99 port 1401 Data: xxadre Dest IP: 74.125.115.99 Source IP: 128.174.13.63 Protocol: UDP Dest port: 1401 Source port: 23421 Data: xxadre IP: 74.125.115.99 An application is identified by the hosts IP address, transport protocols, and port UDP port 1401 Data: xxadre

Transport layer multiplexing TCP Applications accept new connections based on the destination port –An client app that would like to communicate a server app must know the IP of the host that is running the server app and the port on which the server app is listening When a connection is created, a socket is made. –Data is sent and received over this socket –This socket is identified by two pairs of IP-port, and the transport layer protocol, i.e., the tuple (IP, port, IP, port, transport layer) UDP The application is identified by the port on which the application is listening. The application can use the source IP and port to further multiplex

Project 1 – Send a Message via TCP and UDP UDP –Client Set up socket Send message Wait for reply If no reply comes, give up If reply comes, print it –Server Set up socket Wait for message When message arrives –Print message –Send reply TCP –Client Set up socket Send message Wait for reply If no reply comes, give up If reply comes, print it –Server Set up socket Wait for connection When connect arrives, get socket for connection Wait for message over connection socket When message arrives –Print message –Send reply Make client program to send message to server and then wait for message from server Make server program to wait for message from client and then respond with a message

Steps in Visual Studio Open visual studio File -> new ->project –Win32 console application Select name and directory –Leave defaults Console app Uncheck empty project Checked precompiled header –Paste code from web page into source code Build->build solution –The output window is in the lower middle frame –Might need to adjust the frames to see the output window –There will be many warnings, but, hopefully, no errors –the bottom of the output gives the directory where the program is located Open two command windows Change to the directory where the program is located Run the program in each command windows –Testudp 1 testudp 0 Start the server first (the one with 1 as an argument

What transport service does an app need? Data reliability some apps (e.g., audio) can tolerate some loss other apps (e.g., file transfer, telnet) require 100% reliable data transfer Timing some apps (e.g., Internet telephony, interactive games) require low delay to be “effective” Throughput some apps (e.g., multimedia) require minimum amount of throughput to be “useful” (i.e., in order for the user to gain utility) other apps (“elastic apps”) make use of whatever throughput they get Security Encryption, data integrity, …

Transport service requirements of common apps Application file transfer e-mail Web documents real-time audio/video stored audio/video interactive games instant messaging Data loss no loss loss-tolerant no loss Throughput elastic some what elastic audio: 5kbps-1Mbps video:10kbps-5Mbps same as above few kbps up elastic Time Sensitive no not really yes, 100’s msec yes, few secs yes, 100’s msec yes and no

Internet transport protocols services TCP service: connection-oriented: setup required between client and server processes reliable transport between sending and receiving process does not provide: timing, minimum throughput guarantees, or even when packets are transmitted flow control: sender won’t overwhelm receiver congestion control: throttle sender when network overloaded UDP service: No connection set-up needed unreliable data transfer between sending and receiving process Packets can be sent at any rate/time desired (but this might be cause considerable congestion) does not provide: flow control, congestion control, timing, throughput guarantee, or security

Internet apps: application, transport protocols Application e-mail remote terminal access Web file transfer streaming multimedia Internet telephony Application layer protocol SMTP [RFC 2821] Telnet [RFC 854] HTTP [RFC 2616] FTP [RFC 959] HTTP (eg Youtube), RTP [RFC 1889] SIP, RTP, proprietary (e.g., Skype) Underlying transport protocol TCP UDP typically UDP

Road Map Application basics Web Email FTP DNS P2P DHT

Web and HTTP Web page consists of objects Object can be HTML file, JPEG image, Java applet, audio file,… Web page consists of base HTML-file which includes several referenced objects The browser first requests the base file The base file specifies text and URLs of objects The browser requests these objects, where ever they are (not always on the same server) HTTP is used to request the base file and all the other files Note, that HTTP can be used for other applications besides web Each object is addressable by a URL Example URL: www.someschool.edu/someDept/pic.gif host name path name

HTTP overview HTTP: hypertext transfer protocol Web’s application layer protocol client/server model –client: browser that requests, receives, “displays” Web objects –server: Web server sends objects in response to requests PC running Explorer Server running Apache Web server Mac running Navigator HTTP request HTTP response

HTTP overview (continued) Uses TCP: client initiates TCP connection (creates socket) to server, port 80 server accepts TCP connection from client HTTP messages (application- layer protocol messages) exchanged between browser (HTTP client) and Web server (HTTP server) TCP connection closed HTTP is “stateless” server maintains no information about past client requests Protocols that maintain “state” are complex! past history (state) must be maintained if server/client crashes, their views of “state” may be inconsistent, must be reconciled aside

HTTP connections Nonpersistent HTTP At most one object is sent over a TCP connection. Persistent HTTP Multiple objects can be sent over single TCP connection between client and server.

Nonpersistent HTTP Suppose user enters URL www.someSchool.edu/someDepartment/home.index 1a. HTTP client initiates TCP connection to HTTP server (process) at www.someSchool.edu on port 80 2. HTTP client sends HTTP request message (containing URL) into TCP connection socket. Message indicates that client wants object someDepartment/home.index 1b. HTTP server at host www.someSchool.edu waiting for TCP connection at port 80. “accepts” connection, notifying client 3. HTTP server receives request message, forms response message containing requested object, and sends message into its socket time (contains text, references to 10 jpeg images) 5. HTTP client receives response message containing html file, displays html. Parsing html file, finds 10 referenced jpeg objects 6. Steps 1-5 repeated for each of 10 jpeg objects 4. HTTP server closes TCP connection.

file received time to transmit file initiate TCP connection RTT request file RTT time Non-Persistent HTTP: Response time Definition of RTT: time for a small packet to travel from client to server and back. Response time: one RTT to initiate TCP connection one RTT for HTTP request and first few bytes of HTTP response to return file transmission time total = 2RTT + data transmit time 10 objects require 20RTT+10 data transmit times

Persistent HTTP Suppose user enters URL www.someSchool.edu/someDepartment/home.index 1a. HTTP client initiates TCP connection to HTTP server (process) at www.someSchool.edu on port 80 2. HTTP client sends HTTP request message (containing URL) into TCP connection socket. Message indicates that client wants object someDepartment/home.index 1b. HTTP server at host www.someSchool.edu waiting for TCP connection at port 80. “accepts” connection, notifying client 3. HTTP server receives request message, forms response message containing requested object, and sends message into its socket time (contains text, references to 10 jpeg images) 5. HTTP client receives response message containing html file, displays html. Parsing html file, finds 10 referenced jpeg objects, requests these 10 references 4. HTTP server sends each of the 10 objects.

base received time to transmit file initiate TCP connection RTT request file RTT (Non) Persistent HTTP: Response time time to transmit file RTT request file RTT initiate TCP connection 10 objects require 20RTT+10 data transmit times Non-persistent base received time to transmit file initiate TCP connection RTT request file RTT time to transmit file RTT Request files 10 objects require 3RTT+10 data transmit times Persistent Object 1 received Object 2 received

(non) Persistent HTTP Advantages of persistent HTTP are only valid if the objects are the same server. –Usually some objects are on the server, but many are not Instead of using a single persistent HTTP connection, a browser could use many nonpersistent connections in parallel –This is a bit unfair. As far as I know, persistent HTTP is usually not supported by the server or browser base received time to transmit file initiate TCP connection RTT request file RTT time to transmit file RTT request 5 files RTT Initiate 5 TCP connection Non-persistent Object 1 received Object 2 received Object 3 received Object 4 received Object 5 received

HTTP request message two types of HTTP messages: request, response HTTP request message: –ASCII (human-readable format) GET /somedir/page.html HTTP/1.1 Host: www.someschool.edu User-agent: Mozilla/4.0 Connection: close Accept-language:fr (extra carriage return, line feed) request line (GET, POST, HEAD commands) header lines Carriage return, line feed indicates end of message

HTTP request message: general format

HTTP response message HTTP/1.1 200 OK Connection close Date: Thu, 06 Aug 1998 12:00:15 GMT Server: Apache/1.3.0 (Unix) Last-Modified: Mon, 22 Jun 1998 …... Content-Length: 6821 Content-Type: text/html data data data data data... status line (protocol status code status phrase) header lines data, e.g., requested HTML file

HTTP response status codes 200 OK –request succeeded, requested object later in this message 301 Moved Permanently –requested object moved, new location specified later in this message (Location:) 400 Bad Request –request message not understood by server 404 Not Found –requested document not found on this server 505 HTTP Version Not Supported In first line in server->client response message. A few sample codes:

Trying out HTTP (client side) for yourself 1. Telnet to your favorite Web server: Opens TCP connection to port 80 (default HTTP server port) at www.eecis.udel.edu. Anything typed in sent to port 80 at www.eecis.udel.edu telnet www.eecis.udel.edu 80 2. Type in a GET HTTP request: GET / HTTP/1.1 Host: www.eecis.udel.edu By typing this in (hit carriage return twice), you send this minimal (but complete) GET request to HTTP server 3. Look at response message sent by HTTP server!

Wireshark (ethereal) Wireshark captures all packets that pass through the hosts interface To run Wireshark, libpcap (linux) or winpcap (windows) must be installed. It comes with wireshark package Then, run wireshark Select Capture Find the active interface –E.g., not generic dialup, nor vnp, nor packet scheduler, but wireless …. With IP address –Then select prepare –Let’s watch TCP packets on port 80 Next to capture filter, enter TCP port 80 –Select update in realtime and autoscroll –Might need to enable or disable “capture in promiscuous mode” –Press start –Press close Load www.eecis.udel.edu page in browser Press stop in Wireshark Find http request to 128.4.40.10. –Right click and select follow TCP stream

User-server state: cookies Many major Web sites use cookies Four components: 1) cookie header line of HTTP response message 2) cookie header line in HTTP request message 3) cookie file kept on user’s host, managed by user’s browser 4) back-end database at Web site Example: Susan always access Internet always from PC visits specific e- commerce site for first time when initial HTTP requests arrives at site, site creates: –unique ID –entry in backend database for ID

Cookies: keeping “state” (cont.) client server usual http response msg cookie file one week later: usual http request msg cookie: 1678 cookie- specific action access ebay 8734 usual http request msg Amazon server creates ID 1678 for user create entry usual http response Set-cookie: 1678 ebay 8734 amazon 1678 usual http request msg cookie: 1678 cookie- spectific action access ebay 8734 amazon 1678 backend database

Cookies (continued) What cookies can bring: authorization shopping carts recommendations user session state (Web e-mail) Cookies and privacy: rcookies permit sites to learn a lot about you ryou may supply name and e- mail to sites aside How to keep “state”: rprotocol endpoints: maintain state at sender/receiver over multiple transactions rcookies: http messages carry state

Web Serving Systems LAMP Stack – very popular –Linux (OS) –Apache (web sever) Receives http request and generates http response The generation of response can involve many steps Other servers are also popular: –nginx – open source, reverse proxy, load balancer, popularity: apache, microsoft, nginx, google –lighttpd – open source, small and fast. Good for high load. E.g., youtube, meeboo. Can handle 1000 hits per sec –MySQL or MariaDB (mySQL is oracle. MariaDB is a branch of mySQL, but not under oracle’s control) Open source database Very popular PostgreSQL is also very popular New, noSQL (not only SQL), databases are also playing a role –Casandra –mongoDB –Php Php scripts make the html that is delivered by the web server to the client Other application –Perl –Python –Java (with a tomcat server) Facebook converts php to C and save 30% in speed (which means 30% in servers, which is a huge amount of money)

Simple LAMP Topology

Faster TopologyApache+PHP+APC MySQL Squid memcached Web cache: holds recently requested pages and generate response if the desired page is in cache. Otherwise, the request is forward to the web server Cache for sql queries. A giant hash table. Give it a string (key) and if the response is in cache, it gets it. Otherwise, the request is sent to database and the result is also saved the response in cache The key can be anything. The programmer decides. APC is a cache for compiled php code

MySQL Replication MySQL master slaveslave All writes are to the master Reads are from the master or the slaves There is a slight delay from when the master is updated and the update is reflected by all slaves Where’d my edit go???

Data Sharding MySQL MySQL group s1 English-language Wikipedia Next 19 biggest wikis MySQL group s2 MySQL group s3 Next 764 wikis

Load Balancer Apache tomcat 1/3 of HTTP request to each server Might keep request with the same cookie to the same server (sticky sessions) Might decrypt SSL Also load balances Load balancers also check if machines are healthy and will stop sending requests if they seem unhealthy

Content distribution networks Even with a very fast server architecture, RTT is still large to some users Locations of Amazon’s “cloudFront” servers CDNs allow you to put parts of your web page (e.g., logo, javascripts, audio, video, multicast live video) on servers that are close to the client Act as web proxy. See next slides If the content is always unique and changing (facebook), then the design of the CDN is more complicated

CDN example: AWS CloudFront cloudFront machines are scattered around the world and are close to most people –Close in terms of RTT Documents and media can be cached in a cloud –If the document is not in the CloudFront machine, the request is forwarded to the permanent storage/original server and given to the user and saved in cache E.g., –Create a distribution for the image in cloudfront, and get a new url, e.g., http://mydomain.cloudfront.nethttp://mydomain.cloudfront.net –If image is at http://mydomain.com/images/pic1.jpg –Your web pages should refer to http://mydomain.cloudfront.net/images/pic1.jpg not to http://mydomain.com/images/pic1.jpghttp://mydomain.cloudfront.net http://mydomain.com/images/pic1.jpg A request to mydomain.cloudfront.com should go to the nearest cloudfront server farm. –How? If the cache document changes, then different cloudfront machines will have different versions. Eventually they will be updated, usually within a few minutes, depending on the timeout value set

Web caches (proxy server) user sets browser: Web accesses via cache browser sends all HTTP requests to cache –object in cache: cache returns object –else cache requests object from origin server, then returns object to client Goal: reduce network utilization by satisfying client request without involving original server client Proxy server client HTTP request HTTP response HTTP request origin server origin server HTTP response

More about Web caching cache acts as both client and server typically cache is installed by ISP (university, company, residential ISP, e.g., satellite-based ISP) or as part of a CDN Why Web caching? reduce response time for client request reduce traffic on an institution’s access link. Internet dense with caches: enables “poor” content providers to effectively deliver content (similar objective as P2P file sharing)

Caching example Assumptions average object size = 100,000 bits avg. request rate from institution’s browsers to origin servers = 15/sec delay from institutional router to any origin server and back to router = 2 sec Consequences utilization on LAN = 15% utilization on access link = 100% total delay = Internet delay + access delay + LAN delay = 2 sec + minutes + milliseconds origin servers public Internet institutional network 10 Mbps LAN 1.5 Mbps access link institutional cache

Caching example (cont) possible solution increase bandwidth of access link to, say, 10 Mbps consequence utilization on LAN = 15% utilization on access link = 15% Total delay = Internet delay + access delay + LAN delay = 2 sec + msecs + msecs often a costly upgrade origin servers public Internet institutional network 10 Mbps LAN 10 Mbps access link institutional cache

Caching example (cont) possible solution: install cache suppose hit rate is 0.4 consequence 40% requests will be satisfied almost immediately 60% requests satisfied by origin server utilization of access link reduced to 60%, resulting in negligible delays (say 10 msec) total avg delay = Internet delay + access delay + LAN delay =.6*(2.01) secs +.4*milliseconds < 1.4 secs origin servers public Internet institutional network 10 Mbps LAN 1.5 Mbps access link institutional cache

Conditional GET Goal: don’t send object if cache has up-to-date cached version cache: specify date of cached copy in HTTP request If-modified-since: server: response contains no object if cached copy is up-to- date: HTTP/1.0 304 Not Modified cache server HTTP request msg If-modified-since: HTTP response HTTP/1.0 304 Not Modified object not modified HTTP request msg If-modified-since: HTTP response HTTP/1.0 200 OK object modified

Road Map Application basics Web FTP Email DNS P2P DHT

FTP: the file transfer protocol transfer file to/from remote host client/server model –client: side that initiates transfer (either to/from remote) –server: remote host ftp: RFC 959 ftp server: listens on port 21 file transfer FTP server FTP user interface FTP client local file system remote file system user at host

FTP is weird: separate control and data connections FTP client contacts FTP server at port 21, TCP is transport protocol client authorized over control connection –This is done in “clear text” (i.e., unencrypted) –So if some one if sniffing packets, your password might be learned. –Sniffing packets is difficult on ethernet, encrypted wifi, and DSL, but is possible on cable modems, and unencrypted wifi client browses remote directory by sending commands over control connection. Data is transferred over different connections. Two approaches –Active –Passive FTP client FTP server TCP control connection port 21 TCP data connection port 20 Active –The client opens a TCP socket with on some port (port number >1024) –The client sends the server the port –The server connects to the client’s port where the servers source port is 20 Active mode is a problem for firewalls –If my desktop is not a server, it should not receive any requests for connections. –But FTP servers will make such a requests

FTP Passive mode When a file is to be transferred, the server opens a port (number>1024 and not 20) The server sends this port number information over the command connection The client connects to the servers over this port. FTP client FTP server TCP control connection port 21 TCP data connection high port Drawback of passive –Some enterprises (companies) like to control which applications are used E.g., web browsing is ok, but skype is not –One way to do this is to block out going connections based on the port. –However, this will cause FTP to fail, unless the device that blocks connections is smart

Email Protocol Design Basic assumption: weak user agents and strong mail servers –The user wants to send the mail and leave –The user wants to get the mail –The user may come and go whenever (e.g., roaming laptop) –It should be possible to send mail between users even if neither user is online at the same time. –We conclude that there must be a middle man/mail server. Servers are not that strong: The protocol must be as robust as possible to servers being offline –No single server – why Single point of failure The server would have to be too big (congestion) –We conclude that there should be many mail servers Two types of hosts –Users –Mail servers Each user has a mail box in its mail server –Users retrieve mail from their mail server at their convenience Users give mail to their mail servers to deliver the mail Mail servers communicate with –The users that have mail boxes in the server –Other mail servers user agent mail server mail server user agent

Email Protocol Design Two types of hosts –Users –Mail servers Each user has a mail box in its mail server –Users retrieve mail from their mail server at there convenience Users give mail to their mail servers to deliver the mail Mail servers communicate with –The users that have mail boxes in the server –Other mail servers user agent mail server mail server user agent User composes mail and sends it to its mail server (or a mail server that will send mail for it) Mail server finds the destination mail server and attempts to send the mail Destination user requests emails from mailbox Destination server gives mails to user

Email Protocol Design Two types of hosts –Users –Mail servers Each user has a mail box in its mail server –Users retrieve mail from their mail server at there convenience Users give mail to their mail servers to deliver the mail Mail servers communicate with –The users that have mail boxes in the server –Other mail servers user agent mail server mail server user agent User composes mail and sends it to its mail server (or a mail server that will send mail for it) Mail server finds the destination mail server and attempts to send the mail Destination user requests emails from mailbox Destination server gives mails to user SMTP POP3 IMAP …

Electronic Mail: Details Three major components: user agents mail servers simple mail transfer protocol: SMTP User Agent a.k.a. “mail reader” composing, editing, reading mail messages e.g., Eudora, Outlook, elm, Mozilla Thunderbird Put outgoing on server (with SMTP) Get incoming messages from server user mailbox outgoing message queue mail server user agent user agent user agent mail server user agent user agent mail server user agent SMTP

Electronic Mail: mail servers Mail Servers mailbox contains incoming messages for user message queue of outgoing (to be sent) mail messages SMTP protocol between mail servers to send email messages –client: sending mail server –“server”: receiving mail server Reliable: several attempts and provide notification if delivery fails mail server user agent user agent user agent mail server user agent user agent mail server user agent SMTP

Electronic Mail: SMTP [RFC 2821] uses TCP to reliably transfer email message from client to server, port 25 direct transfer: sending server to receiving server Emails are pushed to servers (but users pull messages from servers) three phases of transfer –handshaking (greeting) –transfer of messages –closure command/response interaction –commands: ASCII text –response: status code and phrase messages must be in 7-bit ASCII –Makes it difficult to send attachments

Scenario: Alice sends message to Bob 1) Alice uses UA to compose message and “to” bob@someschool.edu 2) Alice’s UA sends message to her mail server; message placed in message queue 3) Client side of SMTP opens TCP connection with Bob’s mail server 4) SMTP client sends Alice’s message over the TCP connection 5) Bob’s mail server places the message in Bob’s mailbox 6) Bob invokes his user agent to read message user agent mail server mail server user agent 1 2 3 4 5 6

Sample SMTP interaction S: 220 hamburger.edu C: HELO crepes.fr S: 250 Hello crepes.fr, pleased to meet you C: MAIL FROM: S: 250 alice@crepes.fr... Sender ok C: RCPT TO: S: 250 bob@hamburger.edu... Recipient ok C: DATA S: 354 Enter mail, end with "." on a line by itself C: Do you like ketchup? C: How about pickles? C:. S: 250 Message accepted for delivery C: QUIT S: 221 hamburger.edu closing connection Client connects to server

Try SMTP interaction for yourself: telnet mail.eecis.udel.edu 25 see 220 reply from server enter HELO, MAIL FROM, RCPT TO, DATA, QUIT commands above lets you send email without using email client (reader)

SMTP: final words SMTP uses persistent connections SMTP requires message (header & body) to be in 7-bit ASCII SMTP server uses CRLF.CRLF to determine end of message Comparison with HTTP: HTTP: pull SMTP: push both have ASCII command/response interaction, status codes HTTP: each object encapsulated in its own response msg SMTP: multiple objects sent in multipart msg

Mail access POP3 and IMAP are two protocols for access mail on a mail server Web-based mail works differently, the web mail server and the mail server can be integrated, so that there is no user agent.

Mail access protocols SMTP: delivery/storage to receiver’s server Mail access protocol: retrieval from server –POP: Post Office Protocol [RFC 1939] authorization (agent server) and download –IMAP: Internet Mail Access Protocol [RFC 1730] more features (more complex) manipulation of stored msgs on server –HTTP: gmail, Hotmail, Yahoo! Mail, etc. user agent sender’s mail server user agent SMTP access protocol receiver’s mail server

DNS – domain name system Change names, like www.yahoo.com into IP address.www.yahoo.com Services provided by DNS –Name to address translation –Host aliasing A host relay1.west-coast.yahoo.com could have two aliases, yahoo.com and www.yahoo.com. In this case, the canonical hostname is relay1.west-coast.yahoo.com. DNS can provide canonical host names –Mail server aliasing When a mail server wants to send a mail to Me@udel.edu, it does not send it to www.udel.edu, but to mail.udel.edu. Or maybe udmail.udel.edu. DNS can translate udel.edu to mail.udel.eduMe@udel.edu –(Cheap) Load distribution Cnn.com has several servers. DNS will respond with all address, but it will reorder the addresses every time. If the client uses the first address listed, then each client will use different servers. Content distribution networks (CDN) are better ways of load balancing

DNS - structure Centralized DNS? –Pros – somewhat easy to maintain (there is only one system). But it must always be online –Cons Single point of failure (the system crashes -> no web) Congestion Server would be far from some hosts (delay) Database would be too big The register bohacek-pc1.pc.udel.edu would require interacting with the big server Instead, a distributed hierarchical database is used.

Domain Hierarchy educomgovmilorgnetukin UDelupennyahoociscowhitehousenasanavyarpaacm eecis art bohacek_pc1 bohacek_pc10

Administrative Zones in the Domain Hierarchy edu comgov milorgnetukin UDupenn yahoociscowhitehousenasa navyarpaacm eecis art bohacek_pc1bohacek_pc10 root It is possible that.edu and.gov are administered together Note that UD administers art but not eecis Some times a single service provider will administer the domains for a large number of.coms

Root servers Each layer in the hierarchy knows about the domain names below it The highest level is the root. –There are 13 root “servers” –Each of these servers is actually several servers, and some of the machines that comprise a server are distributed geographically. 13 root name servers worldwide b USC-ISI Marina del Rey, CA l ICANN Los Angeles, CA e NASA Mt View, CA f Internet Software C. Palo Alto, CA (and 36 other locations) i Autonomica, Stockholm (plus 28 other locations) k RIPE London (also 16 other locations) m WIDE Tokyo (also Seoul, Paris, SF) a Verisign, Dulles, VA c Cogent, Herndon, VA (also LA) d U Maryland College Park, MD g US DoD Vienna, VA h ARL Aberdeen, MD j Verisign, ( 21 locations)

Overview Top-level domain (TLD) servers –There are around 200 top-level domains –These include com, edu, mil, info, in, uk, cn, –Currently, network solutions maintains the TLD servers for com Educause maintains the TLD servers for edu –The root servers know the addresses and names of all top level servers Organizations have a hierarchy of DNS servers

DNS queries Suppose a host needs the IP address of bohacek-pc1.eecis.udel.edu If this IP address is not in cache, the host asks its local DNS server. If the DNS server does not have it in cache, it checks if is had the IP address of the DNS server of eecis.udel.edu in cache If not, it checks if IP address of the dns server of udel.edu in cache If not, it check if it has the IP address of the top-level domain server of edu in cache It not, it asks the root server for the IP address of the edu TLD server –The DNS server always has the IP address of the root servers The local DNS server asks the edu TLD server for address of bohack- pc1.eecis.udel.edu. The TLD server does not know that IP address, but instead gives the IP address of the dns server for UD The local DNS server asks the UD dns server for the address of bohack- pc1.eecis.udel.edu. The UD dns server does not know the address, but instead returns the address of the eecis dns server. The local DNS server asks the eecis dns server for the address of bohacek- pc1.eecis.udel.edu Eecis dns server replies with the address. This address is returned to the host that orginally asked the question.

DNS Queries Browser wants to show www. eecis.udel.edu Browser needs the IP address of www. eecis.udel.edu Host asks local DNS server for IP address of www. eecis.udel.edu Local DNS server checks if it has the IP address of www.eecis.udel.edu in cache.www.eecis.udel.edu If not, it checks if is had the IP address of the DNS server of eecis.udel.edu in cache If not, it checks if IP address of the dns server of udel.edu in cache If not, it check if it has the IP address of the top-level domain server of edu in cache.if not, ….. What is the IP address of www.eecis.udel.edu? Root server (IP address are always known) Root server does not know. Instead, it responds with dns server that might, specifically, the TLD server for.edu What is the ip address of www.eecis.udel.edu? TLD server for.edu TLD server does not know. Instead replies with the name and IP address of the UD DNS server What is the ip address of www.eecis.udel.edu? UD dns server does not know. Instead it replies with the name and IP address of the eecis dns server. What is the ip address of www.eecis.udel.edu? It is 128.4.1.2

Browser wants to show www.eecis.udel.edu DNS Queries Browser needs the IP address of www.eecis.udel.edu Host asks local DNS server for IP address of www.eecis.udel.edu 1.Local DNS server checks if it has the IP address of www.eecis.udel.edu in cache.www.eecis.udel.edu 2.If not, it checks if is had the IP address of the DNS server of eecis.udel.edu in cache 3.If not, it checks if it has the IP address of the DNS server of udel.edu in cache 4.If not, it checks if it has the IP address of the top-level domain server of edu in cache 5..if not, ….. What is the IP address of www.eecis.udel.edu? Root server (IP addresses are always known) Root server does not know. Instead, it responds with name and address of a server that might, specifically, the TLD server for.edu What is the IP address of www.eecis.udel.edu? TLD server for.edu TLD server does not know. Instead replies with the name and IP address of the UDel DNS server What is the ip address of www.eecis.udel.edu? UDel DNS server does not know. Instead it replies with the name and IP address of the eecis dns server. What is the IP address of www.eecis.udel.edu? It is 128.4.1.2 UD DNS server eecis DNS server

Browser wants to show www.eecis.udel.edu DNS Queries Browser needs the IP address of www.eecis.udel.edu Host asks local DNS server for IP address of www.eecis.udel.edu 1.Local DNS server checks if it has the IP address of www.eecis.udel.edu in cache.www.eecis.udel.edu 2.If yes, then return it It is 128.4.1.2

Browser wants to show www.eecis.udel.edu DNS Queries Browser needs the IP address of www.eecis.udel.edu Host asks local DNS server for IP address of www.eecis.udel.edu 1.Local DNS server checks if it has the IP address of www.eecis.udel.edu in cache.www.eecis.udel.edu 2.If not, it checks if is had the IP address of the DNS server of eecis.udel.edu in cache 3.If yes, query it… What is the IP address of www.eecis.udel.edu? It is 128.4.1.2 eecis DNS server

Browser wants to show www.eecis.udel.edu DNS Queries Browser needs the IP address of www.eecis.udel.edu Host asks local DNS server for IP address of www.eecis.udel.edu 1.Local DNS server checks if it has the IP address of www.eecis.udel.edu in cache.www.eecis.udel.edu 2.If not, it checks if is had the IP address of the DNS server of eecis.udel.edu in cache 3.If not, it checks if it has the IP address of the DNS server of udel.edu in cache 4.If not, it checks if it has the IP address of the top-level domain server of edu in cache 5..if so, then query it… What is the IP address of www.eecis.udel.edu? TLD server for.edu TLD server does not know. Instead replies with the name and IP address of the UD DNS server What is the ip address of www.eecis.udel.edu? UD DNS server does not know. Instead it replies with the name and IP address of the eecis dns server. What is the IP address of www.eecis.udel.edu? It is 128.4.1.2 UD DNS server eecis DNS server

International domains (ccTLD) www.videos.cnn.com.cn A root server will provide the name and address of the.cn dns server. –However, this dns server could be able to answer com.cn or cnn.com.cn It is possible that.cn (china) has its own TLD and.com.cn is a subdomain However, usually, com.cn is the TLD –But it does not have to be this way We usually call cnn.com.cn the second level domain, even though it is strictly a third level domain For performance (i.e., reducing the number of DNS servers with which the local DNS must communicate), it is best if the dns server for.cn can answer cnn.com.cn International top-level domains can be identified by two letters, e.g.,.cn, and also in language-native script such as arabic or chinese characters

Attack on DNS Hackers have tried to bring down DNS by performing a DoS on the root servers –DoS – denial of service. Sends more packets or requests for service than the server can accommodate. Resulting in poor service for normal users. This failed because –There are many very strong root servers and have firewalls/filters The attacks used ICMP ping packets DNS requests would have been more effective –It is rare that a root server is needed Usually only the TLD server is needed Or only a domain server.

DNS Message Details DNS Record –(Name, Value, Type, Class, TTL) –If Type = A Name is the host name Value is the IP address of the host –If Type = NS Name is a domain name Value is the name of the DNS server for the domain E.g., (udel.edu, dns.udel.edu, NS, …, …) –Type = MX Name is the domain name Value is the name of the mail server for the domain E.g., (udel.edu, mail.udel.edu, MX, …, …) –Type = CName Name is a host name Value is the canonical name of the host E.g., (www.yahoo.com, relay-east.yahoo.com, CName, …, …)www.yahoo.com –TTL is the time to live, so DNS caches can be timed out –Class is no longer used, it is set as IN

DNS query (Name, Type, Class) (UDel.edu, MX, IN) –Please provide the name of the UD’s mail server (mail.UDel.edu, A, IN) –Please provide the IP address for mail.udel.edu

DNS message format DNS protocol : query and reply messages, both with same message format msg header identification: 16 bit # for query, reply to query uses same # flags: –query or reply –recursion desired –recursion available –reply is authoritative

DNS message format Name, type fields for a query RRs in response to query records for authoritative servers additional “helpful” info that may be used

Browser wants to show www.eecis.udel.edu Browser needs the IP address of www.eecis.udel.edu DNS Queries 1.Local DNS server checks if it has the IP address of www.eecis.udel.edu in cache.www.eecis.udel.edu 2.If not, it checks if is had the IP address of the DNS server of eecis.udel.edu in cache 3.If not, it checks if it has the IP address of the DNS server of udel.edu in cache 4.If not, it checks if it has the IP address of the top-level domain server of edu in cache 5..if not, ….. Root server (IP addresses are always known) TLD server for.edu UD DNS server eecis DNS server 1 0 0 0 (www.eecis.udel.edu, A,IN) 1 0 0 0 0 0 0 4 (edu, edu-serverA.net, NS, IN) (edu-serverA.net, 124.5.1.1, A, IN) (edu, edu-serverB.net, NS, IN) (edu-serverB.net, 124.5.1.2, A, IN) 1 0 0 0 (www.eecis.udel.edu, A,IN) 0 0 0 4 (udel.edu, dns2.udel.edu, NS, IN) (udel.edu, dns2.udel.edu, 128.178.2.2, A, IN) (udel.edu, dns1.udel.edu, NS, IN) (dns1.udel.edu, 128.173.2.1, A, IN) 1 0 0 0 (www.eecis.udel.edu, A,IN) 0 0 0 4 (eecis.udel.edu, dns1.eecis.udel.edu, NS, IN) (dns1.eecis.udel.edu, 128.4.1.10, A, IN) (eecis.udel.edu, dns2.udel.edu, NS, IN) (dns2.udel.edu, 128.4.1.11, A, IN) 1 0 0 0 (www.eecis.udel.edu, A,IN) 0 1 0 0 (www.eecis.udel.edu, 128.4.1.1, A, IN) 0 1 0 0

DNS Header and Flags The DNS header has a query ID –The query has this ID and the server copies this ID into the response Flag indicating query or answer Flag indicating whether the server is the authoritative server for the answer (as oppose to a cached answer) A recursive desired flag indicating that the host/server would like the server to perform the recursive DNS lookup A recursive available flag indicating whether the server is available to to the recursive lookup

Exploring DNS with dig Dig is installed on stimpy and perhaps other eecis linux machines A windows version of dig is also available E.g., –>> dig udel.edu –Returns information about the dns entry for udel E.g., –>>dig @dns.eecis.udel.edu udel.edu –Returns information about the dns entry for udel that is stored in dns.eecis.udel.edu E.g., –>> dig edu –Returns a list of TLD edu servers E.g., –>> dig –Returns list of root servers E.g., –>> dig udel.edu MX –Returns the mail server for udel.edu

DNS Which transport protocol should DNS use? Why?

Captive Portal Problem Use Udel Wifi with a new machine. Instead of google.com, I get some UD web page that requires log in I log in, and it says that I should reboot. I don’t reboot, but when I go to google.com, I still get UD login page. However, if I go to some other page (e..g, NYTimes.com, it works fine) What is going on? Why would reboot fix it? What else would fix the problem Answer: when a new machine connects to UD wifi, all dns request return the IP of UD login page. After logged in, DNS works correctly. However, my machine’s DNS cache will still point to UD login page, until I reboot.

DNS Attack client Local DNS server attacker BoA DNS server Request IP of www.BoA.com Request IP of www.BoA.com IP of www.BoA.com 1.1.1.1 IP of www.BoA.com 1.1.1.1 TCP+HTTP connection to 1.1.1.1 HTTP:: data: “welcome to BoA, please enter your acct# and password Client request IP address of BoA from local DNS server Local DNS server request address from BoA DNS server Before BoA’s server can respond, the attacker sends a response The DNS server sends this response to the client, The client then accesses the incorrect web page, which appears exactly the same as BoA’s web page Note, this attack has nothing to do with the security of your machine.

DNS Attack: really? client Local DNS server attacker BoA DNS server Request IP of www.BoA.com Request IP of www.BoA.com IP of www.BoA.com 1.1.1.1 IP of www.BoA.com 1.1.1.1 TCP+HTTP connection to 1.1.1.1 HTTP:: data: “welcome to BoA, please enter your acct# and password How does the attacker know the exact time to send the fake DNS reply? It does not, instead the attacker continuously sends replies But what information is contained in the DNS response that the attacker must guess correctly? Source IP address of BoA DNS server There are not that many The local DNS’s server’s source port The destination port is always 53, but sometimes the source port is also 53, or changes in a predictable way, e.g., increments The DNS query Id This is typically incremented The attacker, which is a DNS server as well, makes DNS request that cause the DNS server to request to itself. In this way, the attacker can determine various numbers

DNS Attack: really? client Local DNS server attacker BoA DNS server Request IP of www.BoA.com Request IP of www.BoA.com IP of www.BoA.com 1.1.1.1 IP of www.BoA.com 1.1.1.1 TCP+HTTP connection to 1.1.1.1 HTTP:: data: “welcome to BoA, please enter your acct# and password How does the attacker know the exact time to send the fake DNS reply? It does not, instead the attacker continuously sends replies But what information is contained in the DNS response that the attacker must guess correctly? Source IP address of BoA DNS server There are not that many The local DNS’s server’s source port The destination port is always 53, but sometimes the source port is also 53, or changes in a predictable way, e.g., increments The DNS query Id This is typically incremented By adding randomness to the source port and query Id, the DNS attack becomes very difficult. Still, it only needs to succeed once to gain information. Perhaps it will take months or years, but it might work

DNS poisoning A DNS response might include additional responses, e.g., for BoA, but with the incorrect IP. This additional response is cached E.g., send email stating that pictures of naked people can be found at www.evil.com But dns.evil.com includes additional records Solution: ignore this type of additional records DNSSec solves the known security problems with DNS

Good DNS “attack” OpenDNS keeps track of bad web sites and will return a different IP address when the web address for these bad web sites is requested

Peer-to-peer file sharing About P2P –30% or more of the bytes transferred on the Internet are from P2P users –Skype is a very successful P2P VoIP app Written in 3-4 months Topics covered –Scalability –P2P querying –BitTorrent

Pure P2P architecture Review: What is the difference between peer-to-peer and client/server? –Each hosts acts as both a server and a client. no always-on server arbitrary end systems directly communicate peers are intermittently connected and may change IP addresses Pure P2P has significant drawbacks. P2P-like systems with some central servers are more common. But in all cases, the file transfer is between peers, not from servers. peer-peer

File Distribution: Server-Client vs P2P Question : How much time to distribute file from one server to N peers? usus u2u2 d1d1 d2d2 u1u1 uNuN dNdN Server Network (with abundant bandwidth) File, size F u s : server upload bandwidth u i : peer i upload bandwidth d i : peer i download bandwidth

File distribution time: server-client usus u2u2 d1d1 d2d2 u1u1 uNuN dNdN Server Network (with abundant bandwidth) F Time for the server to send a copy to a single client –F/u s Time for the server send N copies: –NF/u s time client i takes F/d i time to download increases linearly in N (for large N) = d cs = max { NF/u s, F/min(d i ) } i Time to distribute F to N clients using client/server approach

File distribution time: P2P usus u2u2 d1d1 d2d2 u1u1 uNuN dNdN Server Network (with abundant bandwidth) F server must send one copy: –F/u s time client i download time –F/d i Total data to be downloaded –NF fastest possible transfer rate: u s +  u i d P2P = max { F/u s, F/min(d i ), NF/(u s +  u i ) } i Can you make a schedule for the download the take this amount?

P2P schedule Important: we model data flow as a continuous stream of data, not packets or even bytes Assume each host has the same upload data rate, u c Server delivers a chunk of size F/N to each host –The server sends chunks to each of the N host simultaneously –Duration = F/u s As data arrives from the server, each host delivers the chunk to all N-1 hosts –Client’s total upload rate is u c –So each client gets data at rate u c /(N-1) –Duration to send its chunk (of size F/N) to all of the other hosts = (F/N) / (u c /(N-1)) = F/ u c  N/(N-1) Duration = max(F/u s, F/u c  N/(N-1)) Server client u s /N usus u c /(N-1)

Scheme –Server send chunk of size z i to host i –Host i send this chunk to all other N-1 hosts The download to the host runs simultaneous with the upload to the other hosts Objective: each host completes its upload at the same time –This a difficult since each host has a different upload data rate Let T be the time for each host to send the chunk to all N-1 other hosts. And assume that T is shorter than the time it takes for the server to deliver the file to all hosts (the server is the bottleneck) Then T = (N-1) z i / u i Solve for z i we get, z i = u i T/(N-1) F = Sum z i F = sum u i T/(N-1) = T/(N-1)  sum u i T/(N-1) = F/sum(u i ) z i =F  u i /sum(u i ) Duration from server = F/ u s Check our assumption, is F/ u s > T ? –Is F/ u s > F  (N-1)/sum(u i ) –Is u s < sum(u i )/(N-1) –Is N u s – u s < sum(u i ) –Is N u s < u s + sum(u i ) –Our assumption is true if u s < (u s + sum(u i ))/N, or u s < sum(u i ))/(N-1)

Suppose that u s > sum(u i ))/(N-1), then the server could send more data than the previous scheme Scheme –The server gives host i chunk of size z i. –Host i sends this chunk to all other hosts –Server also sends chunk of size z s to all hosts Objective, find z i and z s so that each host and the server finish at the same time Let T be the duration of download. T = (N-1) z i / u i –Or z i = T/(N-1)  u i T = (F-zs)/ u s + N  z s / u s –Or z s = (T-F/ u s )/(N-1)* u s Must have F = sum(z i ) + z s Which implies, F = sum(T/(N-1)  u i ) + (T-F/ u s )/(N-1)  u s Which implies F = T/(N-1)(u s + sum(u i )) -F/(N-1) NF = T (u s + sum(u i )) T = NF/ (u s + sum(u i )) Check that z s >=0 –Is (T-F/ u s )/(N-1)* u s >0 –Is (T-F/ u s )>0 –Is NF/ (u s + sum(u i )) > F/ u s –Is (us + sum(u i )) /N < u s, yes

File distribution time: P2P usus u2u2 d1d1 d2d2 u1u1 uNuN dNdN Server Network (with abundant bandwidth) F server must send one copy: –F/u s time client i download time –F/d i Total data to be downloaded –NF fastest possible transfer rate: u s +  u i d P2P = max { F/u s, F/min(d i ), NF/(u s +  u i ) } i Can you make a schedule for the download the take this amount?

Server-client vs. P2P: example Client upload rate = u, F/u = 1 hour, u s = 10u, d min ≥ u s Conclusion: P2P systems are scalable. But the load is distributed to all users, so P2P users have more load than clients in the client-server model.

Project 2

Peer-to-peer querying While the file is transferred from the peer, how to find the file Options –Centralize directory Napster Single point of failure Congestion –Server would be the performance bottleneck Target for the RIAA Always up (as oppose to user machine that goes up and down) Easy to find Easy protocol –Query flooding Gnutella Hosts find other host and forms a network of neighbors (overlay network) Search for a file (covered on the next slide) How to set up the network – bootstrap? –Have a central list of peers –Have distributed lists of peers –Search out a peer by scanning (Project 2) Flood the network to answer query

Flooding Search O searcher

Flooding Search O searcher Searcher: Send a message to all neighbors that searcher is looking for file xyz

Flooding Search O searcher Receive the search message and respond if they have the file, otherwise, …

Flooding Search O searcher Every host that received message: Send a message to all neighbors that searcher is looking for file xyz

Flooding Search O searcher Receive the search message and respond if they have the file, otherwise, …

Flooding Search O searcher Every host that received message: Send a message to all neighbors that searcher is looking for file xyz

Flooding Search O searcher Every host that received message: Send a message to all its neighbors that searcher is looking for file xyz

Flooding Search O searcher Every host that received message: Every host that received the message for the first time: Send a message to all neighbors that searcher is looking for file xyz Received message for the first time Received message again, do not retransmit message

Querying Flooding State Diagram User Request for File wait Generate Id Send request message (with message Id) Originator of search

Querying Flooding State Diagram wait Request arrives Get message Id Have seen request before Check for file in directory Send response to peer that requested file File is in local dir Send request to all neighbors Listening peer

Querying Flood Nodes that don’t have the file Nodes that do have the file Don’t flood the entire network when searching.

Expanding Ring Search O searcher destination Originator: 0. set K=1 1.Generate ID and message with TTL=K 2.Wait TO seconds 1.If answer arrives, then exit 3.K=K+1, go to 1 Peer: 1.Wait for message 2.If message Id is old, go to 1 3.Check is query can be answer, 1.If so, send answer to originator and go to 1 4.Message TTL— 5.If TTL>0, send message to all neighbors, go to 1

Expanding Ring Search O searcher destination TTL=1 Time To Live Originator: 0. set K=1 1.Generate ID and message with TTL=K 2.Wait TO seconds 1.If answer arrives, then exit 3.K=K+1, go to 1 Peer: 1.Wait for message 2.If message Id is old, go to 1 3.Check is query can be answer, 1.If so, send answer to originator and go to 1 4.Message TTL— 5.If TTL>0, send message to all neighbors, go to 1

Expanding Ring Search O searcher destination TTL=2 Originator: 0. set K=1 1.Generate ID and message with TTL=K 2.Wait TO seconds 1.If answer arrives, then exit 3.K=K+1, go to 1 Peer: 1.Wait for message 2.If message Id is old, go to 1 3.Check is query can be answer, 1.If so, send answer to originator and go to 1 4.Message TTL— 5.If TTL>0, send message to all neighbors, go to 1

Expanding Ring Search O searcher destination TTL=4 Originator: 0. set K=1 1.Generate ID and message with TTL=K 2.Wait K*TO seconds 1.If answer arrives, then exit 3.If K>max_K, exit 4.K=K+1, go to 1 Peer: 1.Wait for message 2.If message Id is old, go to 1 3.Check is query can be answer, 1.If so, send answer to originator and go to 1 4.Message TTL— 5.If TTL>0, send message to all neighbors, go to 1

(hierarchical peer-to-peer network) KaZaA Not all peers are equal – super peers (?) –Super peers (group leaders) have higher bit-rate connections, are more stable, etc. Peers connect to group leaders The group leaders keep a list of files shared by all their children. group leaders connect to a small number of other group leaders A child host will ask its group leader for a file, if the group leader does not know where it is, it will flood the network of group leaders. The response from other group leaders follows a reverse path to the asking group leader (so other leader can cache the response) A file is identified with a ID (e.g., MD5) that can take a string (e.g., file) and come to a unique ID. A small change in the file causes a large change in the ID. It is not possible to construct two files that have the same ID. The ID is a finger print. Since files are ID-ed, multiple copies of the same file can be found and these copies can be downloaded from multiple hosts in parallel.

BitTorrent Centralized P2P –A centralized server, or tracker, tracks the clients involved in the P2P transfer –This is similar to Napster –Companies that host these site get sued and are attacked by DDoS Components of BitTorrent System –Torrent Files –Trackers –Seeders –Peers Trackerless is also possible

Torrent File Required to download Can be found on web sites or sent by email Contains information about the file and the tracker –Announce: the URL of the tracker –Creation date –Info Length of file Name of file Length of each piece (except for the last) Pieces – the 20B SHA-1 value of each piece If the download contains multiple files, then a single torrent file will contain information about all files.

Tracker Make a HTTP Get request to the tracker specifying the SHA-1 hash of the file to be downloaded –The request also includes the number of bytes downloaded and the number uploaded –If the client does not upload enough, the tracker might not provide a reply The reply contains –The time when the tracker information should be refreshed (usually 30 minutes) –A list of the peers that have the file IP address and port (usually 6881) Peer ID

File distribution with BitTorrent tracker: tracks peers participating in torrent obtain list of peers trading chunks peer

BitTorrent (1) file divided into 256KB chunks. peer joining torrent: –has no chunks, but will accumulate them over time –registers with tracker to get list of peers, connects to subset of peers (“neighbors”) while downloading, peer uploads chunks to other peers. peers may come and go once peer has entire file, it may (selfishly) leave or (altruistically) remain

BitTorrent (2) Pulling Chunks at any given time, different peers have different subsets of file chunks periodically, a peer (Alice) asks each neighbor for list of chunks that they have. Alice sends requests for her missing chunks –rarest first –So rarest chunks are spread, and chunks are uniformly common Sending Chunks: tit-for-tat Alice sends chunks to four neighbors currently sending her chunks at the highest rate –re-evaluate top 4 every 10 secs every 30 secs: randomly select another peer, starts sending chunks –newly chosen peer may join top 4 –“optimistically unchoke”

BitTorrent: Tit-for-tat (1) Alice “optimistically unchokes” Bob (2) Alice becomes one of Bob’s top-four providers; Bob reciprocates (3) Bob becomes one of Alice’s top-four providers With higher upload rate, can find better trading partners & get file faster!

BitTorrent Pros/Cons Centralized server Slow to get the transfer started –Web transfers start much faster and will achieve a sustained rate Peers must upload –Some peers might not be in position to upload (e.g., mobile phone) Chunks can be corrupted –HBO distributed fake chunks –Since the SHA-1 hash does not match what is given in the Torrent File, the chunk is dropped after it is downloaded This wastes bandwidth and can increase download time

Chapter 2 The Application Layer. Goals of this Chapter To understand common application protocols work –Web (http) – (smtp) –FTP –DNS –P2P –DHT (distributed.

Similar presentations

Presentation on theme: "Chapter 2 The Application Layer. Goals of this Chapter To understand common application protocols work –Web (http) – (smtp) –FTP –DNS –P2P –DHT (distributed."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Chapter 2 The Application Layer. Goals of this Chapter To understand common application protocols work –Web (http) – (smtp) –FTP –DNS –P2P –DHT (distributed.

Similar presentations

Presentation on theme: "Chapter 2 The Application Layer. Goals of this Chapter To understand common application protocols work –Web (http) – (smtp) –FTP –DNS –P2P –DHT (distributed."— Presentation transcript:

Similar presentations

About project

Feedback