1. The Application Layer – HTTP and FTP Tahir Azim

2. Application Layer Protocols QoS lectures postponed to next week This week: Application Layer Protocols –FTP, HTTP, Bittorrent Materials reused from Philip Levis, David Mazieres (Stanford) and Vern Paxson (Berkeley)

3. File Transfer Protocol (FTP) Allows a user to copy files to/from remote hosts –Client program connects to FTP server –… provides a login id and password –… allows the user to explore the directories –… and download and upload files with the server A predecessor of the Web (RFC 959 in 1985) –Requires user to know the name of the server machine –… and have an account on the machine –… and find the directory where the files are stored –… and know whether the file is text or binary –… and know what tool to run to render and edit the file That is, no URL, hypertext, and helper applications

4. How Do You Transfer Files Today? HTTP - the usual Web transfer mechanism (http://) FTP –ftp:// links in web pages (e.g.: in www.kernel.org) sftp and scp –E.g.: to upload your project files to cms.niit.edu.pk BitTorrent and other file-sharing software Any others?

5. Why Study FTP? Helps cement familiarity with text/status-code based protocols Illustrates use of multiple concurrent connections –One for control (commands & replies) –Depending on command, can be additional one for data Illustrates reversal of roles –For data connection, FTP user’s process can play the server role, FTP server can play the client role

6. Example commands Authentication –USER: specify the user name to log in as –PASS: specify the user’s password Exploring the files –LIST: list the files for the given file specification –CWD: change to the given directory Downloading and uploading files –TYPE: set type to ASCII (A) or binary image (I) –RETR: retrieve the given file –STOR: upload the given file Closing the connection –QUIT: close the FTP connection

7. Server Response Codes 1xx: positive preliminary reply –The action is being started, but expect another reply before sending the next command. 2xx: positive completion reply –The action succeeded and a new command can be sent. 3xx: positive intermediate reply –The command was accepted but another command is now required. 4xx: transient negative completion reply –The command failed and should be retried later. 5xx: permanent negative completion reply –The command failed and should not be retried.

8. FTP Data Transfer Separate data connection –To send lists of files (LIST) –To retrieve a file (RETR) –To upload a file (STOR) control data

9. Creating the Data Connection Client acts like a server –Creates a socket Assigned an ephemeral port number by the kernel –Listens on socket –Waits to hear from FTP server socket

10. Creating Data Connection (cont.) But, the server doesn’t know the port number –So after starting to listen, client tells it to the server –Using the PORT command on the control connection –Server can tell the client a port to connect to using PASV or EPSV PORT

11. Why Out-of-Band Control? Avoids need to mark the end of the data transfer –Data transfer ends by closing of data connection –Yet, the control connection stays up Aborting a data transfer –Can abort a transfer without killing the control connection –… which avoids requiring the user to log in again –Done with an ABOR on the control connection Third-party file transfer between two hosts –Data connection could go to a different host –… by sending a different client IP address to the server –e.g., a user can coordinate a transfer between two servers –But: this is rarely needed, and presents security issues

12. Example Collected using: –tcpdump -s 0 -w ftp.trace host www.niit.edu.pkwww.niit.edu.pk –ftp www.niit.edu.pkwww.niit.edu.pk –Ethereal

13. Example blade1:~ # ftp www.niit.edu.pkwww.niit.edu.pk Connected to www.niit.edu.pk. Server sends back the following: 220---------- Welcome to Pure-FTPd [privsep] [TLS] ---------- 220-You are user number 1 of 50 allowed. 220-Local time is now 18:54. Server port: 21. 220-This is a private system - No anonymous login 220 You will be disconnected after 15 minutes of inactivity.

14. Example continued… Name (www.niit.edu.pk:root): tahir USER tahir 331 User tahir OK. Password required Password: PASS tahirazim 230-User tahir has group access to: tahir 230 OK. Current restricted directory is / SYST 215 UNIX Type: L8 FEAT 211-Extensions supported: EPRT IDLE MDTM SIZE REST STREAM MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*; MLSD ESTP PASV EPSV SPSV ESTA AUTH TLS PBSZ PROT 211 End.

15. Example continued… ftp> pwd PWD 257 "/" is your current location ftp> cd public_html/tinyos CWD public_html/tinyos 250 OK. Current directory is /public_html/tinyos

16. Example continued… ftp> ls EPSV 229 Extended Passive mode OK (|||22710|) LIST 150 Accepted data connection 226-Options: -l 226 5 matches total Received from port 22710 of the server: -rw-r--r-- 1 tahir tahir 1143243 Feb 1 14:23 BVR-T2-0.1.tar.gz -rw-r--r-- 1 tahir tahir 535 Jan 18 20:03 Makefile -rw-r--r-- 1 tahir tahir 2788 Jan 18 20:03 RadioCountToLedsAppC.nc -rw-r--r-- 1 tahir tahir 5219 Jan 18 20:03 RadioCountToLedsC.nc -rw-r--r-- 1 tahir tahir 429056 Jan 18 20:03 nesC-wksp.ppt

17. Example continued… ftp> get Makefile TYPE I 200 TYPE is now 8-bit binary SIZE Makefile 213 535 EPSV 229 Extended Passive mode OK (|||9442|) RETR Makefile 150 Accepted data connection 226-File successfully transferred 226 0.000 seconds (measured here), 1.11 Mbytes per second MDTM Makefile 213 20080118150308 From port 9442 of the server: COMPONENT=RadioCountToLedsAppC BUILD_EXTRA_DEPS = RadioCountMsg.py RadioCountMsg.class RadioCountMsg.py: RadioCountToLeds.h.mig python -target=$(PLATFORM) $(CFLAGS) -python-classname=RadioCountMsg RadioCountToLeds.h radio_count_msg -o $@ RadioCountMsg.class: RadioCountMsg.java.javac RadioCountMsg.java RadioCountMsg.java: RadioCountToLeds.h.mig java -target=$(PLATFORM) $(CFLAGS) -java-classname=RadioCountMsg RadioCountToLeds.h radio_count_msg -o $@ #CFLAGS += -I/home/Tahir/T2SerialT1. include $(MAKERULES)

18. Example continued… ftp> exit QUIT 221-Goodbye. You uploaded 0 and downloaded 1 kbytes. 221 Logout.

19. HTTP Server listens on a port (by default, 80) On connection, waits for a request Protocol (but not data) is in ASCII Sends response, maybe closes connection (client can ask it to stay open)

20. Parsing a URL http://www.niit.edu.pk/~tahir/tcpip/index.html Protocol Host File path on host

21. HTTP Request Format Request types: GET, PUT, POST, HEAD, DELETE A trivial browser request: http://localhost:8000

22. A trivial browser request GET / HTTP/1.1 Host: localhost:8000 User-Agent: Mozilla/5.0 (Macinto... Accept: text/xml,application/xm... Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive

23. Other useful header fields Range: Request a partial range of data Authorization: Present authorization credentials to a server (not HTTPS) Proxy-Authorization: Present proxy credentials to a proxy server Referer: URL of the web page the user was on, when the HTTP request was made

24. HTTP Response Format 1xx codes: Informational 2xx codes: Successes 3xx codes: Redirection 4xx codes: Client Error, 5xx codes: Server Error

25. Response from thenews.com.pk HTTP/1.0 200 OK Date: Fri, 14 Mar 2008 10:13:05 GMT Server: Microsoft-IIS/6.0 X-Powered-By: ASP.NET Content-Length: 54063 Content-Type: text/html Set-Cookie: ASPSESSIONIDAADTQQAR=PFKAADCBBDGBDDJLMKLFJNMI; path=/ Cache-Control: private X-Cache: MISS from micronet-proxy.niit.edu.pk X-Cache-Lookup: MISS from micronet-proxy.niit.edu.pk:8080 X-Cache: MISS from proxy-ptcl.niit.edu.pk X-Cache-Lookup: MISS from proxy-ptcl.niit.edu.pk:8080 Via: 1.0 micronet-proxy.niit.edu.pk:8080 (squid/2.6.STABLE5), 1.0 proxy- ptcl.niit.edu.pk:8080 (squid/2.6.STABLE5) Connection: keep-alive The News - International - Friday, March 14, 2008 …

26. HTTP Performance What matters most? Different kinds of requests –Lots of small requests (loading a web page) –Big request (fetching a download) Require different solutions

27. Small requests Latency matters Governed by RTT between hosts Two major causes of delay: –Opening a TCP connection –Data response-request Solutions: –Persistent connections –Pre-fetching –Others??

28. Browser request, revisited GET / HTTP/1.1 Host: localhost:8000 User-Agent: Mozilla/5.0 (Macinto... Accept: text/xml,application/xm... Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive

29. Big requests Problem is throughput on bottleneck links (usually edge links) Use an HTTP proxy cache or mirror –Can also improve latency!

30. Stale caches Items in the cache can go stale (you don’t want to read yesterday’s paper) Cache needs a way to conditionally ask for a document Cache can issue a conditional GET (with an If- modified-since header) –Server can reply with a 304 Not Modified GET / HTTP/1.1 Host: www.niit.edu.pk If-modified-since: Wed, 2 April 2008 08:00:00

31. Client-server vs Peer-to-peer Server can be a bottleneck –Download time can scale down O(n) with n clients –Scaling up server bandwidth can be expensive (CDNs) –Slashdotting/flash crowds Peer-to-peer: get a bunch of end-hosts to collaboratively distribute content A common peer-to-peer challenge is finding whom to collaborate with

32. Bittorrent