1. INFO 203Week #61 IT For Engineers Application and Transport Layers INFO 203 Dr. Jennifer Booker

2. Application Layer The Application Layer is the reason the rest of the network exists – to serve applications Most of the software familiar to end users are applications Email, FTP, newsgroups, chat, the Web, streaming video, video conferencing, IPTV, etc. We focus first on key concepts related to the Application Layer, then discuss some specific applications briefly Week #62 INFO 203

3. Application Layer New applications designed for network implementation need to decide whether the application is based on Client-server architecture Peer to peer (P2P) Or some hybrid combination of the two Week #63 INFO 203

4. Client-server Architecture In client-server architecture, the server Handles requests from many clients, and Is generally always available Often has a fixed IP address Clients generally don’t communicate with each other, and may be on or off independently of each other and the server Client-server applications include email, FTP, the Web, remote login Week #64 INFO 203

5. P2P Architecture P2P architecture assumes the clients are on or off at will, and all are treated equally as potential servers and/or clients Apps include BitTorrent, Skype, and IPTVBitTorrentSkype Client-server and P2P combinations exist, called a Hybrid Architecture Week #65 INFO 203

6. Process Communication Any network application (no matter which architecture) needs to communicate between hosts using processes In this sense, a process is a program running on a client, server, or peer host Processes may communicate with other processes on the same host; this is controlled by the host’s operating system (OS) We are interested in processes that communicate between hosts Week #66 INFO 203

7. Process Communication Processes exchange messages The sending or client process creates a message and sends it into the network The receiving or server process gets the message from the network and might reply Notice that client and server process only relate to their relative roles in sending a message, not the client-server or other architectures mentioned earlier Week #67 INFO 203

8. Addressing Processes For the server process to get the message, it has to be addressed correctly The host address and receiving process are the key parts of the address The host address is its IP address (the 32- or 128-bit address of the host’s network interface) The receiving process is identified by its port number, since many processes can be running at once Week #68 INFO 203

9. Addressing Processes Week #69 Sockets send packetsPorts listen for them INFO 203

10. Port Number Port numbers follow default values, set by the IANA, unless specified otherwiseIANA 21 = FTP 23 = Telnet 25 = SMTP 53 = DNS 80 = HTTP, http://mine.com implies http://mine.com:80 110 = POP3 194 = IRC, and hundreds more Week #610 INFO 203

11. More Protocols Application-layer protocols define how a particular application’s processes are structured What types of messages are allowed The syntax of those messages The meaning of the fields in the syntax Rules for processing messages – when and how to send messages, how to reply, etc. Week #611 INFO 203

12. Application vs its protocols A single application often needs to use several application-layer protocols A web browser might use HTTP, but also FTP, telnet, gopher, etc. An email application might use POP3, SMTP, IMAP, etc. Many app protocols are defined in RFCs But many application-layer protocols are proprietary Week #612 INFO 203

13. RFC Summary The “Internet Official Protocol Standards” RFC used to identify the current standards (STD) for every protocol As a result of RFC 7100, that information is on a website http://www.rfc-editor.org/search/standards.php http://www.rfc-editor.org/search/standards.php For example, STD 9 is the standard for FTP Week #613 INFO 203

14. Application Services The transport layer connects the application layer to everything else Have a choice of two protocols, TCP and UDP, unless you want to write your own! Key services include Reliable data transfer – how important is it? Or is your app loss-tolerant? Week #614 INFO 203

15. Application Services How much bandwidth or throughput does your app need? Does sending rate have to equal receiving rate? Some apps are elastic – can tolerate wide ranges of available bandwidth How sensitive is your app to timing? Games and telephony tend to be sensitive to slow or erratic transmission delays How important is security? Week #615 INFO 203

16. TCP Services TCP provides a connection-oriented service, where the sockets of the client and server recognize a connection for the duration of the session Connection is duplex – messages can go both ways at once TCP is highly reliable – the bits leaving one side all get to the other side, and get put back in the original order Week #616 INFO 203

17. TCP Services TCP also provides congestion control, for benefit of the Internet This throttles the sending processes when the connection is congested, and can limit bandwidth TCP does not guarantee any level of transmission rate, or provide delay guarantees So you’ll get your data across, but we don’t know when Week #617 INFO 203

18. UDP Services UDP is a lightweight protocol – meaning it doesn’t do much! UDP is connectionless UDP is unreliable – data may never get there UDP packets may arrive out of order and not realize it There are no transmission rate guarantees Week #618 INFO 203

19. Services NOT Provided TCP and UDP do not provide guarantees of throughput or timing TCP does nothing for security per se, but SSL can be added on See Chapter 7 in INFO 331 Week #619 INFO 203

20. Application Protocols We’ll examine protocols for Internet-based applications HTTP FTP SMTP POP3 IMAP DNS Week #620 INFO 203

21. HTTP The HyperText Transfer Protocol (HTTP) is the heart of the WebHTTP Defined by RFCs 1945 (v1.0) and 2616 (v1.1) Has client and server programs which communicate via HTTP messages Web pages contain objects – files of various sorts, such as a base HTML file, which cites JPG and/or GIF images, etc. App to use HTTP is a browser Week #621 INFO 203

22. HTTP A Web server houses the objects Apache and Microsoft Internet Information Services (IIS) are common Web server apps Apache IIS HTTP defines the messages that pass between client and server Uses TCP for transport protocol HTTP has no memory of previous actions (a stateless protocol) – so if you ask for a file 126 times, it will send the file 126 times Week #622 INFO 203

23. HTTP vs HTML Don’t confuse HTTP with HTML HTTP is the protocol used to define how files are requested and transferred between server and clients HTML is the format of web pages So an HTML file might be the structure of an entity body transferred using HTTP Week #623 INFO 203

24. HTTP Messages HTTP messages are two types, request messages (from client) and response messages (from server) All HTTP messages are plain ASCII text ‘Both types of message consist of a start-line, zero or more header fields (also known as "headers"), an empty line (i.e., a line with nothing preceding the CRLF) indicating the end of the header fields, and possibly a message-body.’ [RFC 2616, para 4.1] CRLF is a “carriage return and line feed” Week #624 INFO 203

25. HTTP Messages There are many headers which could appear in requests or responses Cache-Control, Connection, Date, Pragma, Trailer, Transfer-Encoding, Upgrade, Via, and/or Warning [RFC 2616, para 4.5] Disclaimer: RFC 2616 is 176 pages long – so we’re just providing a summary! Week #625 INFO 203

26. HTTP Requests Request messages have variable number of lines, depending on the method called General request syntax is Method Request-URI HTTP-Version Methods are OPTIONS, GET, HEAD, POST, PUT, DELETE, TRACE, or CONNECT [RFC 2616, para 5.1.1] Most commonly used is GET Request-URI is the desired Uniform Resource Identifier (URI, commonly called a URL) Week #626 INFO 203

27. HTTP Requests HTTP-Version is what it sounds like, e.g. HTTP/1.1 There are many possible request headers Accept, Accept-Charset, Accept-Encoding, Accept-Language, Authorization, Expect, From, Host, If-Match, If-Modified-Since, If-None-Match, If-Range, If-Unmodified-Since, Max-Forwards, Proxy-Authorization, Range, Referer, TE (extension transfer-codings), and/or User-Agent [RFC 2616, para 5.3] Week #627 INFO 203

28. HTTP Responses HTTP responses go from server to client General syntax starts with HTTP-Version Status-Code Reason-Phrase [RFC 2616, para 6.1] The Status-Code could be dozens of values "200" OK "403" Forbidden "404" Not Found The Reason-Phrase is any text phrase assigned Week #628 INFO 203

29. HTTP Responses Response headers can include Accept-Ranges, Age, ETag, Location, Proxy-Authenticate, Retry-After, Server, Vary, and/or WWW-Authenticate [RFC 2616, para 6.2] Responses usually include entities, unless the HEAD method was used Week #629 INFO 203

30. HTTP Entities An entity is the object sent or returned with an HTTP message Entities can be with requests or responses Entity headers include Allow, Content-Encoding, Content-Language, Content-Length (bytes), Content- Location, Content-MD5, Content-Range, Content- Type, Expires, Last-Modified, and/or extension- header [RFC 2616, para 7.1] Where extension-header is any allowable message-header for that kind of message Week #630 INFO 203

31. HTTP So HTTP describes request and response message formats Both types typically have a first line which tells its purpose (the request or status line) There can be many header lines There might be an entity attached Week #631 INFO 203

32. FTP The File Transfer Protocol is one of the oldest Internet applications (now RFC 959, but started as RFC 114 in 1971) While HTTP and FTP both send files FTP uses two connections – one for control, one for data (control information is out-of-band) User login and commands are on the control connection, files move on the data connection HTTP uses one connection for both purposes (control information is in-band) Week #632 INFO 203

33. FTP FTP uses TCP, and usually connects to the server on ports 20 and 21 The client sends user ID and password FTP may be done to some sites with generic ID, known as anonymous FTP Once logged in, the user may navigate and view directories, and upload (STOR or PUT) or download (RETR or GET) files Week #633 INFO 203

34. Electronic Mail E-mail is another ancient Internet application, with origins in RFC 772 in 1980 It provides asynchronous text communication and allows files to be attached to messages Even voice and video messages Main elements are users (sender and recipient), mail servers, and the Simple Mail Transfer Protocol (SMTP, RFC 5321) Careful, there’s also an SNTP for network time Week #634 INFO 203

35. Electronic Mail Email is composed in a client, which sends it to a mail queue in the sender’s mail server The sending mail server uses SMTP to send the message to the recipient’s mail server If mail can’t be sent successfully, the sender’s mail server will put the message in a queue, and keep trying (typically for 3 days) The recipient is notified that the message is present, which they read with their client Week #635 INFO 203

36. Electronic Mail Each user has a mailbox on the mail server Access to the mailbox is controlled with user name and password SMTP is the main protocol to get email from one mail server to another It uses TCP, not surprisingly Defined in draft standard RFC 5321 Only uses 7-bit ASCII for message AND body Forces binary files to be converted to ASCII & back Week #636 INFO 203

37. Mail Message Formats Email contains header information defined by RFC 822, now RFC 5322 “Internet Message Format”RFC 5322 The sender headers can include: FROM, SENDER, REPLY-TO, RESENT-FROM, RESENT-SENDER, and RESENT-REPLY-TO Receiver headers can be: TO, CC, and BCC Reference headers can be: MESSAGE-ID, IN- REPLY-TO, REFERENCES and KEYWORDS Week #637 INFO 203

38. MIME Multipurpose Internet Mail Extensions (MIME) are used for handling non-ASCII contents in email, e.g. non-Latin character sets, binary files, images, audio, video, etc. MIME (RFC 2045) adds the ability to handle (1) textual message bodies in character sets other than US-ASCII, (2) an extensible set of different formats for non-textual message bodies, (3) multi-part message bodies, and (4) textual header information in character sets other than US-ASCII. Week #638 INFO 203

39. MIME The received message also includes a Received: header added to the top of the message This is familiar in email if you look at the full headers Week #639 INFO 203

40. Mail Access Protocols If you log directly into your email server, SMTP is all you need to handle email But if you wish to access email from a local host, you need to use a mail access protocol The biggies at present are Post Office Protocol version 3 (POP3) and Internet Mail Access Protocol (IMAP) Week #640 INFO 203

41. POP3 POP3 is defined in RFC 1939 It’s a pretty simple protocol compared to many SMTP sends mail between mail servers, and from the user agent (email app) to their mail server POP3 transfers mail from your mail server to your user agent From a user’s view, SMTP handles outgoing email, and POP3 handles incoming email Week #641 INFO 203

42. IMAP IMAP, defined in RFC 3501, allows folders to be defined on the mail server to organize email there IMAP Messages are associated with a folder – first the generic INBOX, then moved by the user Hence state information about the folder for each message must be saved across sessions IMAP also provides search capability within the mailbox Week #642 INFO 203

43. DNS A key need, once the Internet grew beyond a few thousand hosts, was to automate converting human* readable addresses or hostnames (www.microsoft.com) to IP addresses (207.46.198.60) got IP herehere That is the purpose of the Domain Name System (DNS) Before DNS, really big lookup tables were used! Week #643 * Humans who read English, at least! INFO 203

44. Host vs Domain Names A hostname is the name of a particular host computer, such as banner.drexel.edu May really represent multiple computers, but logically they are all the same host A domain name is the top level domain and the specific domain name, like drexel.edu Top level domains are com, edu, gov, mil, org, net, etc. and the country codes uk, de, fr, etc. Top level domains Week #644 INFO 203

45. IP Addresses IP addresses have four groups of bytes, each group from 0 to 255, separated by periods Why called bytes? Each value from 0 to 255 corresponds to a value of from 0 to (2 8 -1), and a byte is eight bits IP addresses are typically static (fixed) for servers and other semi-permanent Internet connections, and dynamic for temporary connections (e.g. dial-up, wireless) Week #645 INFO 203

46. DNS DNS runs over UDP, port 53 (something uses UDP!) DNS is managed by DNS servers, typically running Berkeley Internet Name Domain (BIND) softwareBIND DNS is used by other applications (HTTP, SMTP, FTP) to translate host names to IP addresses You can also do a reverse DNS lookup (convert 205.188.97.2 to www-vd03.evip.aol.com) Week #646 INFO 203

47. DNS DNS also provides other key services Host aliasing allows the true or canonical hostname to have aliases When blah.com works to get to www.blah.com, it’s because blah.com is a host alias of www.blah.com Mail server aliasing – same concept, but for mail server names Load distribution across many servers for the same hostname – so everyone in the world doesn’t use one IP address for microsoft.com Week #647 INFO 203

48. DNS Lookup This would be terribly tedious without caching Common queries are stored on each level of DNS server, so they don’t have to be looked up constantly Cached values are cleared typically every two days or less, in case the data changes Week #648 INFO 203

49. nslookup The command nslookup provides basic IP data for a hostname or domain Nslookup snip.net Server: ns2.snip.net Address: 209.204.64.3 Name: snip.net Address: 216.83.103.123 A registrar makes changes to the DNS database The list of registrars is at http://www.internic.net/ http://www.internic.net/ Week #649 INFO 203

50. Transport Layer The Transport Layer handles logical communication between processes It’s the last layer not used between processes for routing, so it’s the last thing a client process and the first thing a server process sees of a packet By logical communication, we recognize that the means used to get between processes, and the distance covered, are irrelevant Week #650 INFO 203

51. Transport vs Network Notice we didn’t say ‘hosts’ in the previous slide…that’s because The network layer provides logical communication between hosts Week #651 INFO 203

52. Two Choices Here we choose between TCP and UDP In the transport layer, a packet is a segment In the network layer, a packet is a datagram The network layer is home to the Internet Protocol (IP) IP provides logical communication between hosts IP makes a “best effort” to get segments where they belong – no guarantees of delivery, or delivery sequence, or delivery integrity Week #652 INFO 203

53. IP Each host has an IP address Common purpose of UDP and TCP is extend delivery of IP data to the host’s processes This is called transport-layer multiplexing and demultiplexing Both UDP and TCP also provide error checking That’s it for UDP – data delivery and error checking! Week #653 INFO 203

54. TCP TCP also provides reliable data transfer (not just data delivery) Uses flow control, sequence numbers, acknowledgements, and timers to ensure data is delivered correctly and in order TCP also provides congestion control TCP applications share the available bandwidth (they watched Sesame Street!) UDP takes whatever it can get (greedy little protocol) Week #654 INFO 203

55. Segment Header Hence the segment header starts with the source and destination port numbers Each port number is a 16-bit (2 byte) value (0 to 65,535) Well known port numbers are from 0 to 1023 (2 10 - 1) Well known port numbers After the port numbers are other headers, specific to TCP or UDP, then the message Week #655 INFO 203

56. UDP The most minimal transport layer has to do multiplexing and demultiplexing UDP does this and a little error checking and, well, um, that’s about it! UDP was defined in RFC 768 An app that uses UDP almost talks directly to IP Adds only two small data fields to the header, after the requisite source/destination addresses There’s no handshaking; UDP is connectionless Week #656 INFO 203

57. UDP for DNS DNS uses UDP A DNS query is packaged into a segment, and is passed to the network layer The DNS app waits for a response; if it doesn’t get one soon enough (times out), it tries another server or reports no reply Hence the app must allow for the unreliability of UDP, by planning what to do if no response comes back Week #657 INFO 203

58. UDP Advantages Still UDP is good when: You want the app to have detailed control over what is sent across the network; UDP changes it little No connection establishment delay No connection state data in the end hosts; hence a server can support more UDP clients than TCP Small packet header overhead per segment TCP uses 20 bytes of header data, UDP only 8 bytes Week #658 INFO 203

59. UDP Apps Other than DNS, UDP is also used for Network management (SNMP) Routing (RIP) Multimedia & telephony (proprietary protocols) Remote file server (NFS) The lack of congestion control in UDP can be a problem when lost of large UDP messages are being sent – can crowd out TCP apps Week #659 INFO 203

60. Checksum Noise in the transmission lines can lose bits of data or rearrange them in transit Checksums are a common method to detect errors (RFC 1071) To create a checksum: Find the sum of the binary digits of the message The checksum is the 1s (ones) complement of the sum If message is uncorrupted, sum of message plus checksum is all ones 1111111111111… Week #660 INFO 203

61. 1s Complement? The 1s complement is a mirror image of a binary number – change all the zeros to ones, and ones to zeros So the 1s complement of 00101110101 is 11010001010 UDP does error checking because not all lower layer protocols do error checking This provides end-to-end error checking, since it’s more efficient than every step along the way Week #661 INFO 203

62. Reliable Data Transfer Mechanisms Checksum, to detect bit errors in a packet Timer, to know when a packet or its ACK was lost Sequence number, to detect lost or duplicate packets Acknowledgement, to know packet got to receiver correctly Negative acknowledgement, to tell packet was corrupted but received Window, to pipeline many packets at once before an ACK was received for any of them Week #662 INFO 203

63. TCP Intro Now see how all this applies to TCP First in RFC 793, now RFC 2581 Invented circa 1974 by Vint Cerf and Robert Kahn TCP starts with a handshake protocol, which defines many connection variables Connection only at hosts, not in between Routers are oblivious to whether TCP is used! TCP is a full duplex service – data can flow both directions at once, and is connection-oriented Week #663 INFO 203

64. TCP Segment Structure A TCP segment consists of header fields and a data field The data field size is limited by the MSS Typical header size is 20 bytes The header is 32 bits wide (4 bytes), so it has five lines at a minimum Week #664 INFO 203

65. TCP Header Structure The header lines are Source and destination port numbers (16 bit ea.) Sequence number (32 bit) ACK number (32 bit) A bunch of little stuff (header length, URG, ACK, PSH, RST, SYN, and FIN bits), then the receive window (16 bit) Internet checksum, urgent data pointer (16 bit ea.) And possibly several options Week #665 INFO 203

66. TCP Segment Structure We’ve seen the port numbers (16 bits each) Sequence and ACK numbers (32 bits each) keep track of pieces of a file The ‘bunch of little stuff’ includes Header length (4 bits) A flag field includes six one-bit fields: ACK, RST, SYN, FIN, PSH, and URG The URG bit marks urgent data later on that line The receive window is used for flow control Week #666 INFO 203

67. TCP Segment Structure The checksum is used for bit error detection, as with UDP The urgent data pointer tells where the urgent data is located The options include negotiating the MSS, scaling the window size, or time stamping Week #667 INFO 203

68. Telnet Example Telnet (RFC 854) is an old app for remote login via TCP Telnet interactively echoes whatever was typed to show it got to the other side Week #668 INFO 203

69. Timeout Calculation We want the timeout interval larger than EstimatedRTT, but not huge; use TimeoutInterval = EstimatedRTT + 4*DevRTT EstimatedRTT is a running average RTT DevRTT is a running standard deviation for RTT Timeout interval is constantly being calculated, with frequent measurement of SampleRTT to find current values for: Estimated RTT, DevRTT, & TimeoutInterval Week #669 INFO 203

70. Flow Control TCP connection hosts maintain a receive buffer, for bytes received correctly and in order Apps might not read from the buffer for a while, so it can overflow Flow control focuses on preventing overflow of the receive buffer So it also depends on how fast the receiving app is reading the data! Week #670 INFO 203

71. Flow Control Hence the sender in TCP maintains a receive window (RcvWindow) variable – how much room is left in the receive buffer The amount of room in RcvWindow is returned to the sender in the receive window field of every segment If the RcvWindow goes to zero, the sender can’t send more data to the receiver ever! To prevent this, TCP makes the sender transmit one byte messages when RcvWindow is zero, Week #671 INFO 203

72. UDP Flow Control There ain’t none (sic!) UDP adds newly arrived segments to a buffer in front of the receiving socket If the buffer gets full, segments are dropped Bye-bye data! Week #672 INFO 203

73. Congestion Control Now address congestion control issues Congestion is a traffic jam in the middle of the network somewhere Most common cause is too many sources sending data too fast into the network Key lessons are: A congested network forces retransmissions for packets lost due to buffer overflow, which adds to the congestion Week #673 INFO 203

74. Congestion Control And: A congested network can waste its bandwidth by sending duplicate packets which weren’t lost in the first place Dropping a packet wastes the transmission capacity of every upstream link that packet saw If loss and transmission delay are small, CongWin bytes of data can be sent every RTT, for a send rate of CongWin/RTT Week #674 INFO 203

75. Fairness Unequal connections are less fair Lower RTT gets more bandwidth (CongWin increases faster) UDP traffic can force out the more polite TCP traffic Multiple TCP connections from a single host (e.g. from downloading many parts of a Web page at once) get more bandwidth Week #675 INFO 203

76. Are We Done Yet? So we’ve covered transport layer protocols from the terribly simple UDP to a seemingly exhaustive study of TCP Key features along the way include multiplexing/demultiplexing, error detection, acknowledgements, timers, retransmissions, sequence numbers, connection management, flow control, end-to-end congestion control So much for the “edge” of the Internet; next is the network layer, to start looking at the core Week #676 INFO 203