4: Application Protocols: FTP, SMTP , POP and others Last Modified: 4/21/2017 8:52:51 AM 2: Application Layer

FTP Similar to HTTP in many ways Fetching files over the network from a server, sending files to a server Text based commands But differences too Separate data and command Stateful 2: Application Layer

FTP Model (RFC 959) Ftp client Interface FTP server Server User Protocol Interpreter Data Transfer Process Interface FTP server Server Protocol Interpreter FTP Commands/ Replies Server Data Transfer Process Data Connection Client connects to 21 to establish control channel Control channel actually follow the telnet protocol Over control channel specify port Who initiates the connection for the data channel? The FTP commands specify the parameters for the data connection (data port, transfer mode, representation type, and structure) and the nature of file system operation (store, retrieve, append, delete, etc.). The user-DTP or its designate should "listen" on the specified data port, and the server initiate the data connection and data transfer in accordance with the specified parameters . It should be noted that the data port need not be in the same host that initiates the FTP commands via the control connection, but the user or the user-FTP process must ensure a "listen" on the specified data port. It ought to also be noted that the data connection may be used for simultaneous sending and receiving. File System File System 2: Application Layer

FTP: separate control, data connections FTP client contacts ftp server at TCP port 21 Control channel: exchange commands, responses between client, server. “out of band control” Other parallel TCP connections are opened to transfer file data: Data channel: file data to/from server, can be used in either direction Different data channels can be established for each file/data transferred TCP control connection port 21 TCP data connection FTP client FTP server ASCII vs Binary mode for FTP Different platform define end of line characters different ASCII will send the type expected by the receiver Binary will send the type on the sender 2: Application Layer

ftp commands, responses Sample commands: sent as ASCII text over control channel USER username PASS password (sent in clear text!) LIST return list of file in current directory RETR filename retrieves (gets) file STOR filename stores (puts) file onto remote host Sample return codes status code and phrase (as in http) 331 Username OK, password required 125 data connection already open; transfer starting 425 Can’t open data connection 452 Error writing file telnet <server name> ftp 2: Application Layer

FTP server is stateful FTP server maintains state: Current user Established with USER + PASS REIN: deletes this state but leaves the control channel open QUIT: deletes state and closes the control channel Current working directory Set to user’s home directory after USER Changed with CWD Reset with CDUP 2: Application Layer

Transfer Parameters Client can specify many characteristics of the data channel PORT – client can specify IP address and port for server to connect to PORT a1,a2,a3,a4,p1,p2 = IP address a1.a2.a3.a4 port p1*256+p2 Crazy syntax and also security hole! Hides source of attack (appears as FTP server) Servers on same LAN as FTP server may not be properly configured to defend against attacks from within PASV – Server will wait for incoming data connection from client (good if client is behind a firewall/NAT) 2: Application Layer

FTP timeline CONTROL CHANNEL Data channel Data channel for ls Data channel for file retrieval 2: Application Layer

Electronic Mail 2: Application Layer

Electronic Mail POP Components: SMTP IMAP Mail clients Mail servers user mailbox outgoing message queue mail server user agent SMTP POP IMAP Components: Mail clients Composing, editing, reading mail messages e.g., Outlook, Exchange, elm, Netscape Messenger Mail servers Receive outgoing mail from mail clients via SMTP Receive incoming mail from other mail servers via SMTP Allow mail clients to receive incoming mail via POP, IMAP or others Outgoing, incoming messages stored on server 2: Application Layer

Electronic Mail: mail servers user mailbox outgoing message queue mail server user agent SMTP POP IMAP Mail Servers mailbox contains incoming messages (yet to be read) for user message queue of outgoing (to be sent) mail messages (if message cannot be delivered will stay in queue) smtp protocol between mail servers to send email messages Mail server is an SMTP client when sending mail Mail server is an SMTP server” when receiving mail 2: Application Layer

Gmail and Clarkson? Use Gmail to send/read mail? Forward your Clarkson email to Gmail? Ok what happens when send email to your Clarkson address Use HTTP to send to Gmail servers SMTP Gmail servers to Clarkson Stored locally at Clarkson Gmail account uses POP to retrieve Use HTTP to read over Gmail 2: Application Layer

Electronic Mail: smtp [RFC 2821] Uses tcp to reliably transfer email msg from client to server, port 25 direct transfer: sending server to receiving server three phases of transfer handshaking (greeting) transfer of messages closure More stateful than HTTP but less so that FTP command/response interaction commands: ASCII text response: status code and phrase Much like HTTP and FTP 2: Application Layer

SMTP History SMTP has been around a long time RFC done in 1982 In use well before that 2: Application Layer

Sample smtp interaction S: 220 hamburger.edu C: HELO crepes.fr S: 250 Hello crepes.fr, pleased to meet you C: MAIL FROM: <alice@crepes.fr> S: 250 alice@crepes.fr... Sender ok C: RCPT TO: <bob@hamburger.edu> S: 250 bob@hamburger.edu ... Recipient ok C: DATA S: 354 Enter mail, end with "." on a line by itself C: Do you like ketchup? C: How about pickles? C: . S: 250 Message accepted for delivery C: QUIT S: 221 hamburger.edu closing connection telnet example here 2: Application Layer

try smtp interaction for yourself: telnet servername 25 see 220 reply from server enter HELO, MAIL FROM, RCPT TO, DATA, QUIT commands above lets you send email without using email client (reader) How do you know the right server name? Trace it – does your mail data go in the clear? snoball% nslookup -query=MX cs.cornell.edu Note: nslookup is deprecated and may be removed from future releases. Consider using the `dig' or `host' programs instead. Run nslookup with the `-sil[ent]' option to prevent this message from appearing. Server: 128.84.96.115 Address: 128.84.96.115#53 cs.cornell.edu mail exchanger = 20 sundown.cs.cornell.edu. cs.cornell.edu mail exchanger = 30 simon.cs.cornell.edu. cs.cornell.edu mail exchanger = 10 sundial.cs.cornell.edu. 2: Application Layer

What is missing? Some commands processed by SMTP protocol mirror mail headers we are used to seeing in our email messages (To, From, …), but are not the same things Email headers (To, From, CC, Subject, Date, ..) are considered part of the data by SMTP and are not processed SMTP server at all! Email headers are processed by the mail reader software and ignored by SMTP How is Bcc implemented? Another example of “protocol” layering (like HTML and HTTP) BTW, Mail storage format is yet another issue 2: Application Layer

Mail message format Message headers Message body SMTP Data smtp: protocol for exchanging email msgs RFC 2822: standard for text message format (format of data from smtp perspective) header lines, e.g., To: CC: Subject: different from SMTP commands! body the “message”, ASCII characters only Message headers blank line Message body 2: Application Layer

Sample smtp interaction S: 220 hamburger.edu C: HELO crepes.fr S: 250 Hello crepes.fr, pleased to meet you C: MAIL FROM: <alice@crepes.fr> S: 250 alice@crepes.fr... Sender ok C: RCPT TO: <bob@hamburger.edu> S: 250 bob@hamburger.edu ... Recipient ok C: DATA S: 354 Enter mail, end with "." on a line by itself C: To: bob@hamburger.edu C: Subject: dinner preferences C: From: alice@crepes.fr C: C: Do you like ketchup? C: How about pickles? C: . S: 250 Message accepted for delivery C: QUIT S: 221 hamburger.edu closing connection telnet example here 2: Application Layer

SMTP format SMTP requires that message (header & body) be in 7-bit ASCII Made sense in text-based early days Requires encoding for binary data (jpegs, etc.) in 7-bit ASCII (yuck!) SMTP server uses CRLF.CRLF to determine end of message Can’t have CRLF.CRLF inside the message itself. If ever want that put CRLF..CRLF and have the server strip out the extra “.” 2: Application Layer

MIME for sending pictures and other binary data MIME: multimedia mail extension, RFC 2045, 2046 additional lines in msg header declare MIME content type From: alice@crepes.fr To: bob@hamburger.edu Subject: Picture of yummy crepe. MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Type: image/jpeg base64 encoded data ..... ......................... ......base64 encoded data MIME version method used to encode data multimedia data type, subtype, parameter declaration encoded data 2: Application Layer

MIME types: Extensible Content-Type: type/subtype; parameters Text example subtypes: plain, html Image example subtypes: jpeg, gif Audio example subtypes: basic (8-bit mu-law encoded), 32kadpcm (32 kbps coding) Video example subtypes: mpeg, quicktime Application other data that must be processed by reader before “viewable” example subtypes: msword, octet-stream 2: Application Layer

Multipart Type From: alice@crepes.fr To: bob@hamburger.edu Subject: Picture of yummy crepe. MIME-Version: 1.0 Content-Type: multipart/mixed; boundary=98766789 --98766789 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain Dear Bob, Please find a picture of a crepe. Content-Transfer-Encoding: base64 Content-Type: image/jpeg base64 encoded data ..... ......................... ......base64 encoded data --98766789-- 2: Application Layer

Common to send in html and text See sample email 2: Application Layer

SMTP servers add headers As they relay mail, SMTP servers add information about themselves Received: Message ID: Definitely a break in strict protocol layering 2: Application Layer

Valid email 2: Application Layer

Useful in tracking spam mail “Received:” and “MessageID” headers are part of the data Accurate and helpful from legitimate servers and user agents But not trustworthy from spam servers Mail rarely relayed multiple hops anymore 2: Application Layer

Spam/forged mail Start with a legitimate server you trust and verify the Mail From field (resolvable domain and matching IP address) Work backwards and find a “break” in the chain 2: Application Layer

Sample Spam 2: Application Layer Mnemosyne local and trusted From dogboyseven@aol.com Sat Sep 4 16:55:41 1999 Received: from cs2.CS.Berkeley.EDU (cs2.CS.Berkeley.EDU [169.229.60.56]) by mnemosyne.CS.Berkeley.EDU (8.9.1a/) with ESMTP id QAA20836 for <jnm@mailspool.CS.Berkeley.EDU>; Sat, 4 Sep 1999 16:55:38 -0700 (PDT) Received: from mail.everfaster.com (mail.everfaster.com [197.46.220.4]) by cs2.CS.Berkeley.EDU (8.9.1a/8.6.6.Beta11) with ESMTP id LAA18735 for <jnm@cs.berkeley.edu>; Sat, 4 Sep 1999 16:55:04 -0700 (PDT) Received: from gate.hypermoon.com (pool37.qs4w.longlink.net [217.6.1.7]) by mail.everfaster.com (8.8.7/8.8.7) with SMTP id PAA20074; Sat, 4 Sep 1999 19:54:21 -0400 (EDT) Received: from fritz.hotdogcity.com (fritz.hotdogcity.com [221.88.9.16]) by server.big-hello.com (8.8.8/8.8.8) with SMTP id RAA04617; Sat, 4 Sep 1999 19:53:33 -0400 (EDT) Received: by fritz.hotdogcity.com with Internet Mail Service (5.5.248.0) id Q19G494F; Sat, 4 Sep 1999 19:53:25 -0400 (EDT) Date: Sat, 4 Sep 1999 19:53:23 -0400 (EDT) From: Charles Lewis <clewis@hotmail.com> To: jnm@cs.berkeley.edu Subject: You'll never believe this! Message-ID: <19990904195323.H8159@fritz.hotdogcity.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii You won't believe this, but some company just paid me to surf the web! Check out... Mnemosyne local and trusted Gate.hypermoon.com is the likley the source of the traffic because it lies about who it is in Its header (says its server.big-hello.com) – Actually pool37.qs4w.longlink.net [217.6.1.7] Are more likely to be accurate if mail.everfaster.com is not careful about verifying the Reverse mapping of machines it talks to So fritz.hotdogcity.com and hotmail.com likely not involved at all – just to throw off the trail Mail.everfaster.com is configured to relay mail from outside its domain to Outside its domain Complaints should be sent to postmaster@everfaster.com regarding the overly per missive configuration, and to abuse@longlink.net regarding the antisocial behavior of its customer hypermoon.com. Although RFC 2142 documents the conventional "postmaster" and "abuse" addresses, there was some additional knowledge required to determine the domain longlink.net . The syntax of the Received field indicates that the name gate.hypermoon.com maps to the IP address 217.6.1.7, which maps back to the name pool37.qs4w.longlink.net. This is a typical situation for very small customers of internet service providers (ISPs)-- both the ISP and the customer have their own name for the same address. Sending a complaint to hypermoon.com would probably not be effective, because those are the people who sent the spam, but sending a complaint to their ISP is more likely to have an effect. 2: Application Layer

Spamcop In each received line, compare hostname of machine received from versus the IP address Does the hostname resolve to that IP address? Is that IP address listed as an MX for the domain listed in the hostname? Believe first parse line, consider discarding others as bogus Does first receive line list a well know relay? http://www.monkeys.com/upl/index.html If no, discard others 2: Application Layer

Reporting Spam Spamcop uses a combination of tools like dig, nslookup and finger to cross-check all the information in an email header and find the email address of the system administrator responsible for the network from which the mail was sent postmaster@domain or abuse@domain MTA server will use identd to identify username of owner of Process sending the mail (does it match the MAIL FROM line) IDENT user identification protocol RFC 1413 2: Application Layer

Mail Server Anti-spam configuration Do not relay mail except from specified sources (IP addresses within your own domain) Check for valid machine names in “MAIL FROM” Identify based on matches to “spam profiles” (keywords, etc.) Refusing to receive mail from blacklisted IP addresses Spamhaus and Spamcop Reaction: mark as spam or drop before reaches intended recipient? Act silently or give feedback to sender? 2: Application Layer

When mail server can’t send Warnings from mailserver Failures from mailserver 2: Application Layer

SMTP vs HTTP Smtp: persistent connections like HTTP 1.1 Both have ASCII command/response interaction, status codes http: each object is encapsulated in its own response message smtp: multiple objects message sent in a multipart message http: pull; smtp: push 2: Application Layer

SMTP = outgoing Notice we didn’t see any SMTP commands to “get” or “retrieve” mail SMTP is for outgoing mail only How do we get mail? Early days: log on to server and read mail from a mailbox = file on server How many people still read mail that way? Today most people read mail on their PC How do they get their mail from the mail server? 2: Application Layer

Incoming mail? SMTP SMTP POP3 IMAP HTTP Mailbox file Other? user agent user agent sender’s mail server receiver’s mail server Mailbox file POP: Post Office Protocol [RFC 1939] authorization (agent <-->server) and download IMAP: Internet Mail Access Protocol [RFC 1730] more features (more complex) manipulation of stored messages on server HTTP: Gmail, Hotmail , Yahoo! Mail, etc. Why not use HTTP to transfer random things like email? Convenient – don’t need mail reader just the ubiquitous web browser Other? 2: Application Layer

Why not just SMTP server on local machine? “Push not pull” means your PC must be constantly on to accept “push” 2: Application Layer

POP3 protocol authorization phase C: list transaction phase, client: S: +OK POP3 server ready C: user alice S: +OK C: pass hungry S: +OK user successfully logged on authorization phase client commands: user: declare username pass: password server responses +OK -ERR transaction phase, client: list: list message numbers retr: retrieve message by number dele: delete Quit C: list S: 1 498 S: 2 912 S: . C: retr 1 S: <message 1 contents> C: dele 1 C: retr 2 C: dele 2 C: quit S: +OK POP3 server signing off Try it yourself : telnet pop.enabled.mail-server 110 pobox 2: Application Layer

try POP interaction for yourself: telnet servername 110 see “OK POP3 server ready” reply from server enter user, pass, list, retr, dele commands above lets you send get you own email without using email client (reader) Trace it – do your password and mail data go in the clear? Do you configure your mail reader to pop mail every X minutes? Same as announcing your password regularly (unless over Kerberos etc)! 2: Application Layer

IMAP Allows user to set up and maintain multiple folders (for sorting mail) on the remote server Can get headers for and manipulate messages without downloading them (can even download individual MIME attachments) Don’t pay cost to download over slow link Don’t leave them on insecure computers Stateful protocol - stores per user information about folders and the status of the messages in them Folder information, actual messages Seen, Deleted, Answered flags per message POP stateful too but just username/password 2: Application Layer

IMAP con’t During an IMAP connection, the server transitions between multiple states Initially non-authenticated Authenticated Selected – folder selected and operations on messages permitted Finally, Logout state 2: Application Layer

Authentication in IMAP Client requests a certain AUTHENTICATION method C: A001 AUTHENTICATE KERBEROS_V4 If server implements that authentication mechanism then it will authenticate via that method S: + AmFYig== C: BAcAQU5EUkVXLkNNVS5FRFUAOCAsho84kL N3/IJmrMG+25a4DT+nZImJjnTNHJUtxAA+o0KPKfH EcAFs9a3CL5Oebe/ydHJUwYFd S: + or//EoAADZI= C: DiAF5A4gA+oOIALuBkAAmw== S: A001 OK Kerberos V4 authentication successful Sever can respond with NO if it does not support that authentication mechanism S: A001 NO authenticate failure Try it yourself : telnet pop.enabled.mail-server 110 pobox 2: Application Layer

Authentication in IMAP (cont) Client can try various authentication mechanisms in decreasing order of preference looking for one the server supports In the worst case, a client may authenticate with plain text login C: a001 LOGIN SMITH SESAME S: a001 OK LOGIN completed 2: Application Layer

Once authenticated, client can: SELECT a mailbox C: A142 SELECT INBOX S: * 172 EXISTS S: * 1 RECENT S: * OK [UNSEEN 12] Message 12 is first unseen S: * OK [UIDVALIDITY 3857529045] UIDs valid S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) S: * OK [PERMANENTFLAGS (\Deleted \Seen \*)] Limited S: A142 OK [READ-WRITE] SELECT completed CREATE, RENAME or DELETE mailboxes FETCH messages from a mailbox SEARCH through messages APPEND messages to a mailbox 2: Application Layer

Pop vs IMAP Similarities Differences Mail delivered to a shared, constantly connected server New mail accessible anywhere in network on a variety of platforms For access only, Need SMTP to send mail Differences POP simpler and more established (more clients and servers that support it) IMAP keeps more state and has more features; POP uses less server resources IMAP = prioritize download time; POP = shorter overall connection time 2: Application Layer

Mailservers play different roles Be careful to identify the role played by a specific mailserver Incoming mail to a domain May refuse to take RCPT TO other domains Allow users to get their incoming mail May not do SMTP at all Typically do require authentication Outgoing mail from a domain May require authentication from valid user May require that MAIL FROM match domain Can have one mail server do it all and can have no checking OR can divy up into specific roles and enforce those 2: Application Layer

Gmail and Clarkson? Use Gmail to send/read mail? Forward your Clarkson email to Gmail? Ok what happens when send email to your Clarkson address Use HTTP to send to Gmail servers SMTP Gmail servers to Clarkson Stored locally at Clarkson Gmail account uses POP to retrieve Use HTTP to read over Gmail 2: Application Layer

Sender Policy Framework (SPF) RFC 4408 Allows the owner of a domain to specify their mail sending policy E.g. they can specify which mail servers they use to send mail *from* their domain SPF record in DNS SPF query tool: http://www.kitterman.com/spf/validate.html 2: Application Layer

2: Application Layer

nslookup set query=txt clarkson.edu v=spf1 mx a:mymail.clarkson.edu a:lists.clarkson.edu a:janus.clarkson.edu a:web2.clarkson.edu a:milhouse.clarkson.edu a:outbound.clarkson.edu a:bulkmail.clarkson.edu 2: Application Layer

More Application Level Protocols? Telnet, Rlogin, SNMP (Simple Network Management Protocol), Instant Messenger (AIM), DHCP (BOOTP) , RPC, NFS, X,Finger, Whois,IDENT………………….. Any ones that you really wish we’d cover? You now know how to investigate any of these on your own RFCs for open protocols, Run apps and trace them, Get client/server source,… It would be a lot more fun to learn more than application level protocols though, right? 2: Application Layer

Roadmap We’ve looked at a bunch of application level protocols (HTTP, DNS, FTP, SMTP, POP, IMAP,, ..) – Lessons? Many were human readable – why? High level examples of protocol layering (SMTP, HTTP) Some ran on TCP, some on UDP, one on both – why? Used telnet/nslookup to interact with these protocols more directly Traced them (What went in clear text?!) Food-for-thought: Design a “Telephone Protocol” other choices? Next.. How would we implement an application level protocol ourselves? Socket API After that down to transport layer 2: Application Layer

Outtakes 2: Application Layer

Thanks to Jeffrey Vinocur (NNTP presentation, Spring 2002) Network News Thanks to Jeffrey Vinocur (NNTP presentation, Spring 2002) 2: Application Layer

What is Usenet? Reading/posting to Usenet newsgroups Conceptually: a semi-organized collection of forums (“newsgroups”) for public discussion Technically: a system for distributing email-like messages Most people here have used newsgroups. We’re going to focus today on the networks portion of Usenet. Essentially, all of the content of usenet is made possible by the underlying layer, which we can view as an enormously massive distributed database. JNM: Keyed by unique message id 2: Application Layer

Usenet Messages Format: like email, but a bit stricter and with some extra headers (e.g., Newsgroups) – we don’t care about this today, except for two important headers Message-ID: unlike email, every message truly needs to have a globally unique identifier Path: we’ll see this header later The content of the messages we also don’t care about today, except for two headers that we’ll need later. The first is a unique identifier for each message, and the second is a “trail” for where the message has been (like the Received: headers in email messages). How message-id generated 2: Application Layer

From: meowbot@meowing.net (A Meowbot) Newsgroups: alt.dev.null Path: news.litech.org!lnsnews.lns.cornell.edu!paradoxa.ogoense.net!not-for-meow From: meowbot@meowing.net (A Meowbot) Newsgroups: alt.dev.null Subject: Why? Date: Sun, 27 Jan 2002 23:25:52 +0000 (UTC) Organization: a tyranny of meowing fascist censor cabalists Lines: 4 Approved: nope. Message-ID: <mW.3C548C72.8BC5@K0deZ.scriptkiddie.net> X-Trace: paradoxa.ogoense.net 1012173952 6565 141.154.205.147 (27 Jan 2002 23:25:52 GMT) X-Complaints-To: abuse@ogoense.net X-Meow: Wouf Mail-Copies-To: nobody X-No-Repost: yes Xref: news.litech.org alt.dev.null:492 Because we like you. -- Meow Here’s an example message…a lot like email, really. What is not-for-meow – the originating hostname? 2: Application Layer

Network Topology Users connect to a local site Each site may have several servers for better throughput Sites are connected by (manually-requested and -configured) peering links to other sites Major sites have hundreds of peers So, how is this worldwide network arranged? Each user connects to a local server, usually run by the same people who are providing email and other services. Each server is connected to a few other servers, perhaps as many as a few hundred, known as “peers”. This connections are made manually, mostly by people sending out email to a remote site and asking nicely. 2: Application Layer

So I post…then what? The goal is for every article to make it to every server in the world – the “floodfill” model This can be as fast as a few seconds or as long as a few days (normally a few hours) When a new article appears on the network, the idea is that it flows along the links between peers and eventually reaches every corner of the world. 2: Application Layer

Serious bandwidth Credit: CAIDA (1999) 2: Application Layer This wouldn’t be hard, except that the amount of data involved is enormous -- that green stripe in the middle of the screen is using something like ten percent of all of the traffic in the entire Internet. JNM: Imagine if each web server got a copy of all the web pages in the world everyday (or Even all the new web pages). Web servers for small sites rarely get that many hits but news servers even for small sites get a large flow of news. 2: Application Layer

An article arrives… This can be either a new post from a user or an article being “fed” from a peering server. The server’s “name” added to the Path header (history of where the article has been) The server stores the article so users can read it For each of the server’s peers, determine if the peer has seen the article already (first check for peer’s name in Path header, then ask the peer about the Message-ID) Send the article to peers who do not have it How does all of this traffic know where to go? Every article keeps a “history” of where it’s been already -- this is the Path header. As soon as an article shows up, the server adds its name to this history. Then the server tries to figure out where to send the article. Any server listed in the Path header has definitely seen this article already. Otherwise, we have to query the remote server to find out if it has already seen this article from somebody else. Let’s see an example. 2: Application Layer

Path headers and Message-IDs Let’s trace an article. The initial component (at the end!) of the Path header marks the original posting server; then the originating server adds its name: Path: paradoxa.ogoense.net!not-for-meow Then this article gets fed to a another server and then add their hostname: Path: lnsnews.lns.cornell.edu!paradoxa.ogoense.net!not-for-meow And then it gets fed to another server… Path: news.litech.org!lnsnews.lns.cornell.edu!paradoxa.ogoense.net!not-for-meow First news server to get it was not-for-meow..then paradoxa. Then lnsnews.lns.cornell..then To news.litech.org If we already received another the message from another source we won’t Ask to receive it Try one telnet newserver nntp telnet newsstand.cit.cornell.edu help mode reader list Authinfo user <user> Authinfo passwd <passwd> group <newsgrsoup name> Group cornell.class.cs519 Head <article #> Article <article #> How to list all subject lines? quit 2: Application Layer

Usenet, 1980 reed phs \ / \ uok---duke-unc / \ research vax135 | Credit: Mark Horton reed phs \ / \ uok---duke-unc / \ research vax135 | ucbvax 2: Application Layer

Usenet, 1981 Credit: ucbvax!mark pdp (Misc) ! (NC) (Misc) decvax sii reed phs--unc--grumpy duke34 utzoo cincy teklabs ! ! ! ! ! ! ! ! ! ! ! +--+----+-----+-+--+-------------+-------+------+ ! ! ! ! ! duke ! ! +------+---+-----------------------+--------+ ! ! ! ! ! ! ! ucbopt ! hocsr--mhtsa----research mh135a harpo-----chico : ! ! ! ! ucbcory ! ! eagle ihnss vax135 (Bell Labs) (UCB) : ! ! ! ! ! ucbvax--++----------+--+--+-----+--+------+--------+ : @ ! ! ! (Silicon Valley) ucbarpa @ (UCSD) sdcsvax ! menlo70--hao : @ sdcattb-----+ ! ! ! ucbonyx @ +-----ucsfcgl sytek sri-unix @ phonlab-----+ cca-unix sdcarl !- Uucp links : Berknet links @ Arpanet links 2: Application Layer

Usenet, 1993 Credit: Brian Reid 2: Application Layer

Usenet today 1.4 million articles daily ~ 360 GB daily Credit: Karl L. Swartz 1.4 million articles daily ~ 360 GB daily Over a 100 Mbit/sec link is > 8 hours! How long is that over a 100Mbit/sec link = over 8 hours Amazing that this is still done … USENET historically significanct and still supported – for how long? > Someone asked a question about the "articles per week" graph for > 1991-1998. Do you know what the periodic dip early in each year is? Christmas! You'll notice there's a lull in mid-year also; Usenet was (and is) a significantly academic-core community. So many, many groups get very quiet over winter break. In fact, news.announce.newgroups (where the group-creation process for Big-8 occurs) has an official Christmas vacation so that no votes occur during a three-week period around that time. 2: Application Layer

– Professor Gene Spafford, Purdue University Usenet is like a herd of performing elephants with diarrhea – massive, difficult to redirect, awe-inspiring, entertaining, and a source of mind-boggling amounts of excrement when you least expect it. – Professor Gene Spafford, Purdue University 2: Application Layer

Multimedia Applications 2: Application Layer

Multimedia Applications Audio/video conferencing, streaming audio, etc. On-demand playback: could download before beginning playback; could support rewind, fast forward etc.; start-up time and RTT not very important Live transmission: usually broadcast from one source like TV or radio; much like on demand; no rewind or fast forward; more sensitive to delay (how close to live?) Conferencing: interactive, start-up time and RTT matter alot Examples: vic (video conferencing), vat (audio conferencing), RealAudio, Quicktime, WindowsMedia 2: Application Layer

Requirements of multimedia Several methods for compressing and encoding voice/video; sender and receiver negotiate Ability to display stream (at degraded quality) with lost packets Ability to specify the timing requirements between packets of related data for smooth playback Frame boundary indication Synchronization of related audio and video streams No retransmission of lost packets 2: Application Layer

Real-time Transport Protocol (RTP) TCP overhead to high; UDP not good enough Initially, each application had its own protocol, implementing only those parts of TCP it really needed on top of UDP RTP offers generalized real time transport services Thin protocol; Runs on top of UDP Implements functionality commonly needed by multimedia applications - timing reconstruction, loss detection, security and content identification RFC 1889 2: Application Layer

Realtime Transport (?) Protocol Is this an application level protocol or a transport protocol? Done at application level If TCP implemented at application level (good project ), does that make it an application level protocol or a transport level protocol? Where is the right place to put these features? 2: Application Layer

Real-time Streaming Protocol (RTSP ) Network “Remote Control” Like FTP has data channel and control channel; RTSP is the control channel for streaming audio/video Not used to deliver data; often uses RTP for the data portion Establishes and controls audio and video delivery Single or multiple audio/video streams (time synchronization if desired) Live feeds or stored clips Industry consortium announced in 1996 – since then? Mostly development continued on proprietary versions: Real Network’s (originally Progressive Networks) RealMedia, RealAudio and RealPlayer , Quicktime, WindowsMedia??? 2: Application Layer

RTSP Requests DESCRIBE – description of presentation OPTIONS - get supported methods; capability announcements SETUP – establish a new session PLAY – start playback/streaming; reposition ANNOUNCE – change description of presentation RECORD – start recording REDIRECT – redirect client to a new server; for load balancing PAUSE –stop delivery but keep state TEARDOWN – stop delivery, remove state 2: Application Layer

Trying RTSP telnet servername 554 C: DESCRIBE rtsp://streamserver/rafile.rm RTSP/1.0\n\n S: RTSP/1.0 200 2: Application Layer

Trying RTSP (2) C: SETUP rtsp://audio.example.com/twister/audio RTSP/1.0 Transport: rtp/udp; compression; port=3056; mode=PLAY S: RTSP/1.0 200 1 OK Session 4231 C: PLAY rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231 Range: npt=0- C: PAUSE rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Range: npt=37 C: TEARDOWN rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 S: 200 3 OK 2: Application Layer

RTSP vs HTTP RTSP actually derived from HTTP Avoid mistakes (like always specify full URI) More methods of course RTSP server needs to maintain state from SETUP to control PLAY command; HTTP server is stateless (uses cookies to trick client into remembering it) Data can be delivered in or out of band with RTSP; HTTP data delivered in band RTSP is a symmetric protocol (client and server can both issue requests); HTTP client issues requests Ex. server can announce new available streams (audio from a new participant in a conference) 2: Application Layer

Session Description Formats Format for describing the number and sources for all streams in a presentation May offer alternatives Different audio channels in various languages Different quality of audio/video for various BW connections Specify timing requirements between various streams Examples: SDF, SDP 2: Application Layer

SDP example session (v 0)(o mhandley 2890844526 2890842807 IN IP4 126.16.64.4) (s Sd seminar)(i A seminar on the session description protocol) (u http://www.cs.ucl.ac.uk/staff/M.Handley/sdp.01.ps) (e M.Handley@cs.ucl.ac.uk (Mark Handley)) (c IN IP4 224.2.17.12/127)(t 2873397496 2873404696) (a recvonly) (all (media (m audio 3456 VAT PCMU)) (media (m video 2232 RTP H261)) (media (m whiteboard 32416 UDP WB)(orient portrait)) )) Session description v= (protocol version) o= (owner/creator and session identifier). s= (session name) i=* (session information) u=* (URI of description) e=* (email address) p=* (phone number) c=* (connection information - not required if included in all media) b=* (bandwidth information) One or more time descriptions (see below) z=* (time zone adjustments) k=* (encryption key) a=* (zero or more session attribute lines) Zero or more media descriptions (see below) Time description t= (time the session is active) r=* (zero or more repeat times) Media description m= (media name and transport address) i=* (media title) c=* (connection information - optional if included at session-level) b=* (bandwidth information) k=* (encryption key) a=* (zero or more media attribute lines) From: http://www.cs.columbia.edu/~hgs/rtsp/sdf.html 2: Application Layer

From URL in web page to streaming audio/video <EMBED SRC=“http://server/foo.sdf” TYPE = “application/x-audio”> HTTP gets session or presentation description file ( not part of RTSP) from a web server Presentation Description indicates RTSP server to contact Note: RTSP is presentation description format neutral RTSP sets up a stream to control delivery RTSP used to indicate server that will actually stream the data and by what protocol Ex. specify an RTP server to deliver the data Note: possibly 3 servers involved! 2: Application Layer

Alternative: HTTP Streaming Many sites simply send audio and video over HTTP When object arrives will be opened by appropriate application just like Doc files or PDF files Estimate when it is safe to begin playback without the playback outpacing the download Download mode and a limited streaming mode can be supported this way Rewind? Fast forward? Can support full streaming if delays ok 2: Application Layer

Audio and Video on the Internet Quicktime HTTP streaming or RTP and RTSP RealServer one control channel: RTSP over TCP one data channel: PNA (Progressive Networks Audio) over UDP (?) Also can use RTSP to interleave data and control onto one TCP channel (common configuration) WindowsMedia Similar to RealPlayer: control channel and data channel Harder to find details of protocols (surprise, surprise) But formats are not compatible (surprise, surprise) 2: Application Layer

Email viruses Often attachments which once opened run with the users full privileges and corrupt the system on which mail is read Viruses tend to target Windows as it is the platform used by the majority of people 2: Application Layer

More FTP Details about TYPE and MODE commands REST – restart at specified data checkpoint 2: Application Layer

FTP minimum requirements MINIMUM IMPLEMENTATION In order to make FTP workable without needless error messages, the following minimum implementation is required for all servers: TYPE - ASCII Non-print MODE - Stream STRUCTURE - File, Record COMMANDS - USER, QUIT, PORT, TYPE, MODE, STRU, for the default values RETR, STOR, NOOP. The default values for transfer parameters are: TYPE - ASCII Non-print MODE - Stream STRU - File All hosts must accept the above as the standard defaults. 2: Application Layer

telnet source We’ve been using telnet to examine various application protocols telnet basically opens a TCP connection to the specified port Getting the telnet source and examining it would be a good exercise 2: Application Layer

Real Time Control Protocol (RTCP) Real-time conferencing of groups of any size within an internet. Provides source identification, quality-of-service feedback from receivers to the multicast group, synchronization of different media streams More info on RTCP or drop? 2: Application Layer

ReSerVation Protocol (RSVP) Host can use to request specific quality of service from the network for a specific flow of data Must be processed and honored at each router to be meaningful Works much like dynamic routing protocols; messages processed by applications at user level If a flow is “admitted” then resource reservation decisions will be made in form of packet classifier and schedulers that will prioritize the use of resources Cisco’s take on RSVP http://www.cisco.com/univercd/cc/td/doc/cisintwk/ito_doc/rsvp.htm More info on this or drop? 2: Application Layer

Multiple recipients When you send mail to your outgoing mail server, transfer one copy of message regardless of how many recipients Mail servers could play the same trick Look at RCPT to list If more than one recipient per destination mail server then transfer just one mail Could also send one copy per recipient Recommended configuration? 2: Application Layer