1. 4: Application Protocols: FTP, SMTP , POP and others
Last Modified:
4/21/2017 8:52:51 AM
2: Application Layer

2. FTP Similar to HTTP in many ways
Fetching files over the network from a server, sending files to a server
Text based commands
But differences too
Separate data and command
Stateful
2: Application Layer

3. FTP Model (RFC 959) Ftp client Interface FTP server Server User
Protocol
Interpreter
Data
Transfer
Process
Interface
FTP server
Server
Protocol
Interpreter
FTP Commands/ Replies
Server
Data
Transfer
Process
Data Connection
Client connects to 21 to establish control channel
Control channel actually follow the telnet protocol
Over control channel specify port
Who initiates the connection for the data channel?
The FTP commands specify the parameters for the data connection (data port, transfer mode, representation type, and structure) and the nature of file system operation (store, retrieve, append, delete, etc.).
The user-DTP or its designate should "listen" on the specified data port, and the server initiate the data connection and data transfer in accordance with the specified parameters
. It should be noted that the data port need not be in the same host that initiates the FTP commands via the control connection, but the user or the user-FTP process must ensure a "listen" on the specified data port. It ought to also be noted that the data connection may be used for simultaneous sending and receiving.
File System
File System
2: Application Layer

4. FTP: separate control, data connections
FTP client contacts ftp server at TCP port 21
Control channel: exchange commands, responses between client, server.
“out of band control”
Other parallel TCP connections are opened to transfer file data:
Data channel: file data to/from server, can be used in either direction
Different data channels can be established for each file/data transferred
TCP control connection
port 21
TCP data connection
FTP
client
FTP
server
ASCII vs Binary mode for FTP
Different platform define end of line characters different
ASCII will send the type expected by the receiver
Binary will send the type on the sender
2: Application Layer

5. ftp commands, responses
Sample commands:
sent as ASCII text over control channel
USER username
PASS password (sent in clear text!)
LIST return list of file in current directory
RETR filename retrieves (gets) file
STOR filename stores (puts) file onto remote host
Sample return codes
status code and phrase (as in http)
331 Username OK, password required
125 data connection already open; transfer starting
425 Can’t open data connection
452 Error writing file
telnet <server name> ftp
2: Application Layer

6. FTP server is stateful FTP server maintains state: Current user
Established with USER + PASS
REIN: deletes this state but leaves the control channel open
QUIT: deletes state and closes the control channel
Current working directory
Set to user’s home directory after USER
Changed with CWD
Reset with CDUP
2: Application Layer

7. Transfer Parameters
Client can specify many characteristics of the data channel
PORT – client can specify IP address and port for server to connect to
PORT a1,a2,a3,a4,p1,p2 = IP address a1.a2.a3.a4 port p1*256+p2
Crazy syntax and also security hole!
Hides source of attack (appears as FTP server)
Servers on same LAN as FTP server may not be properly configured to defend against attacks from within
PASV – Server will wait for incoming data connection from client (good if client is behind a firewall/NAT)
2: Application Layer

8. FTP timeline CONTROL CHANNEL Data channel Data channel
for ls
Data channel
for file retrieval
2: Application Layer

9. Electronic Mail
2: Application Layer

10. Electronic Mail POP Components: SMTP IMAP Mail clients Mail servers
user mailbox
outgoing
message queue
mail
server
user
agent
SMTP
POP
IMAP
Components:
Mail clients
Composing, editing, reading mail messages
e.g., Outlook, Exchange, elm, Netscape Messenger
Mail servers
Receive outgoing mail from mail clients via SMTP
Receive incoming mail from other mail servers via SMTP
Allow mail clients to receive incoming mail via POP, IMAP or others
Outgoing, incoming messages stored on server
2: Application Layer

11. Electronic Mail: mail servers
user mailbox
outgoing
message queue
mail
server
user
agent
SMTP
POP
IMAP
Mail Servers
mailbox contains incoming messages (yet to be read) for user
message queue of outgoing (to be sent) mail messages (if message cannot be delivered will stay in queue)
smtp protocol between mail servers to send messages
Mail server is an SMTP client when sending mail
Mail server is an SMTP server” when receiving mail
2: Application Layer

12. Gmail and Clarkson?
Use Gmail to send/read mail? Forward your Clarkson to Gmail?
Ok what happens when send to your Clarkson address
Use HTTP to send to Gmail servers
SMTP Gmail servers to Clarkson
Stored locally at Clarkson
Gmail account uses POP to retrieve
Use HTTP to read over Gmail
2: Application Layer

13. Electronic Mail: smtp [RFC 2821]
Uses tcp to reliably transfer msg from client to server, port 25
direct transfer: sending server to receiving server
three phases of transfer
handshaking (greeting)
transfer of messages
closure
More stateful than HTTP but less so that FTP
command/response interaction
commands: ASCII text
response: status code and phrase
Much like HTTP and FTP
2: Application Layer

14. SMTP History SMTP has been around a long time RFC done in 1982
In use well before that
2: Application Layer

15. Sample smtp interaction
S: 220 hamburger.edu
C: HELO crepes.fr
S: 250 Hello crepes.fr, pleased to meet you
C: MAIL FROM:
S: 250 Sender ok
C: RCPT TO:
S: Recipient ok
C: DATA
S: 354 Enter mail, end with "." on a line by itself
C: Do you like ketchup?
C: How about pickles?
C: .
S: 250 Message accepted for delivery
C: QUIT
S: 221 hamburger.edu closing connection
telnet example here
2: Application Layer

16. try smtp interaction for yourself:
telnet servername 25
see 220 reply from server
enter HELO, MAIL FROM, RCPT TO, DATA, QUIT commands
above lets you send without using client (reader)
How do you know the right server name?
Trace it – does your mail data go in the clear?
snoball% nslookup -query=MX cs.cornell.edu
Note: nslookup is deprecated and may be removed from future releases.
Consider using the `dig' or `host' programs instead. Run nslookup with
the `-sil[ent]' option to prevent this message from appearing.
Server:
Address: #53
cs.cornell.edu mail exchanger = 20 sundown.cs.cornell.edu.
cs.cornell.edu mail exchanger = 30 simon.cs.cornell.edu.
cs.cornell.edu mail exchanger = 10 sundial.cs.cornell.edu.
2: Application Layer

17. What is missing?
Some commands processed by SMTP protocol mirror mail headers we are used to seeing in our messages (To, From, …), but are not the same things
headers (To, From, CC, Subject, Date, ..) are considered part of the data by SMTP and are not processed SMTP server at all!
headers are processed by the mail reader software and ignored by SMTP
How is Bcc implemented?
Another example of “protocol” layering (like HTML and HTTP)
BTW, Mail storage format is yet another issue
2: Application Layer

18. Mail message format Message headers Message body
SMTP Data
smtp: protocol for exchanging msgs
RFC 2822: standard for text message format (format of data from smtp perspective)
header lines, e.g.,
To:
CC:
Subject:
different from SMTP commands!
body
the “message”, ASCII characters only
Message headers
blank
line
Message body
2: Application Layer

19. Sample smtp interaction
S: 220 hamburger.edu
C: HELO crepes.fr
S: 250 Hello crepes.fr, pleased to meet you
C: MAIL FROM:
S: 250 Sender ok
C: RCPT TO:
S: Recipient ok
C: DATA
S: 354 Enter mail, end with "." on a line by itself
C: To:
C: Subject: dinner preferences
C: From: C:
C: Do you like ketchup?
C: How about pickles?
C: .
S: 250 Message accepted for delivery
C: QUIT
S: 221 hamburger.edu closing connection
telnet example here
2: Application Layer

20. SMTP format
SMTP requires that message (header & body) be in 7-bit ASCII
Made sense in text-based early days
Requires encoding for binary data (jpegs, etc.) in 7-bit ASCII (yuck!)
SMTP server uses CRLF.CRLF to determine end of message
Can’t have CRLF.CRLF inside the message itself. If ever want that put CRLF..CRLF and have the server strip out the extra “.”
2: Application Layer

21. MIME for sending pictures and other binary data
MIME: multimedia mail extension, RFC 2045, 2046
additional lines in msg header declare MIME content type
From:
To:
Subject: Picture of yummy crepe.
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Type: image/jpeg
base64 encoded data .....
......base64 encoded data
MIME version
method used
to encode data
multimedia data
type, subtype,
parameter declaration
encoded data
2: Application Layer

22. MIME types: Extensible Content-Type: type/subtype; parameters
Text
example subtypes: plain, html
Image
example subtypes: jpeg, gif
Audio
example subtypes: basic (8-bit mu-law encoded), 32kadpcm (32 kbps coding)
Video
example subtypes: mpeg, quicktime
Application
other data that must be processed by reader before “viewable”
example subtypes: msword, octet-stream
2: Application Layer

23. Multipart Type From: alice@crepes.fr To: bob@hamburger.edu
Subject: Picture of yummy crepe.
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary=
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain
Dear Bob,
Please find a picture of a crepe.
Content-Transfer-Encoding: base64
Content-Type: image/jpeg
base64 encoded data .....
......base64 encoded data
2: Application Layer

24. Common to send in html and text
See sample
2: Application Layer

25. SMTP servers add headers
As they relay mail, SMTP servers add information about themselves
Received:
Message ID:
Definitely a break in strict protocol layering
2: Application Layer

26. Valid
2: Application Layer

27. Useful in tracking spam mail
“Received:” and “MessageID” headers are part of the data
Accurate and helpful from legitimate servers and user agents
But not trustworthy from spam servers
Mail rarely relayed multiple hops anymore
2: Application Layer

28. Spam/forged mail
Start with a legitimate server you trust and verify the Mail From field (resolvable domain and matching IP address)
Work backwards and find a “break” in the chain
2: Application Layer

29. Sample Spam 2: Application Layer Mnemosyne local and trusted
From Sat Sep 4 16:55:
Received: from cs2.CS.Berkeley.EDU (cs2.CS.Berkeley.EDU [ ])
by mnemosyne.CS.Berkeley.EDU (8.9.1a/) with ESMTP id QAA20836 for
Sat, 4 Sep :55: (PDT)
Received: from mail.everfaster.com (mail.everfaster.com [ ])
by cs2.CS.Berkeley.EDU (8.9.1a/8.6.6.Beta11) with ESMTP id LAA18735 for Sat, 4
Sep :55: (PDT)
Received: from gate.hypermoon.com (pool37.qs4w.longlink.net [ ])
by mail.everfaster.com (8.8.7/8.8.7) with SMTP id PAA20074; Sat, 4 Sep :54: (EDT)
Received: from fritz.hotdogcity.com (fritz.hotdogcity.com [ ])
by server.big-hello.com (8.8.8/8.8.8) with SMTP id RAA04617; Sat, 4 Sep :53: (EDT)
Received: by fritz.hotdogcity.com with Internet Mail Service ( )
id Q19G494F; Sat, 4 Sep :53: (EDT) Date: Sat, 4 Sep :53: (EDT)
From: Charles Lewis
To:
Subject: You'll never believe this!
Message-ID:
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
You won't believe this, but some company just paid me to surf the web! Check out...
Mnemosyne local and trusted
Gate.hypermoon.com is the likley the source of the traffic because it lies about who it is in
Its header (says its server.big-hello.com) – Actually pool37.qs4w.longlink.net [ ]
Are more likely to be accurate if mail.everfaster.com is not careful about verifying the
Reverse mapping of machines it talks to
So fritz.hotdogcity.com and hotmail.com likely not involved at all – just to throw off the trail
Mail.everfaster.com is configured to relay mail from outside its domain to
Outside its domain
Complaints should be sent to regarding
the overly per missive configuration, and to
regarding the antisocial behavior of its customer
hypermoon.com. Although RFC 2142 documents the conventional
"postmaster" and "abuse" addresses, there was some additional
knowledge required to determine the domain longlink.net . The syntax
of the Received field indicates that the name gate.hypermoon.com maps
to the IP address , which maps back to the name
pool37.qs4w.longlink.net. This is a typical situation for very small
customers of internet service providers (ISPs)-- both the ISP and the
customer have their own name for the same address. Sending a complaint
to hypermoon.com would probably not be effective, because those are
the people who sent the spam, but sending a complaint to their ISP is
more likely to have an effect.
2: Application Layer

30. Spamcop
In each received line, compare hostname of machine received from versus the IP address
Does the hostname resolve to that IP address?
Is that IP address listed as an MX for the domain listed in the hostname?
Believe first parse line, consider discarding others as bogus
Does first receive line list a well know relay?
If no, discard others
2: Application Layer

31. Reporting Spam
Spamcop uses a combination of tools like dig, nslookup and finger to cross-check all the information in an header and find the address of the system administrator responsible for the network from which the mail was sent or
MTA server will use identd to identify username of owner of
Process sending the mail (does it match the MAIL FROM line)
IDENT user identification protocol RFC 1413
2: Application Layer

32. Mail Server Anti-spam configuration
Do not relay mail except from specified sources (IP addresses within your own domain)
Check for valid machine names in “MAIL FROM”
Identify based on matches to “spam profiles” (keywords, etc.)
Refusing to receive mail from blacklisted IP addresses
Spamhaus and Spamcop
Reaction: mark as spam or drop before reaches intended recipient? Act silently or give feedback to sender?
2: Application Layer

33. When mail server can’t send
Warnings from mailserver
Failures from mailserver
2: Application Layer

34. SMTP vs HTTP Smtp: persistent connections like HTTP 1.1
Both have ASCII command/response interaction, status codes
http: each object is encapsulated in its own response message
smtp: multiple objects message sent in a multipart message
http: pull; smtp: push
2: Application Layer

35. SMTP = outgoing
Notice we didn’t see any SMTP commands to “get” or “retrieve” mail
SMTP is for outgoing mail only
How do we get mail?
Early days: log on to server and read mail from a mailbox = file on server
How many people still read mail that way?
Today most people read mail on their PC
How do they get their mail from the mail server?
2: Application Layer

36. Incoming mail? SMTP SMTP POP3 IMAP HTTP Mailbox file Other?
user
agent
user
agent
sender’s mail
server
receiver’s mail
server
Mailbox file
POP: Post Office Protocol [RFC 1939] authorization (agent <-->server) and download
IMAP: Internet Mail Access Protocol [RFC 1730] more features (more complex) manipulation of stored messages on server
HTTP: Gmail, Hotmail , Yahoo! Mail, etc.
Why not use HTTP to transfer random things like ?
Convenient – don’t need mail reader just the ubiquitous web browser
Other?
2: Application Layer

37. Why not just SMTP server on local machine?
“Push not pull” means your PC must be constantly on to accept “push”
2: Application Layer

38. POP3 protocol authorization phase C: list transaction phase, client:
S: +OK POP3 server ready
C: user alice
S: +OK
C: pass hungry
S: +OK user successfully logged on
authorization phase
client commands:
user: declare username
pass: password
server responses
+OK
-ERR
transaction phase, client:
list: list message numbers
retr: retrieve message by number
dele: delete
Quit
C: list
S: 1 498
S: 2 912
S: .
C: retr 1
S: <message 1 contents>
C: dele 1
C: retr 2
C: dele 2
C: quit
S: +OK POP3 server signing off
Try it yourself : telnet pop.enabled.mail-server 110
pobox
2: Application Layer

39. try POP interaction for yourself:
telnet servername 110
see “OK POP3 server ready” reply from server
enter user, pass, list, retr, dele commands
above lets you send get you own without using client (reader)
Trace it – do your password and mail data go in the clear?
Do you configure your mail reader to pop mail every X minutes? Same as announcing your password regularly (unless over Kerberos etc)!
2: Application Layer

40. IMAP
Allows user to set up and maintain multiple folders (for sorting mail) on the remote server
Can get headers for and manipulate messages without downloading them (can even download individual MIME attachments)
Don’t pay cost to download over slow link
Don’t leave them on insecure computers
Stateful protocol - stores per user information about folders and the status of the messages in them
Folder information, actual messages
Seen, Deleted, Answered flags per message
POP stateful too but just username/password
2: Application Layer

41. IMAP con’t
During an IMAP connection, the server transitions between multiple states
Initially non-authenticated
Authenticated
Selected – folder selected and operations on messages permitted
Finally, Logout state
2: Application Layer

42. Authentication in IMAP
Client requests a certain AUTHENTICATION method
C: A001 AUTHENTICATE KERBEROS_V4
If server implements that authentication mechanism then it will authenticate via that method
S: + AmFYig==
C: BAcAQU5EUkVXLkNNVS5FRFUAOCAsho84kL
N3/IJmrMG+25a4DT+nZImJjnTNHJUtxAA+o0KPKfH
EcAFs9a3CL5Oebe/ydHJUwYFd
S: + or//EoAADZI=
C: DiAF5A4gA+oOIALuBkAAmw==
S: A001 OK Kerberos V4 authentication successful
Sever can respond with NO if it does not support that authentication mechanism
S: A001 NO authenticate failure
Try it yourself : telnet pop.enabled.mail-server 110
pobox
2: Application Layer

43. Authentication in IMAP (cont)
Client can try various authentication mechanisms in decreasing order of preference looking for one the server supports
In the worst case, a client may authenticate with plain text login
C: a001 LOGIN SMITH SESAME
S: a001 OK LOGIN completed
2: Application Layer

44. Once authenticated, client can:
SELECT a mailbox
C: A142 SELECT INBOX
S: * 172 EXISTS S: * 1 RECENT
S: * OK [UNSEEN 12] Message 12 is first unseen
S: * OK [UIDVALIDITY ] UIDs valid
S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
S: * OK [PERMANENTFLAGS (\Deleted \Seen \*)] Limited
S: A142 OK [READ-WRITE] SELECT completed
CREATE, RENAME or DELETE mailboxes
FETCH messages from a mailbox
SEARCH through messages
APPEND messages to a mailbox
2: Application Layer

45. Pop vs IMAP Similarities Differences
Mail delivered to a shared, constantly connected server
New mail accessible anywhere in network on a variety of platforms
For access only, Need SMTP to send mail
Differences
POP simpler and more established (more clients and servers that support it)
IMAP keeps more state and has more features; POP uses less server resources
IMAP = prioritize download time; POP = shorter overall connection time
2: Application Layer

46. Mailservers play different roles
Be careful to identify the role played by a specific mailserver
Incoming mail to a domain
May refuse to take RCPT TO other domains
Allow users to get their incoming mail
May not do SMTP at all
Typically do require authentication
Outgoing mail from a domain
May require authentication from valid user
May require that MAIL FROM match domain
Can have one mail server do it all and can have no checking OR can divy up into specific roles and enforce those
2: Application Layer

47. Gmail and Clarkson?
Use Gmail to send/read mail? Forward your Clarkson to Gmail?
Ok what happens when send to your Clarkson address
Use HTTP to send to Gmail servers
SMTP Gmail servers to Clarkson
Stored locally at Clarkson
Gmail account uses POP to retrieve
Use HTTP to read over Gmail
2: Application Layer

48. Sender Policy Framework (SPF)
RFC 4408
Allows the owner of a domain to specify their mail sending policy
E.g. they can specify which mail servers they use to send mail *from* their domain
SPF record in DNS
SPF query tool:
2: Application Layer

49. 2: Application Layer

50. nslookup
set query=txt
clarkson.edu
v=spf1 mx a:mymail.clarkson.edu a:lists.clarkson.edu a:janus.clarkson.edu a:web2.clarkson.edu a:milhouse.clarkson.edu a:outbound.clarkson.edu a:bulkmail.clarkson.edu
2: Application Layer

51. More Application Level Protocols?
Telnet, Rlogin, SNMP (Simple Network Management Protocol), Instant Messenger (AIM), DHCP (BOOTP) , RPC, NFS, X,Finger, Whois,IDENT…………………..
Any ones that you really wish we’d cover?
You now know how to investigate any of these on your own
RFCs for open protocols, Run apps and trace them, Get client/server source,…
It would be a lot more fun to learn more than application level protocols though, right?
2: Application Layer

52. Roadmap
We’ve looked at a bunch of application level protocols (HTTP, DNS, FTP, SMTP, POP, IMAP,, ..) – Lessons?
Many were human readable – why?
High level examples of protocol layering (SMTP, HTTP)
Some ran on TCP, some on UDP, one on both – why?
Used telnet/nslookup to interact with these protocols more directly
Traced them (What went in clear text?!)
Food-for-thought: Design a “Telephone Protocol” other choices?
Next.. How would we implement an application level protocol ourselves?
Socket API
After that down to transport layer
2: Application Layer

53. Outtakes
2: Application Layer

54. Thanks to Jeffrey Vinocur (NNTP presentation, Spring 2002)
Network News
Thanks to Jeffrey Vinocur (NNTP presentation, Spring 2002)
2: Application Layer

55. What is Usenet? Reading/posting to Usenet newsgroups
Conceptually: a semi-organized collection of forums (“newsgroups”) for public discussion
Technically: a system for distributing -like messages
Most people here have used newsgroups. We’re going to focus today on the networks portion of Usenet. Essentially, all of the content of usenet is made possible by the underlying layer, which we can view as an enormously massive distributed database.
JNM: Keyed by unique message id
2: Application Layer

56. Usenet Messages
Format: like , but a bit stricter and with some extra headers (e.g., Newsgroups) – we don’t care about this today, except for two important headers
Message-ID: unlike , every message truly needs to have a globally unique identifier
Path: we’ll see this header later
The content of the messages we also don’t care about today, except for two headers that we’ll need later. The first is a unique identifier for each message, and the second is a “trail” for where the message has been (like the Received: headers in messages).
How message-id generated
2: Application Layer

57. From: meowbot@meowing.net (A Meowbot) Newsgroups: alt.dev.null
Path: news.litech.org!lnsnews.lns.cornell.edu!paradoxa.ogoense.net!not-for-meow
From: (A Meowbot)
Newsgroups: alt.dev.null
Subject: Why?
Date: Sun, 27 Jan :25: (UTC)
Organization: a tyranny of meowing fascist censor cabalists
Lines: 4
Approved: nope.
Message-ID:
X-Trace: paradoxa.ogoense.net (27 Jan
:25:52 GMT)
X-Complaints-To:
X-Meow: Wouf
Mail-Copies-To: nobody
X-No-Repost: yes
Xref: news.litech.org alt.dev.null:492
Because we like you. --
Meow
Here’s an example message…a lot like , really.
What is not-for-meow – the originating hostname?
2: Application Layer

58. Network Topology Users connect to a local site
Each site may have several servers for better throughput
Sites are connected by (manually-requested and -configured) peering links to other sites
Major sites have hundreds of peers
So, how is this worldwide network arranged? Each user connects to a local server, usually run by the same people who are providing and other services. Each server is connected to a few other servers, perhaps as many as a few hundred, known as “peers”. This connections are made manually, mostly by people sending out to a remote site and asking nicely.
2: Application Layer

59. So I post…then what?
The goal is for every article to make it to every server in the world – the “floodfill” model
This can be as fast as a few seconds or as long as a few days (normally a few hours)
When a new article appears on the network, the idea is that it flows along the links between peers and eventually reaches every corner of the world.
2: Application Layer

60. Serious bandwidth Credit: CAIDA (1999) 2: Application Layer
This wouldn’t be hard, except that the amount of data involved is enormous -- that green stripe in the middle of the screen is using something like ten percent of all of the traffic in the entire Internet.
JNM: Imagine if each web server got a copy of all the web pages in the world everyday (or
Even all the new web pages). Web servers for small sites rarely get that many hits but news servers even for small sites get a large flow of news.
2: Application Layer

61. An article arrives… This can be either a new post from a user or an article being “fed” from a peering server.
The server’s “name” added to the Path header (history of where the article has been)
The server stores the article so users can read it
For each of the server’s peers, determine if the peer has seen the article already (first check for peer’s name in Path header, then ask the peer about the Message-ID)
Send the article to peers who do not have it
How does all of this traffic know where to go? Every article keeps a “history” of where it’s been already -- this is the Path header. As soon as an article shows up, the server adds its name to this history. Then the server tries to figure out where to send the article. Any server listed in the Path header has definitely seen this article already. Otherwise, we have to query the remote server to find out if it has already seen this article from somebody else.
Let’s see an example.
2: Application Layer

62. Path headers and Message-IDs
Let’s trace an article. The initial component (at the end!) of the Path header marks the original posting server; then the originating server adds its name:
Path: paradoxa.ogoense.net!not-for-meow
Then this article gets fed to a another server and then add their hostname:
Path: lnsnews.lns.cornell.edu!paradoxa.ogoense.net!not-for-meow
And then it gets fed to another server…
Path: news.litech.org!lnsnews.lns.cornell.edu!paradoxa.ogoense.net!not-for-meow
First news server to get it was not-for-meow..then paradoxa. Then lnsnews.lns.cornell..then
To news.litech.org
If we already received another the message from another source we won’t
Ask to receive it
Try one
telnet newserver nntp
telnet newsstand.cit.cornell.edu
help
mode reader
list
Authinfo user <user>
Authinfo passwd <passwd>
group <newsgrsoup name>
Group cornell.class.cs519
Head <article #>
Article <article #>
How to list all subject lines?
quit
2: Application Layer

63. Usenet, 1980 reed phs \ / \ uok---duke-unc / \ research vax135 |
Credit:
Mark Horton
reed phs
\ / \
uok---duke-unc
/ \
research vax135 |
ucbvax
2: Application Layer

64. Usenet, 1981 Credit: ucbvax!mark pdp (Misc) ! (NC) (Misc)
decvax sii reed phs--unc--grumpy duke34 utzoo cincy teklabs
! ! ! ! ! ! ! ! ! !
! !
! ! !
! duke !
! !
! ! ! ! ! !
ucbopt ! hocsr--mhtsa----research mh135a harpo-----chico
: ! ! ! !
ucbcory ! ! eagle ihnss vax135 (Bell Labs)
(UCB) : ! ! ! ! !
ucbvax
: @ ! ! ! (Silicon Valley)
(UCSD) sdcsvax ! menlo70--hao
: @ sdcattb ! ! !
ucsfcgl sytek sri-unix
@ phonlab-----+
cca-unix sdcarl
!- Uucp links
: Berknet links
@ Arpanet links
2: Application Layer

65. Usenet, 1993
Credit:
Brian Reid
2: Application Layer

66. Usenet today 1.4 million articles daily ~ 360 GB daily
Credit:
Karl L. Swartz
1.4 million articles daily
~ 360 GB daily
Over a 100 Mbit/sec link is > 8 hours!
How long is that over a 100Mbit/sec link = over 8 hours
Amazing that this is still done …
USENET historically significanct and still supported – for how long?
> Someone asked a question about the "articles per week" graph for
> Do you know what the periodic dip early in each year is?
Christmas!
You'll notice there's a lull in mid-year also; Usenet was (and is) a
significantly academic-core community. So many, many groups get very
quiet over winter break. In fact, news.announce.newgroups (where the
group-creation process for Big-8 occurs) has an official Christmas
vacation so that no votes occur during a three-week period around that time.
2: Application Layer

67. – Professor Gene Spafford, Purdue University
Usenet is like a herd of performing elephants with diarrhea – massive, difficult to redirect, awe-inspiring, entertaining, and a source of mind-boggling amounts of excrement when you least expect it.
– Professor Gene Spafford, Purdue University
2: Application Layer

68. Multimedia Applications
2: Application Layer

69. Multimedia Applications
Audio/video conferencing, streaming audio, etc.
On-demand playback: could download before beginning playback; could support rewind, fast forward etc.; start-up time and RTT not very important
Live transmission: usually broadcast from one source like TV or radio; much like on demand; no rewind or fast forward; more sensitive to delay (how close to live?)
Conferencing: interactive, start-up time and RTT matter alot
Examples: vic (video conferencing), vat (audio conferencing), RealAudio, Quicktime, WindowsMedia
2: Application Layer

70. Requirements of multimedia
Several methods for compressing and encoding voice/video; sender and receiver negotiate
Ability to display stream (at degraded quality) with lost packets
Ability to specify the timing requirements between packets of related data for smooth playback
Frame boundary indication
Synchronization of related audio and video streams
No retransmission of lost packets
2: Application Layer

71. Real-time Transport Protocol (RTP)
TCP overhead to high; UDP not good enough
Initially, each application had its own protocol, implementing only those parts of TCP it really needed on top of UDP
RTP offers generalized real time transport services
Thin protocol; Runs on top of UDP
Implements functionality commonly needed by multimedia applications - timing reconstruction, loss detection, security and content identification
RFC 1889
2: Application Layer

72. Realtime Transport (?) Protocol
Is this an application level protocol or a transport protocol?
Done at application level
If TCP implemented at application level (good project ), does that make it an application level protocol or a transport level protocol?
Where is the right place to put these features?
2: Application Layer

73. Real-time Streaming Protocol (RTSP )
Network “Remote Control”
Like FTP has data channel and control channel; RTSP is the control channel for streaming audio/video
Not used to deliver data; often uses RTP for the data portion
Establishes and controls audio and video delivery
Single or multiple audio/video streams (time synchronization if desired)
Live feeds or stored clips
Industry consortium announced in 1996 – since then?
Mostly development continued on proprietary versions: Real Network’s (originally Progressive Networks) RealMedia, RealAudio and RealPlayer , Quicktime, WindowsMedia???
2: Application Layer

74. RTSP Requests DESCRIBE – description of presentation
OPTIONS - get supported methods; capability announcements
SETUP – establish a new session
PLAY – start playback/streaming; reposition
ANNOUNCE – change description of presentation
RECORD – start recording
REDIRECT – redirect client to a new server; for load balancing
PAUSE –stop delivery but keep state
TEARDOWN – stop delivery, remove state
2: Application Layer

75. Trying RTSP telnet servername 554
C: DESCRIBE rtsp://streamserver/rafile.rm RTSP/1.0\n\n
S: RTSP/
2: Application Layer

76. Trying RTSP (2)
C: SETUP rtsp://audio.example.com/twister/audio RTSP/1.0
Transport: rtp/udp; compression; port=3056; mode=PLAY
S: RTSP/ OK
Session 4231
C: PLAY rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0
Session: 4231
Range: npt=0-
C: PAUSE rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0
Range: npt=37
C: TEARDOWN rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0
S: OK
2: Application Layer

77. RTSP vs HTTP RTSP actually derived from HTTP
Avoid mistakes (like always specify full URI)
More methods of course
RTSP server needs to maintain state from SETUP to control PLAY command; HTTP server is stateless (uses cookies to trick client into remembering it)
Data can be delivered in or out of band with RTSP; HTTP data delivered in band
RTSP is a symmetric protocol (client and server can both issue requests); HTTP client issues requests
Ex. server can announce new available streams (audio from a new participant in a conference)
2: Application Layer

78. Session Description Formats
Format for describing the number and sources for all streams in a presentation
May offer alternatives
Different audio channels in various languages
Different quality of audio/video for various BW connections
Specify timing requirements between various streams
Examples: SDF, SDP
2: Application Layer

79. SDP example
session (v 0)(o mhandley IN IP )
(s Sd seminar)(i A seminar on the session description protocol) (u
(e (Mark Handley))
(c IN IP /127)(t )
(a recvonly) (all (media (m audio 3456 VAT PCMU))
(media (m video 2232 RTP H261))
(media (m whiteboard UDP WB)(orient portrait)) ))
Session description v= (protocol version) o= (owner/creator and session identifier). s= (session name) i=* (session information) u=* (URI of description) e=* ( address) p=* (phone number) c=* (connection information - not required if included in all media) b=* (bandwidth information) One or more time descriptions (see below) z=* (time zone adjustments) k=* (encryption key) a=* (zero or more session attribute lines) Zero or more media descriptions (see below) Time description t= (time the session is active) r=* (zero or more repeat times) Media description m= (media name and transport address) i=* (media title) c=* (connection information - optional if included at session-level) b=* (bandwidth information) k=* (encryption key) a=* (zero or more media attribute lines)
From:
2: Application Layer

80. From URL in web page to streaming audio/video
<EMBED SRC=“ TYPE = “application/x-audio”>
HTTP gets session or presentation description file ( not part of RTSP) from a web server
Presentation Description indicates RTSP server to contact
Note: RTSP is presentation description format neutral
RTSP sets up a stream to control delivery
RTSP used to indicate server that will actually stream the data and by what protocol
Ex. specify an RTP server to deliver the data
Note: possibly 3 servers involved!
2: Application Layer

81. Alternative: HTTP Streaming
Many sites simply send audio and video over HTTP
When object arrives will be opened by appropriate application just like Doc files or PDF files
Estimate when it is safe to begin playback without the playback outpacing the download
Download mode and a limited streaming mode can be supported this way
Rewind? Fast forward?
Can support full streaming if delays ok
2: Application Layer

82. Audio and Video on the Internet
Quicktime
HTTP streaming or RTP and RTSP
RealServer
one control channel: RTSP over TCP
one data channel: PNA (Progressive Networks Audio) over UDP (?)
Also can use RTSP to interleave data and control onto one TCP channel (common configuration)
WindowsMedia
Similar to RealPlayer: control channel and data channel
Harder to find details of protocols (surprise, surprise)
But formats are not compatible (surprise, surprise)
2: Application Layer

83. viruses
Often attachments which once opened run with the users full privileges and corrupt the system on which mail is read
Viruses tend to target Windows as it is the platform used by the majority of people
2: Application Layer

84. More FTP Details about TYPE and MODE commands
REST – restart at specified data checkpoint
2: Application Layer

85. FTP minimum requirements
MINIMUM IMPLEMENTATION In order to make FTP workable without needless error messages, the following minimum implementation is required for all servers: TYPE - ASCII Non-print MODE - Stream STRUCTURE - File, Record COMMANDS - USER, QUIT, PORT, TYPE, MODE, STRU, for the default values RETR, STOR, NOOP. The default values for transfer parameters are: TYPE - ASCII Non-print MODE - Stream STRU - File All hosts must accept the above as the standard defaults.
2: Application Layer

86. telnet source
We’ve been using telnet to examine various application protocols
telnet basically opens a TCP connection to the specified port
Getting the telnet source and examining it would be a good exercise
2: Application Layer

87. Real Time Control Protocol (RTCP)
Real-time conferencing of groups of any size within an internet.
Provides source identification, quality-of-service feedback from receivers to the multicast group, synchronization of different media streams
More info on RTCP or drop?
2: Application Layer

88. ReSerVation Protocol (RSVP)
Host can use to request specific quality of service from the network for a specific flow of data
Must be processed and honored at each router to be meaningful
Works much like dynamic routing protocols; messages processed by applications at user level
If a flow is “admitted” then resource reservation decisions will be made in form of packet classifier and schedulers that will prioritize the use of resources
Cisco’s take on RSVP
More info on this or drop?
2: Application Layer

89. Multiple recipients
When you send mail to your outgoing mail server, transfer one copy of message regardless of how many recipients
Mail servers could play the same trick
Look at RCPT to list
If more than one recipient per destination mail server then transfer just one mail
Could also send one copy per recipient
Recommended configuration?
2: Application Layer