1. 1 Lecture #3 Electronic Mail Protocols HAIT Summer 2005 Shimrit Tzur-David

2. 2 Message Transfer The message transfer system is concerned with relaying messages from the originator to the recipient. The simplest way to do this is to establish a transport connection from the source machine to the destination machine and then just transfer the message. After examining how this is normally done, we will examine some situations in which this does not work and what can be done about them.

3. 3 Mail Servers mailbox contains incoming messages for user message queue of outgoing (to be sent) mail messages SMTP protocol between mail servers to send email messages –client: sending mail server –“server”: receiving mail server mail server user agent user agent user agent mail server user agent user agent mail server user agent SMTP

4. 4 SMTP— The Simple Mail Transfer Protocol Uses TCP to reliably transfer email message from client to server, port 25 Direct transfer: sending server to receiving server Three phases of transfer –handshaking (greeting) –transfer of messages –closure Command/response interaction –commands: ASCII text –response: status code and phrase

5. 5 Sequence of Events After establishing the TCP connection to port 25, the sending machine, operating as the client, waits for the receiving machine, operating as the server, to talk first. The server sends a line of text giving its identity and telling whether it is prepared to receive mail. If it is not, the client releases the connection and tries again later. If the server is willing to accept e-mail, the client announces whom the e-mail is coming from and whom it is going to. If such a recipient exists at the destination, the server gives the client the go-ahead to send the message. The client sends the message and the server acknowledges it. When all the e-mail has been exchanged in both directions, the connection is released.

6. 6 Scenario: Alice sends message to Bob 4) SMTP client sends Alice’s message over the TCP connection 5) Bob’s mail server places the message in Bob’s mailbox 6) Bob invokes his user agent to read message 1) Alice uses UA to compose message “to” bob@someschool.edu 2) Alice’s UA sends message to her mail server; message placed in message queue 3) Client side of SMTP opens TCP connection with Bob’s mail server user agent mail server mail server user agent 1 2 3 4 5 6

7. 7 Sample SMTP interaction S: 220 hamburger.edu SMTP service ready C: HELO crepes.fr S: 250 Hello crepes.fr, pleased to meet you C: MAIL FROM: S: 250 alice@crepes.fr... Sender ok C: RCPT TO: S: 250 bob@hamburger.edu... Recipient ok C: DATA S: 354 Enter mail, end with "." on a line by itself C: Do you like ketchup? C: How about pickles? C:. S: 250 Message accepted for delivery C: QUIT S: 221 hamburger.edu closing connection

8. 8 Try SMTP Interaction for Yourself telnet servername 25 see 220 reply from server enter HELO, MAIL FROM, RCPT TO, DATA, QUIT commands The commands above let you send email without using email client (reader)

9. 9 Mail Message Format SMTP: protocol for exchanging email msgs Format: header lines, e.g., –To: –From: –Subject: different from SMTP commands! body –the “message”, ASCII characters only header body blank line

10. 10 Message Format: Multimedia Extensions MIME: multimedia mail extension. Additional lines in msg header declare MIME content type From: alice@crepes.fr To: bob@hamburger.edu Subject: Picture of yummy crepe. MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Type: image/jpeg base64 encoded data....................................base64 encoded data multimedia data type, subtype, parameter declaration method used to encode data MIME version encoded data

11. 11 Weak Spots of SMTP Even though the SMTP protocol is completely well defined, a few problems can still arise. –Message length - some older implementations cannot handle messages exceeding 64 KB. –Timeouts - if the client and server have different timeouts, one of them may give up while the other is still busy, unexpectedly terminating the connection.

12. 12 ESMTP To get around some of these problems, extended SMTP (ESMTP) has been defined. Clients wanting to use it should send an EHLO message instead of HELO initially. If this is rejected, then the server is a regular SMTP server, and the client should proceed in the usual way. If the EHLO is accepted, then new commands and parameters are allowed.

13. 13 Mail access protocols SMTP: delivery/storage to receiver’s server Mail access protocol: retrieval from server –POP: Post Office Protocol, port TCP/110 authorization (agent server) and download –IMAP: Internet Mail Access Protocol, port 150 more features (more complex) manipulation of stored messages on server –HTTP: Hotmail, Yahoo! Mail, etc. user agent sender’s mail server user agent SMTP access protocol receiver’s mail server

14. 14 POP3 Situation where both sender and receiver having a permanent connection to the Internet Situation where the sender is on-line but the receiver is not

15. 15 POP3 Protocol POP3 begins when the user starts the mail reader. The mail reader calls up the ISP and establishes a TCP connection with the message transfer agent at port 110. Once the connection has been established, the POP3 protocol goes through three states in sequence: –Authorization - deals with having the user log in. –Transactions - deals with the user collecting the e-mails and, for example, marking them for deletion. –Update - causes the e-mails to be deleted.

16. 16 POP3 Protocol – Cont. authorization phase client commands: –user: declare username –pass: password server responses –+OK –-ERR transaction phase, client: list: list message numbers retr: retrieve message by number dele: delete quit C: list S: 1 498 S: 2 912 S:. C: retr 1 S: S:. C: dele 1 C: retr 2 S: S:. C: dele 2 C: quit S: +OK POP3 server signing off S: +OK POP3 server ready C: user bob S: +OK C: pass hungry S: +OK user successfully logged on

17. 17 Gentle Point POP3 protocol supports the ability to download a specific message or set of messages and leave them on the server. Most e-mail programs just download everything and empty the mailbox. This behavior means that in practice, the only copy is on the user's hard disk. If that crashes, all e-mail may be lost permanently.

18. 18 Sequence of Events 1.Alice creates a message for Bob using some e-mail program (i.e., user agent) and clicks on an icon to send it. 2.The e-mail program hands the message over to the message transfer agent on Alice's host. 3.The message transfer agent sees that it is directed to bob@xyz.com so it uses DNS to look up the MX record for xyz.com. This query returns the DNS name of xyz.com's mail server. 4.The message transfer agent now looks up for the IP address of this machine using DNS again. 5.It then establishes a TCP connection to the SMTP server on port 25 of this machine. Using an SMTP command sequence analogous, it transfers the message to Bob's mailbox and breaks the TCP connection.

19. 19 Sequence of Events – Cont. 6.Bob boots up his PC, connects to his ISP, and starts his e-mail program. 7.The e-mail program establishes a TCP connection to the POP3 server at port 110 of the ISP's mail server machine. 8.Bob's e-mail program runs the POP3 protocol to fetch the contents of the mailbox to his hard disk. 9.Once all the e-mail has been transferred, the TCP connection is released. Why?

20. 20 IMAP – Internet Message Access Protocol What happens if a user opens his mailbox in more than one place? Many people have a single e-mail account at work or school and want to access it from work, from their home PC, from their laptop when on business trips, etc. While POP3 allows this, since it normally downloads all stored messages at each contact, the result is that the user's e-mail quickly gets spread over multiple machines, more or less at random, some of them not even the user's.

21. 21 IMAP Policy IMAP assumes that all the e-mail will remain on the server indefinitely in multiple mailboxes. IMAP provides extensive mechanisms for reading messages or even parts of messages, a feature useful when using a slow modem to read the text part of a multipart message with large audio and video attachments. Since the working assumption is that messages will not be transferred to the user's computer for permanent storage, IMAP provides mechanisms for creating, destroying, and manipulating multiple mailboxes on the server. In this way a user can maintain a mailbox for each correspondent and move messages there from the inbox after they have been read.

22. 22 A comparison of POP3 and IMAP FeaturePOP3IMAP Where is emails stored? Where is emails read? Connect time required Use of server resources Multiple mailboxes User control over downloading Partial message downloads Are disk quotas a problem? Simple to implement Keeps user state User’s PCServer Off-line On-line LittleMuch MinimalExtensive NoYes LittleGreat NoYes NoCould be YesNo YesNo

23. 23 More about POP3 Previous example uses “download and delete” mode. Bob cannot re-read e-mail if he changes client. “Download-and-keep”: copies of messages on different clients

24. 24 Delivery Features Many systems provide additional features for processing of incoming e-mail. For example: the ability to set up filters. –These are rules that are checked when e-mail comes in or when the user agent is started. Each rule specifies a condition and an action. –Rule examples - any message received from the boss goes to mailbox number 1, any message from a select group of friends goes to mailbox number 2, and any message containing certain objectionable words in the Subject line is discarded without comment.

25. 25 Delivery Features – Cont. Filter - automatically categorizes incoming e-mail as either important or spam (junk e-mail) and stores each message in the corresponding mailbox. Such filters typically work by first checking to see if the source is a known spammer. They usually examine the subject line. If hundreds of users have just received a message with the same subject line, it is probably spam. Other techniques are also used for spam detection. Forward incoming e-mail to a different address. Installation of a vacation daemon.

26. 26 Web-Mail Some Web sites, for example, Hotmail and Yahoo, provide e- mail service. They have normal message transfer agents listening to port 25 for incoming SMTP connections. To contact, say, Hotmail, you need to establish a TCP connection over which SMTP commands can be sent in the usual way. When the user goes to the e-mail Web page, a form is presented in which the user is asked for a login name and password. When the user clicks on Sign In, the login name and password are sent to the server, which then validates them. The server finds the user's mailbox and builds a listing page, only formatted as a Web page in HTML. The Web page is then sent to the browser for display.