1. Mail Services

2. Pre-survey: How many server/client programs (or services) are needed to send and receive email?1 2 3
4 or more

3. Mail Services Three major mail service protocols:
Simple Mail Transfer Protocol
SMTP
Post Office Protocol
POP3
Internet Mail Access Protocol
IMAP or IMAP4

4. Which one to use? Outbound Mail Inbound Mail (clients)
SMTP for outbound
Port 25 or 2525
Server/repository
Inbound Mail (clients)
POP3 for inbound
Port 110
IMAP for inbound
Port 143

5. What is SMTP?
Simple Mail Transfer Protocol (SMTP) is the standard protocol for sending s across the Internet
SMTP uses TCP port 25 or 2525
Sometimes you may have problems sending messages
ISP may have closed port 25
To determine the SMTP server for a given domain name
MX (Mail eXchange) DNS record is used

6. IMAP or POP3? Both are protocols to receive email
Main differences between IMAP and POP3:
POP3 protocol assumes there is only one client (computer terminal) that ever connects to the mailbox
Connect
Get/send mail
Disconnect
Read and write while offline
IMAP protocol allows simultaneous access by multiple clients (computer terminals)
Allows the same user on the clients
IMAP is also suitable if the mailbox is used by multiple users

7. Material from: http://en.wikipedia.org/wiki/Smtp
SMTP

8. SMTP Simple Mail Transfer Protocol
The de facto standard for transmissions across the Internet
Defined in RFC 821 (STD 10)
amended by RFC 1123 (STD 3) chapter 5.
The protocol used today is also known as ESMTP
Defined in RFC 2821

9. SMTP Relatively simple text-based protocol Client-server protocol
One or more recipients of a message are specified
Most cases verified to exist
The message text is transferred
Client-server protocol
The client transmits an message to the server
Either
an end-user's client, a.k.a. MUA (Mail User Agent), - or -
a relaying server's MTA (Mail Transfer Agents)
Can act as an SMTP client

10. SMTP
An client knows the outgoing mail SMTP server from its configuration
A relaying server typically determines which SMTP server to connect to by looking up the MX (Mail eXchange) DNS record for each recipient's domain name
The part of the address to the right of the at sign
Conformant MTAs (not all) fall back to a simple A record in the case of no MX
A Record: address record
Some current mail transfer agents will also use SRV records
SRV Records: SeRVice records
More general form of MX
These are not widely adopted
Relaying servers can also be configured to use a smart host

11. SMTP The SMTP client initiates a TCP connection
Server's port 25
Unless overridden by configuration
Quite easy to test an SMTP server using telnet
see following example

12. SMTP SMTP is a "push" protocol
Does not allow one to "pull" messages from a remote server on demand
That is it sends only
To do a pull (i.e. receive) a mail client must use POP3 or IMAP
Another SMTP server can trigger a delivery in SMTP
Using ETRN

13. Outgoing mail SMTP server
client requires the name or the IP address of an SMTP server as part of its configuration
Server will deliver messages on behalf of the user
Setting allows for various policies and network designs
End users connected to the Internet can use the services of an provider that is not necessarily the same as their connection provider
Network topology, or the location of a client within a network or outside of a network, is no longer a limiting factor for submission or delivery
Modern SMTP servers typically use a client's credentials (authentication) rather than a client's location (IP address), to determine whether it is eligible to relay

14. Outgoing mail SMTP server
Can use either TCP port 25 (SMTP) or port 587 (Submission) for relaying outbound mail to a mail server
Established by RFC 2476
Many servers support both
Some servers still support port 465 for legacy secure SMTP
Preferable to use encryption on standard ports according to RFC 2487
Some servers are setup to reject all relaying on port 25
Valid users authenticating on port 587 are allowed to relay mail to any valid address.
Server that relays all for all destinations for all clients connecting to port 25 is known as an open relay
Generally considered a bad practice worthy of blacklisting

15. Sample communications
After establishing a connection between the sender (the client) and the receiver (the server), the following page shows a legal SMTP session
In the following conversation
everything sent by the client is prefaced with C:
everything sent by the server is prefaced with S:
On most computer systems, a connection can be established using the telnet command on the client machine, for example:
telnet 25
Opens a TCP connection from the sending machine to the MTA listening on port 25 on host

16. Sample communication (after connect to server)
S: ESMTP Postfix
C: HELO mydomain.com
S: 250 Hello mydomain.com
C: MAIL
S: 250 Ok
C: RCPT
C: DATA
S: 354 End data with <CR><LF>.<CR><LF>
C: Subject: test message
C: From:
C: To: C:
C: Hello,
C: This is a test.
C: Goodbye.
C: .
S: 250 Ok: queued as 12345
C: QUIT
S: 221 Bye

17. Sample communications
Note: the data the client sends in the HELO and MAIL FROM commands can be retrieved in additional headers that the server adds to the message:
Received and Return-Path respectively.
Optional: (and not shown above) nearly all clients ask the server which SMTP extensions the server supports
Uses the EHLO greeting to invoke Extended SMTP (ESMTP)
These clients use HELO only if the server does not respond to EHLO

18. Sample communications
Contemporary clients will use the ESMTP extension keyword SIZE to inquire of the server the maximum message size that will be accepted
Older clients and servers
Try to transfer huge messages that will be rejected
After wasting the network resources
Including a lot of connect time to dialup ISPs
paid by the minute

19. Sample communications
Users can determine the maximum size accepted by ESMTP servers
For the sending of giant files or sending with older clients
The user telnets as above, but substitutes "EHLO mydomain.com" for the HELO command line:
S: 220-serverdomain.com ESMTP {postfix version and date}
S: 220 NO UCE. {etc., terms of service}
C: EHLO mydomain.com
S: 250-serverdomain.com Hello mydomain.com [ ]
S: 250-SIZE
S: 250-PIPELINING
S: 250 HELP
Here, serverdomain.com declares that it:
Will accept a fixed maximum message size no larger than 14,680,064 octets
Depending on the server's actual resource usage, it may be currently unable to accept a message this large
Can pipeline
Can help

20. Sample communications
In the simplest case, an ESMTP server will declare a maximum SIZE with only the EHLO user interaction.
If no number appears after the SIZE keyword, or if the current message limit must be exactly determined:
User can further interact by simulating the ESMTP header of a message with an estimated size.
See External Link RFC 1870

21. Material From: http://en.wikipedia.org/wiki/Post_Office_Protocol
POP

22. POP Local e-mail clients use the Post Office Protocol version 3 (POP3)
An application-layer Internet standard protocol
Retrieves from a remote server over a TCP/IP connection
Many subscribers to individual Internet service provider accounts access their with client software that uses POP3

23. POP Overview POP3 has made earlier versions of the protocol obsolete
POP (formally referred to POP1 and POP2)
Now, the term POP almost always means POP3 in the context of protocols

24. POP Overview
Design of POP3 and its procedures supports end-users with intermittent connections
(e.g. dial-up connection)
Allows users to retrieve when connected
View and manipulate the retrieved messages without needing to stay connected
Although most clients have an option to leave mail on server, clients using POP3 generally:
Connect
Retrieve all messages
Store them on the user's PC as new messages
Delete them from the server
Disconnect

25. POP Overview
Clients with a leave mail on server option generally use POP3 UIDL command
Unique IDentification Listing
Most POP3 commands identify specific messages by their ordinal number on the mail server
Creates a problem for a client intending to leave messages on the server
Message numbers may change from one connection to the server to another
For example if a mailbox contains five messages at last connect, and a different client then deletes message #3, the next connecting user will find the last two messages' numbers decremented by one

26. POP Overview UIDL provides a mechanism to avoid these numbering issues
Server assigns a string of characters as a permanent and unique ID for the message
When a POP3-compatible client connects to the server
Can use the UIDL command to get the current mapping from these message IDs to the ordinal message numbers
Client can then use this mapping to determine which messages it has yet to download
Saves time when downloading
Note: IMAP has a similar mechanism, using a 32-bit UID (Unique IDentifier) that is required to be strictly ascending.

27. POP Overview Advantage of the numeric UID is with large mailboxes
A client can request just the UIDs greater than its previously stored "highest UID"
In POP, the client must fetch the entire UIDL map

28. POP Overview Whether using POP3 or IMAP to retrieve messages:
clients typically use the SMTP_Submit profile of the SMTP protocol to send messages
clients are commonly categorized as either POP or IMAP clients, but in both cases the clients also use SMTP
There are extensions to POP3 that allow some clients to transmit outbound mail via POP3
Known as "XTND XMIT" extensions.
The Qualcomm qpopper and CommuniGate Pro servers and Eudora clients are examples of systems that optionally utilize the XTND XMIT methods of authenticated client-to-server transmission.
MIME serves as the standard for attachments and non-ASCII text in
Although neither POP3 nor SMTP require MIME-formatted , essentially all Internet comes MIME-formatted
POP clients must also understand and use MIME
IMAP, by design, assumes MIME-formatted

29. POP Overview
POP3 originally supported only an unencrypted login mechanism
Plain text transmission of passwords in POP3 still commonly occurs
POP3 currently supports several authentication methods to provide varying levels of protection against illegitimate access to a user's
APOP: Authenticated POP
Uses the MD5 hash function
Attempt to avoid replay attacks and disclosure of the shared secret
Clients implementing APOP include Mozilla Thunderbird, Opera, Eudora, KMail and Novell Evolution
POP3 clients can also support SASL authentication methods via the AUTH extension. MIT Project Athena also produced a Kerberized version

30. POP Overview POP3 works over a TCP/IP connection
Uses TCP on network port 110
clients can encrypt POP3 traffic using TLS or SSL
A TLS or SSL connection is negotiated using the STLS command
Some clients and servers, like Google Gmail, instead use the deprecated alternate-port method, which uses TCP port 995

31. POP Example (APOP) S: <wait for connection on TCP port 110>
C: <open connection>
S: +OK POP3 server ready
C: APOP mrose c4c9334bac560ecc979e58001b3e22fb
S: +OK mrose's maildrop has 2 messages (320 octets)
C: STAT
S: +OK 2 320
C: LIST S: +OK 2 messages (320 octets)
S: 1 120
S: 2 200
S: .
C: RETR 1
S: +OK 120 octets
S: <the POP3 server sends message 1>
C: DELE 1
S: +OK message 1 deleted
C: RETR 2
S: +OK 200 octets
S: <the POP3 server sends message 2>
C: DELE 2
S: +OK message 2 deleted
C: QUIT
S: +OK dewey POP3 server signing off (maildrop empty)
C: <close connection>
S: <wait for next connection>

32. POP Example (no auth)
POP3 servers without the optional APOP command expect you to log in with the USER and PASS commands:
C: USER mrose
S: +OK User accepted
C: PASS mrosepass
S: +OK Pass accepted

33. POP3 Uses port 25 Leaves email on the server by default
Allows the client to disconnect from the server, then work with the individual s
Is more flexible than IMAP

34. Resume 3/16

35. IMAP

36. IMAP Internet Message Access Protocol
Commonly known as IMAP or IMAP4
An application layer Internet protocol
Operates on port 143
Allows a local client to access on a remote server
Previously called Internet Mail Access Protocol, Interactive Mail Access Protocol , and Interim Mail Access Protocol
Current version is IMAP version 4 revision 1
IMAP4rev1: defined by RFC 3501
IMAP4 and POP3 are the two most prevalent Internet standard protocols for retrieval
Virtually all modern clients and servers support both

37. IMAP IMAP supports both connected and disconnected modes of operation
clients using IMAP generally leave messages on the server until the user explicitly deletes them
Allows multiple clients to access the same mailbox
Clients in this case are different hosts
Most clients support both POP3 and IMAP
Few Internet Service Providers (ISPs) support IMAP
IMAP4 offers access to the mail store
Client may store local copies of the messages
Considered to be a temporary cache
Server's store is authoritative

38. IMAP E-mail messages are generally sent to an e-mail server
Stores received messages in the recipient's mailbox
User later retrieves these messages with either a web browser or an client that uses one of a number of retrieval protocols
Some clients and servers preferentially use vendor specific, typically proprietary protocols, most support the Internet standard protocols
SMTP for sending
POP3 and IMAP4 for retrieving
Allows interoperability with other servers and clients

39. IMAP Proprietary protocols:
Microsoft Outlook client
When communicating with an Exchange server
IBM Notes client
When communicating with a Domino server
Both products also support SMTP, POP3, and IMAP4.
Support for the Internet standard protocols
Allows other clients to access these servers
Allows the clients to be used with other servers
E.g. Qualcomm's Eudora or Mozilla Thunderbird

40. IMAP
clients can generally be configured to use either POP3 or IMAP4 to retrieve and in both cases use SMTP for sending.
Most programs can also use Lightweight Directory Access Protocol (LDAP) for directory services
Many (if not most) subscribers to individual ISP accounts access their with client software that uses POP3
IMAP is often used in large networks
For example, a college campus mail system

41. IMAP
IMAP allows users to access new messages instantly on their computers
The mail is stored on the network
With POP3, users either
download the to their computer
access it via the web
Both methods take longer than IMAP
User must either download any new mail or "refresh" the page to see the new messages

42. POP – IMAP Comparisons

43. Overview
Most clients support either POP3 or IMAP to retrieve messages
Few Internet Service Providers (ISPs) support IMAP
Fundamental differences between POP3 and IMAP4:
POP3 offers access to a mail drop
Mail starts on the server
Downloaded to client when accessed
Deleted from server
If the client leaves some or all messages on the server
The client's message store is considered authoritative
IMAP4 offers access to a mail store
Mail stays on the server after accessed
Client may store local copies of the messages
These are considered to be a temporary cache
The server's store is authoritative

44. IMAP Advantages over POP3
Connected and disconnected modes of operation
When using POP3, clients typically connect to the server briefly
Only as long as it takes to download new messages
When using IMAP4, clients often stay connected as long as the user interface is active
Download message content on demand
For users with many or large messages
IMAP4 usage pattern can result in faster response times.

45. IMAP Advantages over POP3
Multiple clients simultaneously connected to the same mailbox
The POP3 protocol requires the currently connected client to be the only client connected to the mailbox.
The IMAP protocol specifically allows simultaneous access by multiple clients
Provides mechanisms for clients to detect changes made to the mailbox by other, concurrently connected, clients

46. IMAP Advantages over POP3
Access to MIME message parts and partial fetch
Nearly all internet is transmitted in MIME format
Allows messages to have a tree structure where
leaf nodes are any of a variety of single part content types
non-leaf nodes are any of a variety of multipart types
The IMAP4 protocol allows clients to
Separately retrieve any of the individual MIME parts
Retrieve portions of either individual parts or the entire message
Mechanisms allow clients to retrieve the text portion of a message without retrieving attached files or to stream content as it is being fetched

47. IMAP Advantages over POP3
Message state information
flags can keep track of message state
For example: whether or not the message has been read, replied to, or deleted
Flags are stored on the server
Different clients accessing the same mailbox at different times can detect state changes made by other clients
POP3 provides no mechanism for clients to store such state information on the server
If a single user accesses a mailbox with two different POP3 clients state information cannot be synchronized between the clients
E.g. whether a message has been accessed
IMAP4 protocol supports both pre-defined system flags and client defined keywords
System flags indicate state information such as whether a message has been read
Keywords allow messages to be given one or more tags whose meaning is up to the client
(not supported by all IMAP servers)
Adding user created tags to messages is an operation supported by some Webmail services, such as Gmail
Generally not using IMAP

48. IMAP Advantages over POP3
Multiple mailboxes on the server
IMAP4 clients can create, rename, and/or delete mailboxes on the server, and move messages between mailboxes
Usually presented to the user as folders
Multiple mailbox support also allows servers to provide access to shared and public folders
Server-side searches
IMAP4 provides a mechanism for a client to ask the server to search for messages meeting a variety of criteria
Avoids requiring clients to download every message in the mailbox in order to perform searches
Built-in extension mechanism
IMAP4 defines an explicit mechanism by which it may be extended
Many extensions to the base protocol have been proposed and are in common use
IMAP2 did not have an extension mechanism
POP3 now has one defined by RFC 2449

49. IMAP Disadvantages v. POP
IMAP remedies many of the shortcomings of POP
 inherently additional complexity
Much of this complexity is compensated for by server-side workarounds
e.g., multiple clients accessing the same mailbox at the same time
Maildir
Database backends

50. Disadvantages of IMAP
If the mail store and searching algorithms on the server are not carefully implemented:
Client can potentially consume large amounts of server resources when searching massive mailboxes

51. Disadvantages of IMAP
IMAP4 clients need to explicitly request new message content
Potentially causing additional delays on slow connections such as those commonly used by mobile devices.
A private proposal, push IMAP, would extend IMAP to implement push by sending the entire message instead of just a notification
However, push IMAP has not been generally accepted
Current IETF work has addressed the problem in other ways

52. Disadvantages of IMAP
Unlike some proprietary protocols which combine sending and retrieval operations, sending a message and saving a copy in a server-side folder with a base-level IMAP client requires transmitting the message content twice, once to SMTP for delivery and a second time to IMAP to store in a sent mail folder
Remedied by a set of extensions defined by the IETF LEMONADE Working Group for mobile devices
POP3 servers don't support server-side folders so clients have no choice but to store sent items on the client
Many IMAP clients can be configured to store sent mail in a client-side folder
Courier Mail Server offers a non-standard method of sending using IMAP by copying an outgoing message to a dedicated outbox folder

53. IMAP
Uses port 25
Allows a client to monitor from multiple workstations at the same time
By default, uses a lot of the client’s local disk storage to hold
Is the best client service since it is a newer protocol

54. POP Overview
In contrast, the newer, more capable Internet Message Access Protocol (IMAP) supports both connected and disconnected modes of operation.
clients using IMAP generally leave messages on the server until the user explicitly deletes them
This and other facets of IMAP operation allow multiple clients to access the same mailbox

55. Sidebar
The mail on the server is kept in files in a directory until read and/or deleted
POP: until “read”
E.g. until downloaded to client
IMAP: until deleted

56. SMTP Authentication
SMTP-AUTH

57. SMTP Authentication SMTP-AUTH
Extension of the Simple Mail Transfer Protocol (SMTP)
Includes an authentication step
Client effectively logs in to the mail server during the process of sending mail
Servers which support SMTP-AUTH can usually be configured to require clients to use this extension, ensuring the true identity of the sender is known.
SMTP-AUTH is defined in RFC 2554

58. SMTP Authentication SMTP-AUTH provides an access control mechanism
Can be used to allow legitimate users to relay mail while denying relay service to unauthorized users, such as spammers
Does not guarantee the authenticity of either the SMTP envelope sender or the "From:" header
Spoofing is possible even with SMTP-AUTH
(When one sender masquerades as someone else)

59. SMTP Authentication
SMTP-AUTH extension also allows one mail server to indicate to another that the sender has been authenticated when relaying mail
This requires the recipient server to trust the sending server
This aspect of SMTP-AUTH is rarely used on the Internet
The recipient of an message cannot tell whether the sender was authenticated
Use of SMTP-AUTH is only a partial solution to spam

60. SMTP Authentication
While SMTP-AUTH is generally a security improvement over unauthenticated SMTP, it can also introduce a weakness
If authenticated users are allowed to submit messages from IP addresses where unauthenticated users are not — that is, if authenticated users are allowed to relay mail — then an attacker who subverts one user's account is then able to use the authenticated server as an open mail relay
In such a configuration every user's password becomes a key to the mail system's security.
Spammers have attacked SMTP-AUTH mail servers by brute forcing common usernames and passwords
A good password policy can effectively prevent such an attack

61. Pop quiz: How many server/client programs (or services) does an IT pro need to send and receive ? 1 2 3

62. Acronyms

63. Acronym summary ESMTP - Extended SMTP ETRN - Extended Turn
IMAP - Internet Mail Access Protocol
MTA - Mail Transfer Agents
MUA - Mail User Agent
MX - Mail eXchange
POP - Post Office Protocol
SMTP - Simple Mail Transfer Protocol
UIDL - Unique IDentification Listing