1. Advanced Sendmail 8.13.3 Part 1
STARTTLS
Advanced Sendmail Part 1
April 25, 2017
September 2005

2. Advanced Sendmail 8.13.3 Part 1
STARTTLS: Objectives
Advanced Sendmail Part 1
Define STARTTLS and the reasons for using it
Describe how the SSL protocol works
Identify the STARTTLS configuration variables
Describe how the Sendmail clients and servers determine whether a server supports STARTTLS
Identify how to set up a private certificate authority (CA)
Describe how to use gen_cf to create a configuration file for STARTTLS
Describe the line that must be added to the access file.
April 25, 2017
September 2005

3. Advanced Sendmail 8.13.3 Part 1
STARTTLS
Advanced Sendmail Part 1
STARTTLS is the SMTP command to "Start Transport Layer Security”; or in other words to turn on Secure Socket Layer (SSL).
Transport Layer Security (TLS) provides authentication (identification), privacy, confidentiality, and integrity for securing a mail transaction.
TLS uses different STARTTLS algorithms for encryption, signing, and message authentication.
To use Sendmail with STARTTLS, you must install the OpenSSL software on your system from
You can use the /usr/newconfig/etc/mail/cf/cf/gen_cf script to generate the sendmail.cf configuration file that supports the STARTTLS feature.
HP only supports STARTTLS if used in conjunction with the access database. HP does not support the use of custom rulesets.
April 25, 2017
September 2005

4. Step by Step SSL Protocol
Advanced Sendmail Part 1
Server
Certificate
Client suggests/requests information from the SSL server
SSL client (e.g. browser)
SSL server
April 25, 2017
September 2005

5. Step by Step SSL Protocol (cont’d)
Advanced Sendmail Part 1
Server
Certificate
Server responds with its digital certificate and encryption preferences. Encryption level negotiation complete.
SSL client (e.g. browser)
SSL server
April 25, 2017
September 2005

6. Step by Step SSL Protocol (cont’d)
Advanced Sendmail Part 1
Server
Certificate
1) Client verifies the server certificate 2) It computes a shared secret session key 3) It encrypts the shared secret session key using the master’s public key and sends it to the server
SSL client (e.g. browser)
SSL server
April 25, 2017
September 2005

7. Step by Step SSL Protocol (cont’d)
Advanced Sendmail Part 1
Server decrypts the master public key and computes shared session key
Server
Certificate
Server
Certificate
SSL client (e.g. browser)
SSL server
Additional Notes
It is important to note that the generated shared secret session key is encrypted before being sent to the server so nobody can “steal” it.
April 25, 2017
September 2005

8. Step by Step SSL Protocol (cont’d)
Advanced Sendmail Part 1
Server
Certificate
Server
Certificate
Client and server exchange encrypted data using the shared session key
SSL client (e.g. browser)
SSL server
April 25, 2017
September 2005

9. STARTTLS configuration variables
Advanced Sendmail Part 1
UseTLS - Enables the TLS handshake in the SMTP transaction.
CERT_DIR - Specifies the directory for storing Sendmail certificates.
CACERT_PATH - Specifies the path that stores the certificates of all the Certificate Authorities known to the Sendmail server.
CACertFile - Specifies the file containing the certificate of the Certificate Authority that issued the certificate of the Sendmail server.
ServerCertFile and ClientCertfile - Refers to the server and client certificate.
ServerkeyFile and Clientkeyfile - Specifies the private keys that correspond to the certificates of the Sendmail server and the Sendmail client.
Additional information about configuration variables
You can set the UseTLS variable to either True or False. For example:
O UseTLS=True
The CERT_DIR variable may be set as follows:
# CA directory
O CACertPath=/etc/mail/certs/
The SERVER_CERTand CLIENT_CERT variables indicate that the certificate of the server is used when acting as a server and when acting as a client. For example:
# Server Cert O ServerCertFile=/etc/mail/certs/oldcert.pem # Client Cert O ClientCertFile=/etc/mail/certs/oldcert.pem
The SERVER_KEYand CLIENT_KEY variables are shown as follows:
For example:
# Server private key O ServerKeyFile=/etc/mail/certs/oldreq.pem # Client private key O ClientKeyFile=/etc/mail/certs/oldreq.pem
April 25, 2017
September 2005

10. When Sendmail is a Server
Advanced Sendmail Part 1
When Sendmail is a Client
Clients issue the EHLO command during an SMTP session to determine whether the server supports STARTTLS.
If the server supports STARTTLS, it will include it in the list of commands the client can issue.
ehlo localhost
250-inet16.india.hp.com Hello localhost [ ], pleased to meet you
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-EXPN
250-VERB
250-8BITMIME 250-SIZE
250-DSN
250-ETRN
250-STARTTLS <<<<<<<< Note
250-DELIVERBY
250 HELP
If the "next" mail server offers "STARTTLS" as one of its allowed commands, our Sendmail (which has been compiled to support STARTTLS) will always accept the offer and issue a STARTTLS command--even if it has no certificates configured!
Jun 14 10:02:33 inet16 sm-mta[11679]: i5E4WXu : <-- STARTTLS
Jun 14 10:02:33 inet16 sm-mta[11679]: i5E4WXu : Ready to start
TLS
Jun 14 10:02:35 inet16 sm-mta[11679]: STARTTLS: cert verify: depth=0
C=IN/ST=Karnataka/L=Bangalore/O=Hewlett-
state=0,reason=self signed certificate
Jun 14 10:02:35 inet16 sm-mta[11679]: STARTTLS=server, get_verify: 18 get_peer:
0x40082b30
Jun 14 10:02:35 inet16 sm-mta[11679]: STARTTLS=server, relay=inet14.india.hp.com
[ ], version=TLSv1/SSLv3, verif y=FAIL, cipher=DHE-RSA-AES256-SHA,
bits=256/256
Jun 14 10:02:35 inet16 sm-mta[11679]: STARTTLS=server, certsubject=/
cert-issuer=/C=IN/ST=Karnataka/L=Bangalore/O=Hewlett-
Jun 14 10:02:35 inet16 sm-mta[11679]: AUTH: available mech=EXTERNAL, allowed
mech=EXTERNAL GSSAPI KERBEROS_V4 DIGEST-MD5 CRAM-MD5
April 25, 2017
September 2005

11. Advanced Sendmail 8.13.3 Part 1
STARTTLS set up
Advanced Sendmail Part 1
Warning: do not use sendmail –bs for testing the STARTTLS set up.
The set up discussed in this class uses a private CA (certificate authority). Private CAs are often used for STARTTLS within a company.
Install OpenSSL on the Sendmail server that will be the private CA. You can download OpenSSL from software.hp.com (security and manageability, OPENSSL).
Setting up a private CA is for example use only. Always consider a commercial CA prior to using a private CA.
Building a private CA is the most complex method of signing certificates.
Evaluate alternatives to choose the approach suitable for the situation.
The cacert.pem file
You will see the cacert.pem file created in this procedure. This file is the certificate created for the certificate authority. This public key will be distributed to the Sendmail systems that recognize this CA, and it will be referenced in their configurations. When you create a private CA to sign certificates for Sendmail, it is common to make this the root CA in the Sendmail configuration.
April 25, 2017
September 2005

12. Set up a private CA on the Sendmail Server
Advanced Sendmail Part 1
# cd /etc/mail
# mkdir certs
# cd certs
# /opt/openssl/misc/CA.sh – newca
(see notes for output on this command)
# mv demoCA CA
# cd CA
# chmod 0700 private
# cp /opt/openssl/openssl.cnf sendmailssl.cnf
vi sendmailssl.cnf
Change dir =./demoCA
To dir =./etc/mail/certs/CA
Step 4 Output
CA certificate filename (or enter to create) "press enter"
Making CA certificate ...
Generating a 1024 bit RSA private key
writing new private key to './demoCA/private/./cakey.pem'
Enter PEM pass phrase: "enter a secret word, record it for later"
Verifying - Enter PEM pass phrase: "" -----
You are about to be asked to enter information that will be incorporated into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank. For some fields there will be a default value, If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:Georgia
Locality Name (eg, city) []:Atlanta
Organization Name (eg, company) [Internet Widgits Pty Ltd]:HP
Organizational Unit Name (eg, section) []:<enter>
Common Name (eg, YOUR name) []:wtecd350.atl.hp.com "Use the system name" Address
Step 5
The directory structure called demoCA contains the files needed for the private CA, the name is changed to something that sounds less temporary, in this case it is changed to CA.
Step 7
The security is tightened on the private directory which holds the certificate authority’s private key.
April 25, 2017
September 2005

13. Create certificate request (csr)
Advanced Sendmail Part 1
# cd /etc/mail
# mkdir certs
# umask 0066
# openssl req –nodes –new –x509 –keyout key.pem –out newcert.pem
(see notes for output on this command)
# openssl x509 –x509toreq –in newcert.pem –signkey key.pem –out csr.pem
FTP/move the csr.pem to the CA host (wtecd350.atl.hp.com) in the /etc/mail/certs/CA directory.
Step 4 Output
Generating a 1024 bit RSA private key
writing new private key to 'key.pem' -----
You are about to be asked to enter information that will be incorporated into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:Georgia
Locality Name (eg, city) []:Atlanta Organization Name (eg, company) [Internet Widgits Pty Ltd]:HP
Organizational Unit Name (eg, section) []:<enter>
Common Name (eg, YOUR name) []:wtecd350.atl.hp.com
Address
Output from Step 5
Getting request Private Key
Generating certificate request
Regarding Step 6, if If you have two servers (rather than one as we do in this example), you create the certificate on both servers and FTP the certificates to the CA. The certificates would need to be signed on the CA and FTP’d back to the original server.
April 25, 2017
September 2005

14. Sign the certificate request (csr)
Advanced Sendmail Part 1
# cd /etc/mail/certs/CA
# openssl ca –config ./sendmailssl.cnf –policy policy_anything –out cert.pem –infiles csr.pem
(see output in notes)
Move/FTP the cert.pem to the /etc/mail/certs directory on the Sendmail host.
Move/FTP the CA/cacert.pem to the /etc/mail/certs/CA directory on the Sendmail host.
# cd /etc/mail
# chmod –R 600 certs
Additional notes
Step 2 Output
Using configuration from ./sendmailssl.cnf Enter pass phrase for /etc/mail/certs/CA/private/cakey.pem: "the secret word when you created CA" Check that the request matches the signature Signature ok Certificate Details:     Serial Number: 1 (0x1)     Validity         Not Before: May 1 04:49: GMT         Not After : May 1 04:49: GMT     Subject:         countryName = US         stateOrProvinceName = Georgia         localityName = Atlanta         organizationName = HP     commonName = wtecd350.atl.hp.com     Address = X509v3 extensions:     X509v3 Basic Constraints:     CA:FALSE     Netscape Comment:     OpenSSL Generated Certificate X509v3 Subject Key Identifier:     B1:C4:FF:E8:55:EE:DB:B8:5C:8E:40:9B:33:74:0C:4F:9A:A7:58:3D X509v3 Authority Key Identifier:         keyid:48:B2:FF:06:F3:47:88:81:8E:85:A4:16:29:BF:8D:73:8B:4E:6D:9D            serial:BE:EA:34:CA:BE:73:80:26
Certificate is to be certified until May 1 04:49: GMT (365 days)
Sign the certificate? [y/n]:y
1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated
April 25, 2017
September 2005

15. Create a configuration file for STARTTLS
Advanced Sendmail Part 1
The following steps are taken to run gen_cf to create a configuration file for STARTTLS.
# cd /usr/newconfig/etc/mail/cf/cf
./gen_cf
4: Security Options
2: STARTTLS
3: Anti-spamming Options
1: Access DB
5: Generate sendmail.cf
Backup the sendmail.cf file:
cp /etc/mail/sendmail.cf /etc/mail/sendmail.cf/bak
Move the sendmail.cf.gen file to the /etc/mail directory
cp /usr/newconfig/etc.mail/cf/cf/sendmail.cf.gen
Output from Step 2e
Do you want to continue and build the configuration file?
Press any to continue or [n/N] to unset all options
NOTE: To enable Starttls the sendmail.cf file needs to be changed.
To enable Starttls the following options need to be enabled: UseTLS, CACertFile, ServerCertFile, ServerKeyFile, ClientCertFile and ClientKeyFile.
The current value of these variables in the sendmail.cf are:
UseTLS : Commented
CACertFile : Commented
ServerCertFile : Commented
ServerKeyFile : Commented
ClientCertFile : Commented
ClientKeyFile : Commented
April 25, 2017
September 2005

16. Sendmail.cf file configuration additions and changes
Advanced Sendmail Part 1
Important note: There is a bug with the /dev/random file.
If you have a /dev/random file and a /dev/urandom file add the following to the sendmail.cf:
O RandFile=egd:/dev/random
Modify the following parameter:
O UseTLS=True
April 25, 2017
September 2005

17. Allow relaying based on the CA - configuration file changes
Advanced Sendmail Part 1
To allow relaying based on the CA the following changes need to be made to the sendmail.cf configuration file STARTTLS parameters.
O UseTLS=True
O CACertPath=/etc/mail/certs
O CACertFile=/etc/mail/certs/CA/cacert.pem
O ServerCertFile=/etc/mail/certs/cert.pem
O ServerKeyFile=/etc/mail/certs/key.pem
O ClientCertFile=/etc/mail/certs/cert.pem
O ClientKeyFile=/etc/mail/certs/key.pem
# O CRLFile=/etc/mail/certs/crlf
April 25, 2017
September 2005

18. Example - Sendmail STARTTLS relay based on CA Issuer
Advanced Sendmail Part 1
Following is an example set up using linux to linux systems. The two systems are as follows:
linux1.ban.hp.com: This is the sending system
linux4.ban.hp.com: This is the relaying system
On the linux1.ban.hp.com system the configuration parameters are set as follows:
O CACertPath=/etc/mail/certs
O CACertFile=/etc/mail/certs/CA/cacert.pem
O ServerCertFile=/etc/mail/certs/cert.pem
O ServerKeyFile=/etc/mail/certs/key.pem
O ClientCertFile=/etc/mail/certs/cert.pem
O ClientKeyFile=/etc/mail/certs/key.pem
# “Smart” relay host (may be null)
DSlinux4.ban.hp.com
April 25, 2017
September 2005

19. Advanced Sendmail 8.13.3 Part 1
Test the Relay
Advanced Sendmail Part 1
On the linux1 system the following command is entered to test the relay:
mail] # echo “Subject: test starttls” | sendmail –v –oL99
(The log level of 99 is turned on to see maximum logging for some of the STARTTLS output; this is done by turning on -oL99 )
Example output
Connecting to linux4.ban.hp.com. via relay...
220 linux4.ban.hp.com ESMTP Sendmail / ; Sun, 1 May :20:
>>> EHLO linux
250-linux4.ban.hp.com Hello linux1.ban.hp.com [ ], pleased to meet you
250-ENHANCEDSTATUSCODES
250-PIPELINING 250-8BITMIME
250-SIZE 250-DSN
250-ETRN 250-AUTH GSSAPI
250-STARTTLS
250-DELIVERBY
250 HELP
>>> STARTTLS
Ready to start TLS >>> EHLO linux 250-linux4.ban.hp.com Hello linux1.ban.hp.com [ ], pleased to meet you
250-PIPELINING
250-8BITMIME
250-SIZE
250-DSN
250-ETRN
250-AUTH EXTERNAL GSSAPI
>>> MAIL SIZE=23
Sender ok
>>> RCPT
>>> DATA
Recipient ok
354 Enter mail, end with "." on a line by itself
>>> .
j41FKgJV Message accepted for delivery
Sent (j41FKgJV Message accepted for delivery)
Closing connection to linux4.ban.hp.com.
>>> QUIT
linux4.ban.hp.com closing connection
April 25, 2017
September 2005

20. The /var/log/maillog output
Advanced Sendmail Part 1
A look at the /var/log/mailog output shows the relay that is used.
May 1 11:53:49 linux sendmail[1543]: STARTTLS=client, init=1
May 1 11:53:49 linux sendmail[1543]: STARTTLS=client, start=ok
May 1 11:53:49 linux sendmail[1543]: STARTTLS=client, get_verify: 0 get_peer: 0x
May 1 11:53:49 linux sendmail[1543]: STARTTLS=client, relay=linux4.ban.hp.com.,
version=TLSv1/SSLv3, verify=OK, cipher=EDH-RSA-DES CBC3-SHA, bits=168/168
May 1 11:53:49 linux sendmail[1543]: STARTTLS=client,
cert-subject=/C=US/ST=Georgia/L=Atlanta/O=HP/CN=linux4.ban.hp.com/
cert-issuer=/C=US/ST=Georgia/L=Atlanta/O=HP/CN=unix.ban.hp.com/
verifymsg=ok
Continuation of output
May 1 11:53:50 linux sendmail[1543]: j41Frn6h001543: ctladdr=root (0/0), delay=00:00:01, xdelay=00:00:01, maile =relay,
pri=30023, relay=linux4.ban.hp.com. [ ], dsn=2.0.0, stat=Sent (j41G99fb Message accepted for delivery) 
April 25, 2017
September 2005

21. Add a line to the access database
Advanced Sendmail Part 1
On the linux4 (relay system) you must add the following line to the access file and then rebuild the access database.
RELAY
Note that the tag is Address. Make sure you add Address to the access database.
(see additional notes for output that represents an incoming message)
Incoming message representation
May 1 12:09:09 linux4 sendmail[3201]: STARTTLS=server, get_verify: 0 get_peer: 0x
May 1 12:09:09 linux4 sendmail[3201]: STARTTLS=server,
relay=linux1.ban.hp.com [ ], version=TLSv1/SSLv3, verify=OK,
cipher=EDH-RSA-DES-CBC3-SHA, bits=168/168
cert-subject=/C=US/ST=Georgia/L=Atlanta/O=HP/CN=linux1.ban.hp.com/
cert-issuer=/C=US/ST=Georgia/L=Atlanta/O=HP/CN=unix.ban.hp.com/
May 1 12:09:09 linux4 sendmail[3201]: AUTH: available
mech=PLAIN LOGIN GSSAPI ANONYMOUS EXTERNAL,
allowed mech=EXTERNAL GSSAPI KERBEROS_V4 DIGEST-MD5 CRAM-MD5
April 25, 2017
September 2005

22. Advanced Sendmail 8.13.3 Part 1
April 25, 2017
September 2005