1. [1] Control Spam by the Use of Greylisting Torgny Hallenmark LDC - Computing Center Lund University, Sweden Torgny.Hallenmark@ldc.lu.se TERENA Networking Conference 2005 2005-06-07

2. [2] Lund University Lund University, Sweden Located in the very south of Sweden One of the biggest universities in Scandinavia Has almost all faculties 40 000 students 6 000 employees www.lu.se for more infowww.lu.se

3. [3] Lund University LU – employees and students Parallel mail servers used Same software used mainly Sun Java Enterprise System (JES) Sun JES Messaging Server Sun JES Directory Server (LDAP) Sendmail used in mail gateway Controlling systems: LUCAT and LADOK

4. [4] Mail systems LUCAT LDAP employees mail server Cassandra LADOK LDAP students mail server Piraten

5. [5] Mail systems New mail control structure Central mail gateway for spam and virus detection 3-4 parallel SunFire V240 servers In production since July 2004 Spam detection: SpamAssassin, Greylisting Virus detection: Sophos, ClamAV Address verification: Only messages to valid addresses on our domains are accepted

6. [6] Mail systems mail server POP IMAP webmail local mail server mail.lu.se Argus1-3 mail.lu.se Spam and virus detection mail.lu.se

7. [7] Mail systems Mail servers -- central / local Central mail server with services for POP, IMAP and webmail 75% of employees are using the central mail service Local mail servers exist in some departments Local mail servers can use the central mail gateway for spam and virus detection

8. [8] E-mail: Virus detection Virus detection in mail Software: Sophos and ClamAV Flagging in Subject: ***VIRUS*** Virus infected attachments are removed, info text is inserted Also possible to check for ”bad” file types

9. [9] E-mail: Spam detection Spam detection in mail Software: SpamAssassin Spam checks made, giving spam points Also RBL blacklists may give spam points Flagging in header: X-Spam-Flag Flagging in Subject: ***SPAM*** No messages are trashed centrally, only flagging is used (our policy) User must set up filter rules in his mail program

10. [10] Spam control: Greylisting Spam control: Greylisting added Set in production on 1 July 2004 Immediate impact ! Spam is no longer a problem !!! 90-95% of earlier spam is just gone ! Spam messages are not received, means less messages to check (for both spam and virus) No decision to take (if you would like to thrash messages)

11. [11] Spam control: Greylisting Greylisting effect: Messages counted in a user mailbox

12. [12] Spam control: Greylisting Greylisting effect: (Graph from Umeå University)

13. [13] Spam control: Greylisting Some user reactions: This is fantastic! Now you can again use e-mail like in the old days! The spam is gone! How did you do this? Lots of thanks! Really magic! Earlier I got 200 spam messages a day, now I see at most two! Thank you for an excellent work! It is almost sad with so few messages in my inbox … There must be something wrong with the mail system, I hardly get any mail at all …

14. [14] Spam control: Greylisting Greylisting technique Evan Harris: The Next Step in the Spam Control War http://projects.puremagic.com/greylisting/ Using Internet SMTP standard (RFC 821) The trick is following SMTP status: 451 4.7.1 TempFail – Please try again later Status code 451 can be handled by ”real” SMTP mail servers (put message in queue, try to resend it later) Spam spreading programs can not treat 451 status info … (not yet …)

15. [15] Spam control: Greylisting Greylisting technique For incoming messages following ”triplet” is checked: 1) IP address for sending SMTP server 2) Sender address (envelope sender) 3) Receiver address (envelope recipient) If this triplet has not been seen earlier: Send SMTP status 451, TempFail If this triplet has been seen at least 5 minutes ago: Accept the incoming message

16. [16] Spam control: Greylisting Greylisting technique A data base is needed (MySQL):  Time that triplet was first seen  Time that triplet blocking will expire  Time that triplet record itself will expire  Number of blocked delivery attempts  Number of messages sucessfully passed  Some other data

17. [17] Spam control: Greylisting Greylisting technique Some configuration parameters:  Unknown triplet, initial delay (default: 1 hour, our value: 5 minutes)  Lifetime of new triplets that have not yet allowed a mail to pass (default: 5 hours, our value: 30 hours)  Lifetime of auto-whitelisted triplets that have allowed mail to pass (default: 36 days)

18. [18] Spam control: Greylisting Greylisting – not always Manual whitelisting possible:  Can be done for sending SMTP server, sender address, receiver address  Our own IP series are whitelisted (making the mail gateway accept outgoing messages from our local mail clients)  Some ”odd” SMTP servers with problems with Greylisting may be whitelisted. (But why not fix those servers in stead?)

19. [19] Spam control: Greylisting Greylisting – any problems? Possible problems:  First delivery is always delayed (for an unknown triplet)  Some mail servers are really not following Internet SMTP standards, i e they don’t know how to handle SMTP status 451  Some mail servers have enormous spool queues, making resend of messages something that might happen first in a very distant future …  Greylisting is no final solution to the spam problem. Spammers will learn and adapt. But don’t tell them … !?

20. [20] Spam control: Greylisting Greylisting software  We use it together with Sendmail  But Greylisting can work with others:  Exim  Qmail  Qpsmtpd  Postfix  Squirrelmail  Mail proxies

21. [21] Spam control: Greylisting Software used at Lund University  Sendmail 8.13  Greylisting (invoked via Sendmail Milter function)  MailScanner 4.31  SpamAssassin 2.63  Some RBLs (used from SpamAssassin)  Sophos anti-virus  ClamAV anti-virus

22. [22] Spam control: Greylisting More Greylisting information http://projects.puremagic.com/greylisting/

24. [24] Lund University e-mail policy E-mail policy proposed  Outgoing e-mail will be accepted only from a few verified SMTP servers (very few servers running spam and virus programs)  All incoming e-mail must pass a central mail gateway performing spam and virus checking  Also internal e-mail (within the university) should pass spam and virus checking

25. [25] Lund University e-mail policy Why use local mail servers?  Goal: Reduce number of local mail servers  Since earlier: Only certified SMTP servers are accepted in the network (certified servers are listed in routers). Only certified servers can receive SMTP mail (via port 25).  Certification is mainly an open relay check.  Make central mail services better! No need for local mail servers.