1. Blacklist, Whitelist & spamtrap Terena EQUAL Workshop Dec 9 th 2009 amsterdam

2. Index SMTP Blacklist SMTP WhiteList Spamtraps

3. IRISRBL: RedIRIS blacklist system

4. IRISRBL motivations Which/How many Blacklist to use ? SMTP traffic can be slowed with too much DNS checks But better results (more spam blocked) What can we do with the false positives ? How fast can a IP address be removed from a Blacklist system ? How can the NREN provide an additional service to their members ?

5. IRISRLB: Motivations II Commercial Blacklist problems: For the SMTP provider (listed in it): Sometimes outgoing SMTP servers are listed Bounce messages Infected users sending spam …. Politics issues How to be removed from the list ? Need to pay money ? 48 hours delay To the user of the Black list: Messages not received Manual removing of black list / white list No information about why this IP address is listed

6. Blacklist implementation I Based on part of a bigger product, Rks from Sandvine, http://www.sandvine.comhttp://www.sandvine.com Service only for own constituency http://www.rediris.es/servicios/irisrbl/ Integrate different sources: Several blacklist White List & exceptions Events (Spamtraps) Only one query to DNS check the blacklist Small web interface to remove IP in the blacklists Only postmaster of the Blacklists (not IP owner) can remove IP addresses // false positives

7. Blacklist implementation: RKS Custom DNS server based with a database backend. Incremental feed of information Server dont need to restart to add new IP addresses. Flexible policy to define which feeds to add and when a IP is listed. Support for different sources. Different operating system support.

8. IRISRBL Stats More than 60% of RedIRIS constituency is using IRISBL. About 350 DNS queries/second

9. Whitelist

10. White List 2004/2005. Lot of black listing problems between Universities & ISP in Spain. SPF was not widely implemented Most of the mail providers, were using some kind of manual white list. No coordination.

11. Other White listprojects Some discussion in the E-COAT meetings, provide the initial jumpstart information. Dutch ISP WL. http://noc.bit.nl/dnsbl/nlwhitelist/ http://noc.bit.nl/dnsbl/nlwhitelist/ DNSWL.org, http://www.dnswl.orghttp://www.dnswl.org

12. WhiteList motivations Our main motivation is to avoid problems with blacklisting of SMTP server. We only tried a minimum quality requirement for being listed in the whitelist. Its more important to receive the legal email from a blacklisted smtp server than dont receive any email at all You can use other filters (content filters, etc) after the blacklist to avoid this spam

13. WhiteList Vision: button up Organizations usually exchange emails locally (country wide) SME partners and big local ISP are the main problem Including big ISP in the whitelist provide visibilit. Focus locally and exchange information with other similar initiatives.

14. White List format & usage: Two white list zones defined: ESWL: outgoing SMTP server of Abuses members. MTAWL: White list with big international email providers, other organizations and similar initiatives. White list is provided in different formats: DNS based (like blacklist) Configuration files for different SMTP servers. The files can be downloaded from the white list page. All the IP listed has a abuse/technical contact public address for troubleshooting

15. RedIRIS white list: Eswl y MTAwl RedIRIS Telefónica Euskaltel ESwl ONO MTAwl Goverment Yahoo,Gmail, Hotmail Agencias, … zone high DNSwl.org Others RedIRIS witoutSPF Telecable Sarenet Hostalia Ya.com TusProfesionales Pymes Hostalia RedIRIS White List

16. WL policy: Dont spend too much time thinking how to implement it. Simple policy: you are in the list Because you asked for this Someone added (mtawl ) People using the WL, want to have you in the WL. WL, dont provide any kind of reputation good SMTP behaviour, only states that this is the address of an SMTP server that usually dont send too much spam. But also you provide contact information for abuse reporting. And our spamtrap system allow us to monitor IP address behaviour

17. Version 1. Simple Perl scripts. Manual processing of the information Ad-hoc scripts to add information from other White List Success: Used by Universities & Spanish ISPs Great interest from other groups: Bank, local government … Fix most of the black listing problems between ISP & Universities.

18. Version 2. Web interface Registry of changes Most of the task can be done by the domain owners. Protocol to import information from other White List systems.

19. WhiteList soruces Spanish Universities & ISP SME Big SMTP providers Feeds from other sources DNSWL trustedsource

20. Conclusions Use a white list to avoid problems caused by blacklist, not to provide any kind of email assurance. Whitelist are useful if people knows and use it, (and usually they want also to be there). Having different level of quality promotes postmaster to reach the high level, improving the email quality overall. 20 Edificio Bronce Plaza Manuel Gómez Moreno s/n 28020 Madrid. España Tel.: 91 212 76 20 / 25 Fax: 91 212 76 35 www.red.eswww.red.es – www.rediris.eswww.rediris.es

21. SPAMTRAP system

22. Spamtrap Fake emails accounts to receive spam. Provide information for: Bad IP addresses that are sending spam(feed blacklist system) WL SMTP servers sending spam (compromise system, detection of bad usage or compromise) Early detect system of phising attacks.

23. Spamtrap features: Use domains & subdomains never used before. (ej, usr.rediris.es) Avoid collisions with real domains & addresses. Redirect domains to a central machine to avoid parsing receive headers. Source IP address is always in the first received line. Publish email addresses in web pages for crawlers.

24. Spamtrap : implementation Unix server + SMTP server (postfix) Subdomains provided by universities. Simple script to generate fake email addresses for the domains Publish the information in a web page with a warning message. Parsing of the incoming emails to remove bounces from smtp servers.

25. Spamttrap implementation (II) Batch system to avoid system overload Real time check against different DNSzones Detection of Whitelisted servers sending spam URL & binary extraction Extract malware from the files Store evidence for later use

26. Results of Spamtrap Blacklist: IP addresses that sent spam are used to feed the blacklist reputation system in real time (~5 minutes delay) WhiteList: IP addresses are verified against whitelist to detect infected machine and SMTP problems in the whitelist member. Phising/trend reporting: check some patterns to detect phising trends against some organizations in Spain. Provide information for security groups.

27. Expectations: Blacklist: Sharing of blacklist between NRENS Commercial agreement (SCS like) for Terena members ? Improve the tool WhiteList: Sharing of information between different NRENs Spamtrap: Improve the tool More robust sensor network.

28. 28 Edificio Bronce Plaza Manuel Gómez Moreno s/n 28020 Madrid. España Tel.: 91 212 76 20 / 25 Fax: 91 212 76 35 www.red.eswww.red.es – www.rediris.eswww.rediris.es