Exchange deployment at CERN and new ideas for SPAM fighting Michel Christaller, Emmanuel Ormancey, Alberto Pace

CERN Mail infrastructure  14 Servers  8 “Mailbox” stores, 2 Public Folder Stores, 2 Front-end servers, 2 Spare  IMAP (secure), POP (secure), MAPI and secure HTTP  MAPI with Outlook on Windows/Mac  MAPI open (in theory) outside CERN using Microsoft ISA Server  IMAP and POP work with almost any client  HTTP works with any Web browser  Collaborative tools available with MAPI and HTTP  Office XP recommended for collaborative features  Not possible to switch Outlook 2000 from IMO to CW  Allows multi protocol (pop, imap, mapi, webdav)  All information stored at server level, no more PST file problems  Office 2003 being evaluated  MAPI over HTTP  Seamless connected/disconnected/online/offline feature  Optimized for slow network connections

Migration overview  Nothing changes for the user Legacy Server New Server user.mailbox.cern.ch Mail Server Mail Client Mail User The server is replaced, Nothing changes for the client Additional interfaces available imap mapi http imaps pops webdav

Migration: what is done  User are invited to migrate by filling a migration form  The password is kept on the new service and synchronized with the windows password  Unresponsive users are forced to migrate and the password is reset  All folders and mails are copied from the old servers to Exchange  Mail Forwarding configuration is kept if any  Mailbox is not functional during at most 10 minutes, while rebuilding configuration files

Migration Workflow Migration Form Mailbox migrated Keep password typed in migration form Nice and Mail password synchronized Mailbox migrated Password reset Nice and Mail password synchronized “Ask for migration” mail Accept / Delay Form Reminder Mail (3) Accept After n reminders Force migration No answer Click on link

Migration Status  Exchange Users, Total  Only inactive and a few “non cooperative” users remaining  Cleanup: More than 700 Mail accounts deleted following user approval

Current status  1 year of production  Exchange software stable and scalable  No major disaster, only normal hardware failures, solved in operational delays  Usage: 50 % Outlook XP, other 50 % with IMAP, POP and HTTP access  1’000’000 Incoming mails per week, 30% is Spam

Next step, currently in test  Move SMTP Gateways to Exchange  Implement automatic anti flood system  Any server, sender or recipient sending or receiving more than 500 mails in 5 minutes will be banned (numbers to define)  Only solution to improve quality of service, and reduce impact of loops on “regular” mails  Migrate Mailing lists system from majordomo to Exchange  You will hear about this next year

Spam Fighting at Cern Evolution

Legacy system  Sendmail checks:  Lists of banned IP addresses, domains, subject, senders or recipients, and words  Header “consistency” tests (i.e. message id format)  Mail rejected if identified as Spam  Heavy manual work:  Update local banned lists from abuse reports  Remove entries when users report false positive rejections

Current service  Existing market products were reviewed:  Technology too young  Results are not accurate  Missing a per user basis configuration  While the market consolidates …  CERN developed his own Anti-Spam filter  Based on SpamAssassin  Less effort than running after immature commercial technology  Now in production for 1 year  Easy to modify and update detection techniques

How it works  The anti-spam filter calculates the probability for a message to be spam  Regular expressions  “Intelligent” content parsing  Statistical heuristics (Bayesian Filters)  The user sets the threshold at which he wants spam to be rejected  Rejected message can be seen by the user (CERN Spam folder)  Per user configuration (!)  Allows rejection of foreign languages mail (Chinese, Korean, Russian, Japanese, Arabic, etc …)

User configuration Filtering level Language-based rejection

Efficiency  Roughly Incoming mails per day  Spam filter detects from 25% to 35% as spam

Efficiency  False positives are very low  Except for commercial lists (spam that you want)  White lists at user level can be configured to prevent this  Good spam detection  Statistics are hard to build  Standard mailbox filtering statistics:  30 to 40 Spams filtered per day  1 or 2 Spams still go to the INBOX per week  Could still be improved with some optimization  Not enough for some users with “public” address  Old address or published address are more targeted for Spam

Current evolution  Spammer techniques always follow anti-spam techniques  New detection mechanisms work only for a few months  Needs a full time work to have a constantly “up- to-date” filter  Only viable long term solution is to accept only mails from people you know:  ICQ (and other messenger systems) already have this feature  Accept only messages from people in my contact list  Adding someone to the contact list requires validation

New feature (in test)  Good Mails not matching the user’s white list are quarantined  Mail is sent to sender requiring action to validate himself  Once validated, sender is added to white list, mails are moved back to Inbox Move to Inbox.Quarantine Quarantine level Inbox Move to Cern Spam Delete Spam Filter level Delete if evident spam level Mail to sender for validation

What’s next ?  Join forces against Spam  Share rules, regular expressions patterns and Bayesian statistics dictionary with other organizations  Central antispam configuration with Live Update like antivirus definitions is the solution. Therefore …  Long term goal: use a commercial product  Like for antivirus products, only a full time working team will provide up-to-date filters

In addition …  Within Exchange, mail is authenticated  Not possible to forge To: or From: fields  Delivery and Read receipts are reliable  A platform for workflow application  Extend this towards the internet  Mail messages digitally signed with guaranteed origin and dates  (See my presentation on PKI this Thursday)

Conclusion  Users are profiting from the new collaborative services  Shared calendar (already used by 1500 accounts)  Tasks, workflow  Web and webdav interfaces  Spam is a serious issue  Towards accepting only authenticated/verified mail  There is a future for commercial products in this area