Anti-Spam & Anti-Virus WiscMail Implementation University of Wisconsin - Madison CSG Workshop September 21, 2004

Message Composition - Fall 2004

The Spam Threat Users don’t want spam – Lost productivity – Offensive, Embarrassing – Legitimate messages get lost in the sea of spam Spam isn’t going away – People buy from spammers – Legislation has not been effective – The SMTP protocol is inadequate o It allows spammers to forge message information Spam is difficult to detect – Spammers learn how to get past filters – Legitimate messages WILL be lost

The Spam Threat Anti-Spam is difficult to support – Users don’t like misclassifications – Client based anti-spam solutions interfere – Authorized mass-mailers want special treatment Spammers use malware – Viruses “spam” themselves in mass quantities – Disinfected virus message clog Inboxes – Compromised computers DoS attack Anti-Spam services (RBLs) – Compromised computers send spam from inside the network

Anti-Spam Project Goals Reduce spam by 80% from current levels Users must be able to receive spam if they want (Opt-Out) Provide an option to select levels of filtering System must perform well and be scalable as message volumes increase Provide a Web Interface to system Compatible with existing infrastructure Vendor supported system

Anti-Virus & Anti-Spam Integration Why integrate anti-spam and anti-virus? – Faster processing o Messages are only opened once – Server consolidation – Virus messages can be treated as spam o Keeps the clutter out of the Inbox

How it works 1. Scan all incoming messages for spam and viruses – All potentially unsafe messages are scanned – Messages are marked with a spam “score” and then delivered as intended – Virus messages are deleted or disinfected 2. Filter the messages – Users choose whether or not to filter spam messages – Users choose what threshold (based on spam score) to filter spam

Spam Scanning Allow mail from trusted sources to pass unaffected All other mail is marked in the headers – e.g. X-Spam-Score: **** – 7 score levels o 0 asterisks means the message is likely not spam o 7 asterisks means the message is likely spam Deliver all messages to recipient

Virus Scanning Message is infected with a “junk” virus – e.g. netsky, bagel, mydoom, … – Delete messages without notification to sender or recipient o The induced message load from outbreaks causes delays for legitimate mail Message is infected with a virus – Remove virus – Mark message as spam – Append [VIRUS] to subject Message contains a suspicious attachment (exe, pif, scr, …) – Do nothing unless there is an outbreak – During an outbreak, treat these messages like viruses

Spam Filtering Server-side filtering service – Custom built interface that allows users to configure individual filters to move messages into IMAP folders – Based on Sieve RFC – Compatible with IMAP and Web Mail users ‘Junk Mail’ Folder – Reserved IMAP folder – Mail in ‘Junk Mail’ is deleted after 15 days of age

Spam Filtering Junk Mail Filter – Users specify desired spam threshold (based on spam score) – Moves all spam marked at the specified level (or higher) into ‘Junk Mail’ folder Accept List Filter – Keeps all mail from specified senders in the Inbox Block List Filter – Moves all mail from specified senders to the ‘Junk Mail’ folder Mailing Lists Filter – Keeps mail addressed to list addresses in the Inbox Custom Filters – Users can create filters to move messages into IMAP folders – e.g. “If the Subject contains ‘CSG’ move the message into the CSG folder”

Spam Filtering - Issues POP users see “disappearing ” – Mail is “POPed” from the Inbox only – POP users have to use Web Mail to see filtered mail – Alternatively, client-side filters can be used in conjunction with marked spam messages Conflicts with client-based anti-spam filters – More misclassifications o Client filters are looking for spam that isn’t there – Support confusion o Users see two “junk” folders – Most new clients have spam filtering enabled by default o Outlook 2003, Eudora 6, Mozilla variants

How Mail is Treated Server Filters Spam Scan Virus Scan User Filters Incoming Mail from WiscMail Users XXX All Other Incoming Mail XXXX Outgoing Mail XXN/A

Other Tools & Techniques Server Filters – Similar to user-level filters, but applies to all messages – Saves load on spam and virus scanners, by deleting or rejecting at the front door – Hundreds of thousands of SoBig messages stopped during 2003 outbreak – Only works if the messages have definable characteristics Site RBL – Real-time Blocking List – DNS Based – Allows us to dynamically block abusive computers from connecting to our mail servers

Other Tools & Techniques Require SMTP Authentication – Compromised (zombie) machines are becoming the major source of spam Rate Limit incoming and outgoing traffic – Limit abuse from spammers

Traffic Patterns

Traffic Patterns - Virus

Sophos PureMessage Direct integration with our mail software (Sun iMS) Also supports Sendmail and Postfix Uses multiple spam detection technologies – Heuristics, RBLs, checksums Customizable site policy based on Sieve RFC – Allows for specific actions based on message characteristics Many message actions provided – Header/body modification, quarantining, discard, drop/replace attachments Server cluster management

Sophos PureMessage Honey potting Dummy accounts set up to collect spam Misclassification submittal process Hourly automatic anti-spam heuristic updates Hourly automatic anti-virus IDE file updates Integration with anti-virus End-user quarantine management Not in use by UW-Madison

New Technologies Sender Authentication by IP Addresses – What is it? o Helps prevent address spoofing o Allows administrators to specify the computers that are authorized to use addresses in a particular domain o Stores information in DNS – SPF o Open project ( o 16% of mail domains have published SPF records – SenderID o Microsoft’s implementation, formerly called CallerID o proposal is having a hard time getting approved by the IETF o Relies on proprietary technology – The technology is still in too much flux to be fully embraced

New Technologies Sender Authentication with Content Signing – What is it? o Helps prevent address spoofing o Uses SSL certificates to ensure that messages are sent by legitimate senders from the domain – DomainKeys o Specification submitted to IETF by Yahoo o Stores certificates in DNS – This technology is not as advanced as sender authentication with IP addresses

Anti-Spam Technical Alliance – Yahoo!, Microsoft, EarthLink and AOL Recommendations to Help Stop Spam – – Address address forgery with sender authentication – Recommendations for ISPs e.g. rate limiting, limit port 25, close open relays, shut down zombie spammers – Recommendations for consumers e.g. install firewalls and desktop a/v, make use of spam filtering technologies that are provided by their ISP

Future Plans Sender Authentication (SPF) – Publish SPF records – Filter based on SPF Possible use of quarantining – Advantages o Keep spam on spam servers instead of Junk Mail folder o Users can choose what to do with the messages that are quarantined o Users can correct the spam server so that it makes the right decisions in the future – Disadvantages o There are compatibility issues with our infrastructure o Users would have to learn yet another process

Question and Answer