Early Detection of Outgoing Spammers in Large-Scale Service Provider Networks Yehonatan Cohen Daniel Gordon Danny Hendler Ben-Gurion University Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

 Preliminaries  ErDOS: An Early Detection Scheme for Outgoing Spam  Evaluation  Conclusions and Future Work Danny Hendler and Philipp Woelfel, PODC 2009 Talk outline

Preliminaries  Spam Unsolicited mail, typically sent in large quantities  Hazards Malware distribution Phishing Resource consumption Poor user experience  Detection may be attempted when Mail is sent (outgoing spam detection) Mail is received (incoming spam detection) Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

Outgoing spam detection  Spam can be blocked before leaving the Service Provider (ESP)  Advantages Reduces load on ESP infrastructure Prevents damage to ESP reputation Detection may be based on hosted accounts' activity Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

Outgoing spam filtering techniques  Contents-based filtering: Learn & identify messages' textual patterns typical of spam messages May be tricked by manipulating spam content o Image-based o Random string insertion (hash busters) Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013 Non-negligible false negative rate

Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013 Outgoing spam filtering techniques (cont'd)  Inter-account communication patterns analysis: Models accounts' behaviour Based on inter-account social interactions Typically utilizes machine-learning techniques May leverage ESP account identification

 Devise an effective detector of outgoing spammers for large ESPs (the ErDOS detector)  Emphasis on early detection Detects spammers before the contents-based filter  Short training periods Highly adaptive to changing spamming patterns Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013 Our goals

Most relevant related work  Lam & Yeung, CEAS 2007 Introduce “social-network”-based outgoing spam detection Use the k-NN classifier Relatively small dataset (ENRON) Labeling based on simulated spammer accounts  Tseng & Chen, CSE 2009 Uses same set of features Uses SVM classifier Larger, non-ESP dataset (University server) Incremental model update Labeling based on pure accounts Account identification based on “from” header field Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

Comparison with data-sets of previous work Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013 Our data setNTUEnron #mails9.86E72.13E82.86E65.17E5 #accounts5.63E75.81E76.37E53.67E4 #edges7.40E712.90E7-3.68E5 time period 4 days (in/out) 26 days (outgoing) 10 days3.5 years contentsspam & ham ham  Collected by a very large ESP  Consists of incoming and outgoing log files o 4 days of bi-directional data + 22 days of outgoing traffic only  Both incoming and outgoing messages are labeled as spam/ham by a content-based detector

Comparison with data-sets of previous work Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013 Our data setNTUEnron #mails9.86E72.13E82.86E65.17E5 #accounts5.63E75.81E76.37E53.67E4 #edges7.40E712.90E7-3.68E5 time period 4 days (in/out) 26 days (outgoing) 10 days3.5 years contentsspam & ham ham  Collected by a very large ESP  Consists of incoming and outgoing log files o 4 days of bi-directional data + 22 days of outgoing traffic only  Both incoming and outgoing messages are labeled as spam/ham by a content-based detector

Danny Hendler and Philipp Woelfel, PODC 2009  Preliminaries  ErDOS: An Early Detection Scheme for Outgoing Spam Computation Flow Features  Evaluation  Conclusions and Future Work Talk outline

The ErDOS detector: computation flow Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013 Scored accounts Classified data set Classification model Undersampling: extract all spammers and equal number of legitimate accounts as training set Training set Remainder of accounts not in training set Determine accounts' classification Compute account feature values based on a single day of logs Build rotation forest model Assign account scores using classification model Construct suspect accounts list of configurable size Pre-processing Feature values computed

Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013  Preliminaries  ErDOS: An Early Detection Scheme for Outgoing Spam Computation Flow Features  Evaluation  Conclusions and Future Work Talk outline

Legitimate users  Maintain social interactions  Often belong to mailing lists Spammers  Sent messages seldom replied Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013 An account’s IOR = #incoming/#outgoing mails Low IOR characteristic of spammers ErDOS features: IOR

Danny Hendler and Philipp Woelfel, PODC 2009 ErDOS features: IOR (cont'd)

 Communication Reciprocity (CR) Fraction of recipients who responded to an account's s Defined by Gomes et al. IOR is superior for short training periods Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013 ErDOS features: IOR versus CR

 IEBC (Internal/External Behaviour Consistency) An account can send/receive s to/from  Internal addresses (accounts hosted by ESP)  External addresses Legitimate accounts show correlation between internal and external IOR, spammers less so Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013 ErDOS features: IEBC

ErDOS features: #outgoing messages  Number of outgoing messages Spamming accounts send more s than legitimate Insufficient for detecting low-volume spammers Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

 A large fraction of spammers' incoming mail is spam! Legitimate accounts seldom send s to spamming accounts Dictionary attacks may cause spammers to spam each other  Analyse senders' characteristics Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013 ErDOS: Sender Accounts' Characteristics

Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013  Preliminaries  ErDOS: An Early Detection Scheme for Outgoing Spam  Evaluation  Conclusions and Future Work Talk outline

Accuracy for Single-Day training  Evaluate Accuracy attained for single day logs accounts are classified based on the tags of the contents-base detector True Positive (TP) and False Positive (FP) values are averaged over available 4 days of bidirectional data Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013 ErDOSLY-knn MailNET TPFPTPFPTPFP

Early detection evaluation  Spamming accounts detected before the contents-based detector Suspected by detector, send messages tagged as spam only on later days Evaluation uses all 26 days of data  Early detection quality criteria: e-Precision: fraction of early detected accounts out of suspects list. Enrichment Factor (EF): ratio between detector's e-Precision and that of a random accounts list. Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

Early detection  Early detection results, averaged over 4 days:  Prior art’s early detections results compared to ErDOS: Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013 ErDOS’s suspectsEntire population #accounts100 Early detections90.53 e-Precision ErDOSLY-knnMailNET e-Precision EF

Early detection (cont’d)  e-Precision for varying suspects list lengths: Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

 Preliminaries  ErDOS: An Early Detection Scheme for Outgoing Spam  Evaluation  Conclusions and Future Work Talk outline

Conclusions and Future Work  Conclusions The case of outgoing spam detection for ESPs has its unique nature Contents-based filtering is not enough Early detection of spamming accounts can be achieve by a combination of contents-based filter and network level- based detector  Future Work Enhancement of ErDOS’s early detection performance by additional features A low-volume spammers expert detector, based on ErDOS’s computation flow and features Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013