Presentation is loading. Please wait.

Presentation is loading. Please wait.

SureMail Notification Overlay for Email Reliability Sharad Agarwal Venkat Padmanabhan Dilip A. Joseph 8 March 2006.

Similar presentations


Presentation on theme: "SureMail Notification Overlay for Email Reliability Sharad Agarwal Venkat Padmanabhan Dilip A. Joseph 8 March 2006."— Presentation transcript:

1 Sur Notification Overlay for Reliability Sharad Agarwal Venkat Padmanabhan Dilip A. Joseph 8 March 2006

2 8.MAR Outline loss problem Design philosophy Sur design Sur robustness to security attacks Sur implementation

3 8.MAR What is Loss? loss : sent not received Silent loss –Loss w/o notification (no bounceback / DSN) Why? –Aggressive spam filters 90% corp. s thrown away (blacklist) AOL’s strict whitelist rules (must send 100/day) Bouncebacks contribute to spam –Complex mail architecture upgrades / failures SMTP reliability is per hop, not end-to-end

4 8.MAR How Much Loss? Even loss of 1 / user / year is bad –If it’s an important To really measure loss –Monitor many users’ send & receive habits –Count how many sent s not received –Count how many bouncebacks received –Difficult to find enough willing participants that each other across multiple domains

5 8.MAR Prior Work “The State of the Address” –Afergan & Beverly, ACM CCR Rely on bouncebacks; similar to “dictionary” attack 25% of tested domains send bouncebacks 1 sender 0.1% to 5% loss, across 1468 servers, 571 domains “ dependability” –Lang, UNSW B.E. thesis accounts, 16 domains receive s from 1 sender Empty body, sequence number as subject 0.69% silent loss

6 8.MAR Our Loss Study Methodology –Controller composes , sends –Our code for SMTP sending –Outlook for receiving (both inbox & junk mail) –Parse sent and received s into SQL DB –Match on {sender,receiver,subject,attachment} –Heuristics for parsing bouncebacks Want –Many sending, receiving accounts –Real content

7 8.MAR Experiment Details accounts –36 send, 42 receive –Junk filters off if possible subject & body –Enron corpus subset –1266 s w/o spam attachment –70% no attachment –jpg,gif,ppt,doc,pdf,zip,htm –marketing,technical,funny

8 8.MAR Loss Results

9 8.MAR Loss Rates by Account –Loss rate 1.82% to 0.82%

10 8.MAR Loss Rates by Attachment Nothing stands out

11 8.MAR Loss Rates by Subject/Body ~ s sent per subject Without 35% case : loss rate 1.82% to 1.79%

12 8.MAR Summary of Findings loss rates are high –1.82% loss –0.71% conservative silent loss ( 1 / 140 ) Difficult to disambiguate cause of loss –Difference between domains (filters or servers?) –No difference between mailboxes –No difference between attachments –Only 1 body had abnormally high loss

13 8.MAR Outline loss problem Design philosophy Sur design Sur robustness to security attacks Sur implementation

14 8.MAR We Found Loss; Now What? Can try to fix architecture, but –Hard to know exactly what is problem –Spam filters continually evolve; not perfect –Some architectures are very complicated –How many systems are out there? –The current system mostly works

15 8.MAR Fixing the Architecture Improve delivery infrastructure –more reliable servers e.g., cluster-based (Porcupine [Saito ’00]) –server-less systems e.g., DHT-based (POST [Mislove ’03]) –total switchover might be risky “Smarter” spam filtering –moving target  mistakes inevitable –non-content-based filtering still needed to cope with spam load

16 8.MAR Notifications DSN / bouncebacks –Most spam filters don’t generate DSN on drop –Bogus DSNs due to spam w/ bogus sender –Some MTAs block DSN for privacy –MTA crash may not generate DSN –No DSN for loss between MTA and MUA MDN / read receipts –Expose private info (when read, when online) –Can help spammers

17 8.MAR Notification Design Requirements Cause minimal MTA/MUA disruption Cause minimal user disruption Preserve asynchronous operation Preserve user privacy Preserve repudiability Maintain spam and virus defenses Minimize traffic overhead

18 8.MAR Outline loss problem Design philosophy Sur design Sur robustness to security attacks Sur implementation

19 8.MAR Sur Design Requirements Cause minimal MTA/MUA disruption No MTA modification; no Outlook modification Cause minimal user disruption User notified only on loss Preserve asynchronous operation Preserve user privacy Only receiver is notified of loss Preserve repudiability No PKI / authentication Maintain spam and virus defenses s not modified Minimize traffic overhead 85 byte notification per

20 8.MAR Basic Operation Sender S sends to receiver R –S also posts notification to overlay R periodically downloads new –R also downloads notifications from overlay Notification without matching  loss –delay : median 26s, mean 276s, max 36.6 hrs

21 8.MAR Sur Overview Sender S Recipient R You’ve Lost Mail! Request lost message D reg = H 2 (R) Register D not= H 1 (R) Verify GetNotifications H 1 (M new ), H 1 (M old ), T, MAC([T,H 1 (M new )],H 2 (M old ))

22 8.MAR Sur Overview s, MTAs, MUAs unmodified Parallel notification overlay system –Decentralized; limited collusion –Agnostic to actual implementation end-host-based (e.g., always-on user desktops) infrastructure-based (e.g., “NX servers”) Prevent notification snooping & spam – based registration –Reply based shared secret

23 8.MAR Based Registration Goal: prevent hijacking of R’s notifications –Only R can receive s sent to R –Limited collusion among notification nodes One-time operation for initial registration –R sends registration request to H 2 (R), H 3 (R) –H 2 (R), H 3 (R) registration secrets to R To retrieve notifications at H 1 (R) –R uses registration secrets with H 1 (R); H 1 (R) verifies with H 2 (R) H 3 (R), sends back notifications –Neither H 1 (R), H 2 (R), H 3 (R) can associate notifications with R, unless they collude

24 8.MAR Reply-Based Shared Secret Goal: prevent notification spoofing & spam –Only R & S know their conversations –S rarely converses with spammers Reply detection –S sends M old to R, R replies with M’ old –S uses H(M old ) to “prove” identity to R in future Notification for M new from S to R –H 1 (M new ),H 1 (M old ),T,MAC([T,H 1 (M new )],H 2 (M old )) –Only R can identify S –Shared secret can be continually refreshed

25 8.MAR Attacks Defeated by Design X cannot retrieve H 1 (R) notifications H 1 (R) cannot identify R H 2 (R), H 3 (R) cannot see R’s notifications –If they don’t collude; can increase to 3 nodes X, H(R) cannot identify S X, H(R) cannot learn M new, M old X cannot annoy R with bogus notifications X cannot masquerade post to H 1 (R) as S

26 8.MAR First Time Sender What if FTS is lost? FTS & spammer generally indistinguishable But perhaps FTS knows I who knows R – networks have small world properties –I makes shared secret S I with all known parties –FTS sends to R Posts multiple notifications One for every S I it has learned

27 8.MAR Other Issues Reply-detection: –“in-reply-to” header may not always help –indirect checks based on text similarity Reducing overhead: –post notifications only for “important” s –delay posting in hope of receiving implicit ACK (reply) or NACK (bounce-back) Mobility: –reply-based shared secret can be regenerated –web-mail Can support mailing lists

28 8.MAR Outline loss problem Design philosophy Sur design Sur robustness to security attacks Sur implementation

29 8.MAR Sur Implementation Reply detection heuristic for shared secret Notification service –Centralized server running –Chord based DHT running Notification posting, retrieving –Grab in/out bound via Outlook MAPI call No modification to Outlook binaries –XML notification put/get commands –Simple Win32 GUI

30 8.MAR Sur GUI Client UI will see s, will post & retrieve notifications E.g. running on two machines and Lost! Not lost

31 8.MAR Notification Results

32 8.MAR Summary does get lost! –~40 accounts, s, 0.71%-0.91% silent loss Sur –Client based – unmodified , servers, clients; no PKI –User intervention only on lost –Keeps repudiability, privacy, asynchronous, spam & virus defense Separate notification overlay robust –Simple, small message format –No virus, malware, spam filters needed –Provides failure independence Status –ACM Hotnets 05; ACM Sigcomm 06 submission –Prototype implementation


Download ppt "SureMail Notification Overlay for Email Reliability Sharad Agarwal Venkat Padmanabhan Dilip A. Joseph 8 March 2006."

Similar presentations


Ads by Google