1. © Sam Ransbotham The Impact of Immediate Disclosure on Attack Diffusion and Volume Sam Ransbotham Boston College Sabyasachi Mitra Georgia Institute of Technology

2. © Sam Ransbotham 2 Security Vulnerabilities and Disclosure Does immediate disclosure of vulnerabilities affect exploitation attempts? Specifically, does immediate disclosure affect affect… Risk:the likelihood of a vulnerability being exploited? Diffusion:the diffusion of exploitations based on a vulnerability? Volume:the volume of exploitations based on the vulnerability? Methodology Statistical analysis of intrusion detection system attack and NVD data Key Result Immediate disclosure accelerates exploitation attempts, slightly increases number of distinct targets but decreases attack volume.

3. © Sam Ransbotham Disclosure Process as a R&D Race Discovery of Vulnerability Development of Exploit Method Diffusion of Attacks Firm is attacked A TTACK P ROCESS Discovery of Vulnerability Development of Patch by Vendor Diffusion of Patch Firm is patched Development of Countermeasures (e.g. detection signatures) Diffusion of Countermeasures S ECURITY P ROCESS Adapted from Ransbotham, Mitra, Ramsey (forthcoming MIS Quarterly) ?? Public Disclosure?

4. © Sam Ransbotham Tension: Immediate disclosure helps and hurtsAttackers - Disclosure provides information - Opens “window of opportunity” - Tells everyone the window is open Defenders - Can’t close a window you don’t know is open - Disclosure allows countermeasure development - Focuses defender attention - Encourages quick vendor response 4

5. © Sam Ransbotham 5 Research Environment Intrusion Detection System 0101010… Data Stream 0101010… Filtered Data Security Company Alert Database 0101010… Matched Alert Data Operator Signature Database Monitor Signature Updates NVD 400+ million alert subset 2006-2007, 960 firms National Vulnerability Database This paper matched to

6. © Sam Ransbotham 6 Alert (Attack) Data 1. 1.Who was attacked? Firm ID 1. 1.When did the attack occur? Timestamp 2. 2.Where did the attack come from? Source IP address 3. 3.What computer was attacked? Attacked IP address 4. 4.What vulnerability was used in the attack? Signature

7. © Sam Ransbotham 7 NVD Example Begin Date Disclosure(s) Alternative Explanations

8. © Sam Ransbotham 8 Key Control Variables 1. 1.Common Vulnerability Scoring System (CVSS) Assessment A. A.Access required: (local, adjacent, remote) B. B.Complexity: (low, medium, high) C. C.Authentication: (required or not) D. D.Impacts: (confidentiality, data integrity, availability of system resources) E. E.Type 1. 1.Access Validation: incorrect allowance of privileges 2. 2.Input Validation: failure to handle incorrect input 3. 3.Design Error: shortcomings in design of software 4. 4.Exception Error: Insufficient response to unexpected conditions 5. 5.Configuration Error: weak configuration of settings 6. 6.Race Condition: errors due to sequencing of events 2. 2.Patch available 3. 3.Signature available 4. 4.Application affected: Desktop or Server 5. 5.Disclosure through Market (paid) mechanism 6. 6.Age of vulnerability (days since publication)

9. © Sam Ransbotham 9 Summary of Data 1. 1.Alert data from MSSP (400+ million records) 2. 2.CERT/NVD vulnerability information Important unique features Not single firm; multiple firm Extended time period (two years) Real, not honeypot

10. © Sam Ransbotham Vulnerability details 10 Immediate DisclosureNon-Immediate VariableValueCount% % ComplexityLow27050.75%34751.87% Medium19436.47%26339.31% High6812.78%598.82% Confidentiality ImpactNo12122.74%15723.47% Yes41177.26%51276.53% Integrity ImpactNo10419.55%15623.32% Yes42880.45%51376.68% Availability ImpactNo10619.92%9714.50% Yes42680.08%57285.50% VulnerabilityInput18434.59%20630.79% Design7614.29%11116.59% Exception448.27%7210.76% Market DisclosureNo44182.89%60089.69% Yes9117.11%6910.31% Server ApplicationNo51396.43%65197.31% Yes193.57%182.69% Contains SignatureNo46687.59%57686.10% Yes6612.41%9313.90% Patch AvailableNo22442.11%32047.83% Yes30857.89%34952.17%

11. © Sam Ransbotham 11 Does immediate disclosure affect attacks? Three ways to analyze this question… 1. Risk:the likelihood of a vulnerability being exploited? Data summarized by firm, vulnerability, day Dependent variable is yes/no if attack seen on that day Using stratified Cox proportional hazard models 2. Diffusion:the diffusion of attacks based on a vulnerability? Data summarized by vulnerability, day Dependent variable is the cumulative number of firms attacked by that day Using nonlinear regression to estimate diffusion curve 3. Volume:the volume of attacks based on the vulnerability? Data summarized by firm, vulnerability, day Dependent variable is the count of attacks seen on that day Using Heckman two-stage regression

12. © Sam Ransbotham 12 VariableControl ModelTest Model Complexity: Medium-0.215***-0.188*** Complexity: High 0.227*** Confidentiality Impact-0.135***-0.165*** Integrity Impact 0.288***0.298*** Availability Impact 0.296***0.339*** Market Disclosure-1.508***-1.594*** Server Application-0.620***-0.628*** Patch Available 0.009-0.001 Signature Available 1.034***1.075*** Vulnerability Typesindicators Immediate Disclosure0.497*** Cox proportional hazard model of exploitation attempts across 1,152,406 observations of 1201 vulnerabilities in 960 firms; robust standard errors in parentheses; analysis stratified across 960 firms; significance levels: * p<0.05; ** p<0.01; *** p<0.001 Increased risk of exploitation attempt 1. Does immediate disclosure affect exploitation risk?

13. © Sam Ransbotham 13 2. Does immediate disclosure affect diffusion? Delay (D) Rate (R) cumulative penetration Penetration (P)

14. © Sam Ransbotham 14 VariablePenetration (P)Rate (R)Delay (D) Complexity: Medium174.27*** 0.57*** 136.68*** Complexity: High 42.09*** 0.57*** 20.65*** Confidentiality Impact-32.48*** 0.19*** 135.88*** Integrity Impact 11.74*** 0.39*** 91.90*** Availability Impact-11.13***-0.78***-156.51*** Server Application -3.05*-0.10*** 27.30*** Patch Available-19.94***-0.60***-140.87*** Market Disclosure-57.46***-1.15*** 278.74*** Signature Available123.24*** 1.42***-141.58*** Vulnerability Typesindicators Immediate Disclosure 3.69***-0.09*** -5.77** Nonlinear regression on the cumulative number of affected firms; 132,768 daily observations of vulnerabilities exploited in at least one of 960 firms. Robust standard errors in parentheses; significance levels: *p<0.05; **p<0.01; ***p<0.001 2. Does immediate disclosure affect diffusion? ?

15. © Sam Ransbotham 15 2. Does immediate disclosure affect diffusion? Acceleration Increased Penetration (?)

16. © Sam Ransbotham 16 VariableStage 1Stage 2 Complexity: Medium0.100***-0.050*** Complexity: High0.280***-0.037*** Confidentiality Impact0.015***0.031*** Integrity Impact0.501***-0.083*** Availability Impact-0.253***-0.005 Vulnerability Typesindicators Firm effectsindicators Monthly indicatorsPublish monthAlert month Age (in days, log)-0.210*** Server Application-0.325***0.130*** Market Disclosure-0.050***-0.098*** Patch Available-0.432**-0.019*** Signature Available0.738***0.166*** Immediate Disclosure-0.067***0.148*** Heckman two stage regression; n = 1,302,931; 709,090 uncensored; 1201vulnerabilities; standard errors in parentheses; significance levels: * p<0.05; **p<0.01; ***p<0.001 Stage 1: uncensored if exploit attempt for the vulnerability is observed in the sample Stage 2: natural log of the number of exploitation attempts increases volume 3. Does immediate disclosure affect volume of alerts?

17. © Sam Ransbotham 17 Immediate Disclosure can increase the risk, accelerate the diffusion and but decrease volume of attack attempts for vulnerabilities. Adds to the scarce empirical research (most analytical) Not single firm (hundreds) Extended time period (two years) Real attacks (not honeypot) Opens window for attackers But defenders are reacting quickly to close window Attackers seem to abandon attacks quickly as well Main Result

18. © Sam Ransbotham 18 Implications Immediate disclosure affects both actions on window--- closing and opening Forces defenders to react quickly May not be socially optimal; prioritization skewed? Limited disclosure? Unclear if results hold for extreme case (all immediate disclosure) Limited resource budget of defenders; attackers less limited Using “workload index” to help understand this Limitations Working to further clarify first disclosure; results are conservative High volume of noisy data: IDS and NVD Going forward