01/04/2006ecs236 winter Intrusion Detection ecs236 Winter 2006: Intrusion Detection #3: Anomaly Detection Dr. S. Felix Wu Computer Science Department University of California, Davis

01/04/2006ecs236 winter Intrusion Detection Intrusion Detection Model Input event sequence Results Pattern matching

01/04/2006ecs236 winter Scalability of Detection l Number of signatures, amount of analysis l Unknown exploits/vulnerabilities

01/04/2006ecs236 winter Anomaly vs. Signature l Signature Intrusion (Bad things happen!!) –Misuse produces observable bad effect –Specify and look for bad behaviors l Anomaly Intrusion (Good things did not happen!!) –We know what our normal behavior is –Looking for an deviation from the normal behavior, raise early warning

01/04/2006ecs236 winter Reasons for “AND” l Unknown attacks (insider threat) l Better scalability –AND  target/vulnerabilities –SD  exploits

01/04/2006ecs236 winter Another definition… l Signature-based detection –Predefine the signatures of anomalies –Pattern matching l Statistics-based detection –Build statistics profile for expected behaviors –Compare testing behaviors with expected behaviors –Significant deviation Convert our limited/partial understanding/modeling about the target system or protocol into detection heuristics (i.e., BUTTERCUP signatures) Based on our experience, select a set of “features” that will likely to distinguish expected from unexpected behavior.

01/04/2006ecs236 winter What is “vulnerability”?

01/04/2006ecs236 winter What is “vulnerability”? Signature Detection create “effective/strong/scaleable” signatures Anomaly Detection detect/discover “unknown vulnerabilities”

01/04/2006ecs236 winter AND (ANomaly Detection) l Unknown Vulnerabilities/Exploits l Insider Attacks l Understand How and Why these things happened l Understand the limit of AND from both sides

01/04/2006ecs236 winter What is an anomaly?

01/04/2006ecs236 winter For each sample of the statistic measure, X (0, 1] 40% (1, 3] 30% (3, 15] 20% (15, +  ) 10% Input Events SAND

01/04/2006ecs236 winter quantify the anomalies alarm generationthreshold control raw events long term profile “But, which feature(s) to profile??” function F

01/04/2006ecs236 winter What is an anomaly? Events Expected Behavior Model Anomaly Detection

01/04/2006ecs236 winter What is an anomaly? Events Expected Behavior Model Anomaly Detection Knowledge about the Target

01/04/2006ecs236 winter Model vs. Observation the ModelAnomaly Detection Conflicts  Anomalies It could be an attack, but it might well be misunderstanding!!

01/04/2006ecs236 winter Statistic-based ANomaly Detection (SAND) l choose a parameter (a random variable hopefully without any assumption about its probabilistic distribution) l record its statistical “long-term” profile l check how much, quantitatively, its short- term behavior deviates from its long term profile l set the right threshold on the deviation to raise alarms

01/04/2006ecs236 winter decay update clean compute the deviation alarm generationthreshold control timer control raw events long term profile

01/04/2006ecs236 winter Statistical Profiling n Long-Term profile: u capture long-term behavior of a particular statistic measure u e.g., update once per day u half-life: 30 updates F recent 30: 50% F 31-60: 25% F the newer contributes more

01/04/2006ecs236 winter Statistical Pros and Cons l Slower to detect - averaging window l Very good for unknown attacks - as long as “relevant measures” are chosen l Environment (protocol, user, etc) dependency –Need good choices on statistical measures –Statistical profiles might be hard to build –Thresholds might be hard to set

01/04/2006ecs236 winter Long-term Profile l Category, C-Training 4learn the aggregate distribution of a statistic measure l Q Statistics, Q-Training 4learn how much deviation is considered normal l Threshold

01/04/2006ecs236 winter Long-term Profile: C-Training For each sample of the statistic measure, X (0, 50] 20% (50, 75] 30% (75, 90] 40% (90, +  ) 10% l k bins l Expected Distribution, P 1 P 2... P k, where l Training time: months

01/04/2006ecs236 winter Long-term Profile: Q-Training (1) For each sample of the statistic measure, X (0, 50] 20% (50, 75] 40% (75, 90] 20% (90, +  ) 20% l k bins, samples fall into bin l samples in total ( ) l Weighted Sum Scheme with the fading factor  s

01/04/2006ecs236 winter Threshold l Predefined threshold,  l If Prob(Q>q) < , raise alarm

01/04/2006ecs236 winter Long-term Profile: Q-Training (2) l Deviation: 4Example: l Q max 4the largest value among all Q values

01/04/2006ecs236 winter Long-term Profile: Q-Training (3) l Q Distribution 4[0, Qmax) is equally divided into 31 bins and the last bin is [Qmax, +  ) 4distribute all Q values into the 32 bins

01/04/2006ecs236 winter Weighted Sum Scheme l Problems of Sliding Window Scheme 4Keep the most recent N pieces of audit records 4required resource and computing time are O(N) l Assume 4K: number of bins 4Y i : count of audit records falls into i th bin 4N: total number of audit records 4  : fading factor l When E i occurs, update

01/04/2006ecs236 winter FTP Severs and Clients FTP Client SHANG FTP Servers Heidelberg NCU SingNet UIUC

01/04/2006ecs236 winter Q-Measure l Deviation: 4Example: l Q max 4the largest value among all Q values

01/04/2006ecs236 winter

01/04/2006ecs236 winter Threshold l Predefined threshold,  l If Prob(Q>q) < , raise alarm False positive

01/04/2006ecs236 winter

01/04/2006ecs236 winter Mathematics l Many other techniques: –Training/learning –detection

01/04/2006ecs236 winter decay update clean compute the deviation alarm generationthreshold control timer control raw events long term profile

01/04/2006ecs236 winter Dropper Attacks P% Per(K,I,S) Ret(K,S) Ran(K) Intentional or Unintentional??

01/04/2006ecs236 winter Periodical Packet Dropping l Parameters (K, I, S) 4K, the total number of dropped packets in a connection 4I, the interval between two consecutive dropped packets 4S, the position of the first dropped packet. l Example (5, 10, 4) 45 packets dropped in total 41 every 10 packets 4start from the 4 th packet 4The 4 th, 14 th, 24 th, 34 th and 44 th packet will be dropped

01/04/2006ecs236 winter Retransmission Packet Dropping l Parameters (K, S) 4K, the times of dropping the packet's retransmissions 4S, the position of the dropped packet l Example (5, 10) 4first, drops the 10 th packet 4then, drops the retransmissions of the 10 th packet 5 times

01/04/2006ecs236 winter Random Packet Dropping l Parameters (K) 4K, the total number of packets to be dropped in a connection l Example (5) 4randomly drops 5 packets in a connection

01/04/2006ecs236 winter Experiment Setting FTP Internet Divert Socket FTP Client xyz.zip 5.5M FTP Server Attack Agent Data Packets

01/04/2006ecs236 winter Impacts of Packet Dropping On Session Delay

01/04/2006ecs236 winter Compare Impacts of Dropping Patterns PerPD: I=4, S=5 RetPD: S=5

01/04/2006ecs236 winter bone fire redwing light UDP flood FTP data TFN agents TFN target FTP client FTP server congestion air TFN master

01/04/2006ecs236 winter

01/04/2006ecs236 winter TDSAM Experiment Setting FTP Internet Divert Socket FTP Client xyz.zip 5.5M FTP Server Attack Agent TDSAM Data Packets p1, p2, p3, p5, p4 max reordering counting

01/04/2006ecs236 winter

01/04/2006ecs236 winter

01/04/2006ecs236 winter Results: Position Measure

01/04/2006ecs236 winter Results: Delay Measure

01/04/2006ecs236 winter Results: NPR Measure

01/04/2006ecs236 winter Results (good and bad) l False Alarm Rate 4less than 10% in most cases, the highest is 17.4% l Detection Rate 4Position: good on RetPD and most of PerPD > at NCU, 98.7% for PerPD(20,4,5), but 0% for PerPD(100, 40, 5) in which dropped packets are evenly distributed 4Delay: good on those significantly change session delay, e.g., RetPD, PerPD with a large value of K > at SingNet, 100% for RetPD(5,5), but 67.9% for RanPD(10) 4NPR: good on those dropping many packets > at Heidelberg, 0% for RanPD(10), but 100% for RanPD(40)

01/04/2006ecs236 winter Performance Analysis l Good sites correspond to a high detection rate. 4stable and small session delay or packet reordering 4e.g., using Delay Measure for RanPD(10): UIUC (99.5%) > Heidelberg(74.5%) > SingNet (67.9%) > NCU (26.8%) l How to choose the value of nbin is site-specific 4e.g., using Position Measure, lowest false alarm rate occurs when nbin= 5 at Heidelberg(4.0%) and NCU(5.4%), 10 at UIUC(4.5%) and 20 at SingNet(1.6%)

01/04/2006ecs236 winter decay update clean compute the deviation alarm generationthreshold control timer control raw events long term profile

01/04/2006ecs236 winter decay update clean cognitively identify the deviation alarm identification Information Visualization Toolkit raw events cognitive profile

01/04/2006ecs236 winter What is an anomaly?

01/04/2006ecs236 winter What is an anomaly? l The observation of a target system is inconsistent, somewhat, with the expected conceptual model of the same system

01/04/2006ecs236 winter What is an anomaly? l The observation of a target system is inconsistent, somewhat, with the expected conceptual model of the same system l And, this conceptual model can be ANYTHING. –Statistical, logical, or something else

01/04/2006ecs236 winter Model vs. Observation the ModelAnomaly Detection Conflicts  Anomalies It could be an attack, but it might well be misunderstanding!!

01/04/2006ecs236 winter The Challenge Events Expected Behavior Model Anomaly Detection Knowledge about the Target False Positives & Negatives

01/04/2006ecs236 winter Challenge l We know that the detected anomalies can be either true-positive or false-positive. l We try all our best to resolve the puzzle by examining all information available to us. l But, the “ground truth” of these anomalies is very hard to obtain –even with human intelligence

01/04/2006ecs236 winter Problems with AND l We are not sure about whatever we want to detect… l We are not sure either when something is caught… l We are still in the dark… at least in many cases…

01/04/2006ecs236 winter Anomaly Explanation l How will a human resolve the conflict? l The Power of Reasoning and Explanation –We detected something we really want to detect  reducing false negative –Our model can be improved  reduce false positive

01/04/2006ecs236 winter Without Explanation l AND is not as useful?? l Knowledge is the power to utilize information! –Unknown vulnerabilities –Root cause analysis –Event correlation

01/04/2006ecs236 winter Anomaly Explanation the ModelAnomaly Detection Anomaly Analysis and Explanation EBL Explaining both the attack and the normal behavior

01/04/2006ecs236 winter Explanation Simulation Experiments Or Observatinon Conflicts  Anomalies

01/04/2006ecs236 winter the Model model-based event analysis observed system events SBL-based Anomaly Detection analysis reports Example Selection Explanation Based Learning model update

01/04/2006ecs236 winter AND  EXPAND l Anomaly Detection –Detect –Analysis and Explanation –Application

01/04/2006ecs236 winter

01/04/2006ecs236 winter

01/04/2006ecs236 winter

01/04/2006ecs236 winter