Download presentation

Presentation is loading. Please wait.

Published byCandace Rawls Modified over 2 years ago

1
The Effect of Decentralized Behavioral Decision Making on System-Level Risk Kim Kaivanto Department of Economics Lancaster University TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A AAA

2
In computer networks, system-level risk depends on the actions and choices of a collection of ‘lay’ users. How should we model the decision making of these lay users ? Does it matter whether our modeling assumptions reflect normative rationality or heuristics & biases?

3
Outline Classical SDT under normative rationality Behavioral factors From the decision-making literature: CPT From the phishing & deception literatures Re-derivation of optimal cutoff threshold under CPT-SDT Using T&K92 probability weighting function Using neo-additive probability weighting function Incorporating the psychology of deception Beyond comparative statics: comparative simulation results Implications Spam filtering Education & training End

4
Classical SDT Elements: - a ‘score’ variable, - known distributions of the score under and - as the cutoff threshold is varied, traces out the Receiver Operating Characteristics (ROC) curve

5
FP% + TN% = 100% FN% + TP% = 100% ‘rate’; ‘frequency’; ‘likelihood’ β + (1–β) = 1 α + (1–α) = 1 For the ‘healthy’: For the ‘diseased’: ROC curve:

6
Classical SDT

7
Binormal ROC curves d' = 1.7-1 = 0.7 Low-AUC ROC curve: d' = 2.7-1 = 1.7 High-AUC ROC curve:

8
Classical SDT Elements: - a ‘score’ variable, - known distributions of the score under and - as the cutoff threshold is varied, traces out the Receiver Operating Characteristics (ROC) curve Task: identify optimal cutoff threshold such that the observed score is either in the acceptance interval or in the rejection interval where the null hypothesis and the alt. hypothesis Problem: choose the optimal cutoff threshold by solving

9
Classical SDT The expected cost of using the SDT mechanism is: Setting the total differential of expected cost to zero it follows that the slope of each iso-E(C) line is the probability weighted ratio of the incremental cost of misclassifying a non-malicious email to the incremental cost of misclassifying a malicious email

10
Classical SDT The optimal cutoff threshold is a function of - a misclassification cost matrix - the baserate odds of (email) being non-malicious - risk preferences n.b. There is no reason for the misclassification costs to be the same for the user as for the organization n.b. Classical SDT admits that costs can be replaced by their utilities, but in fact proceeds (exclusively) with mini- mizing expected cost, i.e. assuming risk neutrality. n.b. In the literature, the ‘optimal classifier’ is computed under risk neutrality (!)

11
Classical SDT Only the difference between the misclassification and the correct classification matters for optimal cutoff placement

12
Behavioral factors Descriptively, behavioral decision makers display -reference dependence, framing effects -non-linear probability weighting -loss aversion -ambiguity aversion -four-fold pattern of risk aversion All incorporated in Cumulative Prospect Theory (CPT) Deception deploys employ -peripheral-route persuasion -authority, scarcity, similarity & identification, reciprocation, consistency, social proof -visceral emotions -urgency -contextual cues

13
CPT extension of SDT Using a conventional CPT specification

14
CPT extension of SDT (1-p)p (1-α) (1-β) α β C TN C FP C FN C TP

15
CPT extension of SDT Assume Further, wlog - this becomes the CPT reference point Then the CPT value function, in terms of and :

16
CPT extension of SDT We form the total differential of, set it to zero, and solve for the slope of the iso- contours in ROC space.

17
CPT extension of SDT Where the baserate probability term

18
CPT extension of SDT Iso- contours in ROC space

19
CPT extension of SDT Implication of non-linearity of iso- contours: Optimal operating point possibly non-unique. Uniqueness cannot be ensured with the T&K92 pwf.

20
CPT extension of SDT Let there be a set of constants so Whereby

21
CPT extension of SDT The implication of Is that the CPT extension of SDT also creates a ‘bias’ relative to the benchmark calculated under risk neutrality, which is consistent (in directionality) with the experimental psychology sense of conservative cutoff placement.

22
CPT extension of SDT with neo-additive pwf Consider the piece-wise linear neo-additive pwf “among the most promising candidates regarding the optimal tradeoff of parsimony and fit” (Wakker, 2010). Captures the possibility effect, the certainty effect, the overweighting of small probabilities, and the underweighting of large probabililties.

23
CPT extension of SDT with neo-additive pwf

24
Solving for the slope of the iso- contours in ROC space The iso- contours are staight lines, just as in classical SDT. This ensures uniqueness of the optimal cutoff threshold.

25
CPT-SDT with psychology of deception Successful deception deploys: -peripheral-route persuasion -visceral emotions -urgency -contextual cues The deception-perpetrator’s skill and effort. Mark i’s ploy-specific discriminability at time t:

26
CPT-SDT comparative statics The difference between classical SDT and CPT-SDT optimal trade-offs entails that the bias of incorrectly assuming normative rationality is larger for agents with a lower d’, i.e. a lower ROC curvature and AUC. CPT-SDT shifts the optimal cutoff and the optimal operating point more for agents with a lower ROC curvature and AUC. The psychology of deception magnifies the effect of be- havioral decision making under risk and uncertainty.

27
Comparative simulation results Are the individual-level behavioral effects quantitatively consequential at the level of the whole network? M1Classical SDT model M2CPT-SDT model M3CPT-SDT with psych of deception, We simulate (ABM, NetLogo) a 3-week spear-phishing attack on an organization with 100 users. Each user receives 250 emails per working week. 1/250=0.004 of emails are malicious. During an attack, a user may be fooled at most once. The users learn from their mistakes.

28
Comparative simulation results Are the individual-level behavioral effects quantitatively consequential at the level of the whole network? M1Classical SDT model M2CPT-SDT model M3CPT-SDT with psych of deception with probability

29
Comparative simulation results Distribution of security breaches in 10,000 repetitions

30
Comparative simulation results Distribution of security breaches in 10,000 repetitions

31
Comparative simulation results Distribution of security breaches in 10,000 repetitions

32
Comparative simulation results Distribution of security breaches in 10,000 repetitions

33
Comparative simulation results Distribution of security breaches in 10,000 repetitions

34
Comparative simulation results

37
Implications for education and training Target: discriminability prior probability susceptibility to deception Note: good spam filtering lowers p !

38
Conclusion Individual-level behavioral effects matter for system-level risk!

Similar presentations

OK

Chapter 4 Pattern Recognition Concepts: Introduction & ROC Analysis.

Chapter 4 Pattern Recognition Concepts: Introduction & ROC Analysis.

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google