1. The Economics and Psychology of Security Ross Anderson Cambridge University

2. Social Science and Security The link between economics and security atrophied after WW2 The link between economics and security atrophied after WW2 Since 2000, we have started to apply economic analysis to IT security and dependability Since 2000, we have started to apply economic analysis to IT security and dependability Economic analysis often explains failure better then technical analysis! Economic analysis often explains failure better then technical analysis! Infosec mechanisms are used increasingly to support business models (DRM, accessory control) rather than to manage risk Infosec mechanisms are used increasingly to support business models (DRM, accessory control) rather than to manage risk Economic analysis is also vital for the public policy aspects of security Economic analysis is also vital for the public policy aspects of security Sociology and psychology are now engaged too Sociology and psychology are now engaged too

3. Traditional View of Infosec People used to think that the Internet was insecure because of lack of features – crypto, authentication, filtering People used to think that the Internet was insecure because of lack of features – crypto, authentication, filtering So engineers worked on providing better, cheaper security features – AES, PKI, firewalls … So engineers worked on providing better, cheaper security features – AES, PKI, firewalls … About 1999, we started to realize that this is not enough About 1999, we started to realize that this is not enough

4. Incentives and Infosec Electronic banking: UK banks were less liable for fraud, so ended up suffering more internal fraud and more errors Electronic banking: UK banks were less liable for fraud, so ended up suffering more internal fraud and more errors Distributed denial of service: viruses now don’t attack the infected machine so much as using it to attack others Distributed denial of service: viruses now don’t attack the infected machine so much as using it to attack others Health records: hospitals, not patients, buy IT systems, so they protect hospitals’ interests rather than patient privacy Health records: hospitals, not patients, buy IT systems, so they protect hospitals’ interests rather than patient privacy Why is Microsoft software so insecure, despite market dominance? Why is Microsoft software so insecure, despite market dominance?

5. New View of Infosec Systems are often insecure because the people who guard them, or who could fix them, have insufficient incentives Systems are often insecure because the people who guard them, or who could fix them, have insufficient incentives Bank customers suffer when poorly-designed bank systems make fraud and phishing easier Bank customers suffer when poorly-designed bank systems make fraud and phishing easier Patients suffer when hospital systems break privacy Patients suffer when hospital systems break privacy Casino websites suffer when infected PCs run DDoS attacks on them Casino websites suffer when infected PCs run DDoS attacks on them Insecurity is often what economists call an ‘externality’ – a side-effect, like environmental pollution Insecurity is often what economists call an ‘externality’ – a side-effect, like environmental pollution

6. New Uses of Infosec Xerox started using authentication in ink cartridges to tie them to the printer – and its competitors soon followed Xerox started using authentication in ink cartridges to tie them to the printer – and its competitors soon followed Motorola then started authenticating mobile phone batteries to the phone Motorola then started authenticating mobile phone batteries to the phone Carmakers make ‘chipping’ harder, and plan to authenticate major components Carmakers make ‘chipping’ harder, and plan to authenticate major components DRM: Apple grabs control of music download, MS trying to do the same for HD video content DRM: Apple grabs control of music download, MS trying to do the same for HD video content

7. IT Economics (1) The first distinguishing characteristic of many IT product and service markets is network effects The first distinguishing characteristic of many IT product and service markets is network effects Metcalfe’s law – the value of a network is the square of the number of users Metcalfe’s law – the value of a network is the square of the number of users Real networks – phones, fax, email Real networks – phones, fax, email Virtual networks – PC architecture versus MAC, or Symbian versus WinCE Virtual networks – PC architecture versus MAC, or Symbian versus WinCE Network effects tend to lead to dominant firm markets where the winner takes all Network effects tend to lead to dominant firm markets where the winner takes all

8. IT Economics (2) Second common feature of IT product and service markets is high fixed costs and low marginal costs Second common feature of IT product and service markets is high fixed costs and low marginal costs Competition can drive down prices to marginal cost of production Competition can drive down prices to marginal cost of production This can make it hard to recover capital investment, unless stopped by patent, brand, compatibility … This can make it hard to recover capital investment, unless stopped by patent, brand, compatibility … These effects can also lead to dominant-firm market structures These effects can also lead to dominant-firm market structures

9. IT Economics (3) Third common feature of IT markets is that switching from one product or service to another is expensive Third common feature of IT markets is that switching from one product or service to another is expensive E.g. switching from Windows to Linux means retraining staff, rewriting apps E.g. switching from Windows to Linux means retraining staff, rewriting apps Shapiro-Varian theorem: the net present value of a software company is the total switching costs Shapiro-Varian theorem: the net present value of a software company is the total switching costs This is why so much effort goes into managing switching costs – once you have $3000 worth of songs on a $300 iPod, you’re locked into iPods This is why so much effort goes into managing switching costs – once you have $3000 worth of songs on a $300 iPod, you’re locked into iPods

10. IT Economics and Security High fixed/low marginal costs, network effects and switching costs all tend to lead to dominant- firm markets with big first-mover advantage High fixed/low marginal costs, network effects and switching costs all tend to lead to dominant- firm markets with big first-mover advantage So time-to-market is critical So time-to-market is critical Microsoft philosophy of ‘we’ll ship it Tuesday and get it right by version 3’ is not perverse behaviour by Bill Gates but quite rational Microsoft philosophy of ‘we’ll ship it Tuesday and get it right by version 3’ is not perverse behaviour by Bill Gates but quite rational Whichever company had won in the PC OS business would have done the same Whichever company had won in the PC OS business would have done the same

11. IT Economics and Security (2) When building a network monopoly, you must appeal to vendors of complementary products When building a network monopoly, you must appeal to vendors of complementary products That’s application software developers in the case of PC versus Apple, or now of Symbian versus WinCE, or WinMP versus Real That’s application software developers in the case of PC versus Apple, or now of Symbian versus WinCE, or WinMP versus Real Lack of security in earlier versions of Windows made it easier to develop applications Lack of security in earlier versions of Windows made it easier to develop applications So did the choice of security technologies that dump most costs on the user (SSL, PKI, …) So did the choice of security technologies that dump most costs on the user (SSL, PKI, …) Once you’re a monopolist, lock it all down! Once you’re a monopolist, lock it all down!

12. Why are so many security products ineffective? Akerlof’s Nobel-prizewinning paper, ‘The Market for Lemons’ introduced asymmetric information Akerlof’s Nobel-prizewinning paper, ‘The Market for Lemons’ introduced asymmetric information Suppose a town has 100 used cars for sale: 50 good ones worth $2000 and 50 lemons worth $1000 Suppose a town has 100 used cars for sale: 50 good ones worth $2000 and 50 lemons worth $1000 What is the equilibrium price of used cars in this town? What is the equilibrium price of used cars in this town? If $1500, no good cars will be offered for sale … If $1500, no good cars will be offered for sale …

13. Security and Liability Why did digital signatures not take off? Why did digital signatures not take off? Industry thought: legal uncertainty. So EU passed electronic signature law Industry thought: legal uncertainty. So EU passed electronic signature law But: customers and merchants resist transfer of liability by bankers for disputed transactions But: customers and merchants resist transfer of liability by bankers for disputed transactions If you’re a customer, best stick with credit cards, so fraud remains largely the bank’s problem If you’re a customer, best stick with credit cards, so fraud remains largely the bank’s problem

14. Privacy Most people say they value privacy, but act otherwise. Most privacy technology firms failed Most people say they value privacy, but act otherwise. Most privacy technology firms failed Acquisti – people care about privacy when buying clothes, but not cameras (data relating to body or image are more privacy sensitive) Acquisti – people care about privacy when buying clothes, but not cameras (data relating to body or image are more privacy sensitive) Issue for mobile phone industry – phone viruses worse for image than PC viruses Issue for mobile phone industry – phone viruses worse for image than PC viruses Varian – you can maybe fix privacy by giving people property rights in personal information Varian – you can maybe fix privacy by giving people property rights in personal information Odlyzko – technology makes price discrimination both easier and more attractive Odlyzko – technology makes price discrimination both easier and more attractive

15. Why Bill wasn’t interested in security While Microsoft was growing, the two critical factors were speed, and appeal to application developers While Microsoft was growing, the two critical factors were speed, and appeal to application developers Security markets were over-hyped and driven by artificial factors Security markets were over-hyped and driven by artificial factors Issues like privacy and liability were more complex than they seemed Issues like privacy and liability were more complex than they seemed The public couldn’t tell good security from bad anyway The public couldn’t tell good security from bad anyway

16. Why is Bill now changing his mind? Security can help lock customers in, and extend power from one market to another Security can help lock customers in, and extend power from one market to another Information Rights Management changes ownership of a file from the machine owner to the file creator Information Rights Management changes ownership of a file from the machine owner to the file creator Remember: value of software company = total switching costs. And once documents can’t be converted without creators’ permission, the switching cost is much higher Remember: value of software company = total switching costs. And once documents can’t be converted without creators’ permission, the switching cost is much higher And: will WMP/Vista let Microsoft do to high definition movies what Apple did for music? And: will WMP/Vista let Microsoft do to high definition movies what Apple did for music?

17. Open versus Closed? Are open-source systems more dependable? It’s easier for the attackers to find vulnerabilities, but also easier for the defenders to find and fix them Are open-source systems more dependable? It’s easier for the attackers to find vulnerabilities, but also easier for the defenders to find and fix them Theory: openness helps both equally if bugs are random and standard dependability model assumptions apply Theory: openness helps both equally if bugs are random and standard dependability model assumptions apply Statistics: bugs are correlated in a number of real systems (‘Milk or Wine?’) Statistics: bugs are correlated in a number of real systems (‘Milk or Wine?’) Trade-off: the gains from this, versus the risks to systems whose owners don’t patch Trade-off: the gains from this, versus the risks to systems whose owners don’t patch

18. How Much to Spend? How much should the average company spend on information security? How much should the average company spend on information security? Governments, vendors say: much much more than at present! Governments, vendors say: much much more than at present! But they’ve been saying this for 20 years! But they’ve been saying this for 20 years! Measurements of security return-on- investment suggest about 20% p.a. overall Measurements of security return-on- investment suggest about 20% p.a. overall So the total expenditure may be about right So the total expenditure may be about right

19. Skewed Incentives Why do large companies spend too much on security and small companies too little? Why do large companies spend too much on security and small companies too little? Research shows there’s an adverse selection effect Research shows there’s an adverse selection effect Corporate security managers tend to be risk- averse people, often from accounting / finance Corporate security managers tend to be risk- averse people, often from accounting / finance More risk-loving people may become sales or engineering staff, or small-firm entrepreneurs More risk-loving people may become sales or engineering staff, or small-firm entrepreneurs There’s also due-diligence, government regulation, and insurance to think of There’s also due-diligence, government regulation, and insurance to think of

20. Skewed Incentives (2) If you are DirNSA and have a nice new hack on XP and Vista, do you tell Bill? If you are DirNSA and have a nice new hack on XP and Vista, do you tell Bill? Tell – protect 300m Americans Tell – protect 300m Americans Don’t tell – be able to hack 400m Europeans, 1000m Chinese,… Don’t tell – be able to hack 400m Europeans, 1000m Chinese,… If the Chinese hack US systems, they keep quiet. If you hack their systems, you can brag about it to the President If the Chinese hack US systems, they keep quiet. If you hack their systems, you can brag about it to the President So offence can be favoured over defence So offence can be favoured over defence

21. Large Project Failure Maybe 30% of large projects fail Maybe 30% of large projects fail But we build much bigger failures nowadays than 30 years ago so… But we build much bigger failures nowadays than 30 years ago so… Why do more public-sector projects fail? Why do more public-sector projects fail? Consider what the incentives are on project managers versus ministers – and what sort of people will become successful project managers versus ministers! Consider what the incentives are on project managers versus ministers – and what sort of people will become successful project managers versus ministers!

22. Security and Sociology There’s a lot of interest recently in using social networks to analyse interactions and systems There’s a lot of interest recently in using social networks to analyse interactions and systems Barabási and Albert showed that a scale-free network could be attacked efficiently by targeting its high-order nodes Barabási and Albert showed that a scale-free network could be attacked efficiently by targeting its high-order nodes Think: rulers target Saxon landlords / Ukrainian kulaks / Tutsi schoolteachers /… Think: rulers target Saxon landlords / Ukrainian kulaks / Tutsi schoolteachers /… Can we use evolutionary game theory ideas to figure out how networks evolve? Can we use evolutionary game theory ideas to figure out how networks evolve? Idea: run many simulations between different attack / defence strategies Idea: run many simulations between different attack / defence strategies

23. Security and Sociology (2) Vertex-order attacks with: Black – normal (scale- free) node replenishment Green – defenders replace high-order nodes with rings Cyan – they use cliques (c.f. system biology …)

24. Psychology and Security Fastest growing online crime is phishing – it only started in 2004, but by 2006 it cost the UK £35m and the USA perhaps $200m Fastest growing online crime is phishing – it only started in 2004, but by 2006 it cost the UK £35m and the USA perhaps $200m ‘Pretexting’ always existed (see Mitnick’s book), but phishing industrializes it ‘Pretexting’ always existed (see Mitnick’s book), but phishing industrializes it In a company you can train the staff in operational security (though many don’t). It’s harder when the target is your users! In a company you can train the staff in operational security (though many don’t). It’s harder when the target is your users! Maybe more secure machines would inevitably drive the bad guys to target the people instead Maybe more secure machines would inevitably drive the bad guys to target the people instead What can security folks learn from psychology? What can security folks learn from psychology?

25. Psychology and Security (2) Security usability research is fairly new and the results are pessimistic: most security products don’t work well or at all Security usability research is fairly new and the results are pessimistic: most security products don’t work well or at all Over half of all SSL certificates are ‘wrong’ Over half of all SSL certificates are ‘wrong’ No problem – we train people to keep on clicking ‘OK’ until they can get their work done No problem – we train people to keep on clicking ‘OK’ until they can get their work done Banks react to phishing by ‘blame and train’ efforts towards customers – but we know from the safety-critical world that this doesn’t work Banks react to phishing by ‘blame and train’ efforts towards customers – but we know from the safety-critical world that this doesn’t work Systems designed by geeks discriminate against women, the elderly and the less educated Systems designed by geeks discriminate against women, the elderly and the less educated

26. Psychology and Security (3) Social psychology has long been relevant to us! Social psychology has long been relevant to us! Solomon Asch showed most people would deny the evidence of their eyes to conform to a group Solomon Asch showed most people would deny the evidence of their eyes to conform to a group Stanley Milgram showed that 60% of people will do downright immoral things if ordered to Stanley Milgram showed that 60% of people will do downright immoral things if ordered to Philip Zimbardo’s Stanford Prisoner Experiment showed roles and group dynamics were enough Philip Zimbardo’s Stanford Prisoner Experiment showed roles and group dynamics were enough The disturbing case of ‘Officer Scott’ The disturbing case of ‘Officer Scott’ How can systems resist abuse of authority? How can systems resist abuse of authority? Why do people need enemies? Why do people need enemies? Why does terrorism work? Why does terrorism work?

27. Psychology and Security (4) Evolutionary psychology may eventually explain cognitive biases. It is based on the massive modularity hypothesis and the use of FMRI to track brain function Evolutionary psychology may eventually explain cognitive biases. It is based on the massive modularity hypothesis and the use of FMRI to track brain function Simon Baron-Cohen’s work on autism suggests a ‘theory of mind’ module central to empathy for others’ mental states Simon Baron-Cohen’s work on autism suggests a ‘theory of mind’ module central to empathy for others’ mental states This is how we differ from the great apes This is how we differ from the great apes It helps us lie, and to detect lies told by others It helps us lie, and to detect lies told by others So are we really homo sapiens sapiens – or homo sapiens deceptor? So are we really homo sapiens sapiens – or homo sapiens deceptor?

28. The Information Society More and more goods contain software More and more goods contain software More and more industries are starting to become like the software industry More and more industries are starting to become like the software industry The good: flexibility, rapid response The good: flexibility, rapid response The bad: frustration, poor service The bad: frustration, poor service The ugly: monopolies The ugly: monopolies How will society evolve to cope? How will society evolve to cope?

29. The Research Agenda We need to figure out how to balance competing social goals, as we have in the physical world We need to figure out how to balance competing social goals, as we have in the physical world Security economics gives us tools to understand what’s going on and to analyse policy options Security economics gives us tools to understand what’s going on and to analyse policy options Sociology also gives some useful insights Sociology also gives some useful insights And security psychology is not just a side discipline relevant to usability and phishing – it has the potential to bring us fundamental insights, just as security economics has And security psychology is not just a side discipline relevant to usability and phishing – it has the potential to bring us fundamental insights, just as security economics has

30. More … Economics and Security Resource Page – www.cl.cam.ac.uk/~rja14/econsec.html (or follow link from www.ross-anderson.com) Economics and Security Resource Page – www.cl.cam.ac.uk/~rja14/econsec.html (or follow link from www.ross-anderson.com) www.cl.cam.ac.uk/~rja14/econsec.htmlwww.ross-anderson.com www.cl.cam.ac.uk/~rja14/econsec.htmlwww.ross-anderson.com WEIS – Annual Workshop on Economics and Information Security – next at CMU, June 7–8 2006 WEIS – Annual Workshop on Economics and Information Security – next at CMU, June 7–8 2006 Foundation for Information Policy Research – www.fipr.org Foundation for Information Policy Research – www.fipr.orgwww.fipr.org