1. Web App Security – The Good, the Bad and the Ugly Ross Anderson Cambridge University

2. Krakow May 13th 2009 Is Web 2.0 Reinventing the Whole World? html, javascript FBML SQLFBQL SMTP FB Mail Usenet FB Groups Open ID FB Connect Blogger FB Notes Twitter FB Status Updates craigslist FB Marketplace

3. Krakow May 13th 2009 So what’s changed? A cynic might say that IT just goes in cycles! A cynic might say that IT just goes in cycles! Back in the 60s and 70s, we had mainframe bureau services Back in the 60s and 70s, we had mainframe bureau services Then we had minis, then PCs Then we had minis, then PCs The pendulum seems to be swinging back – server farms do what mainframes used to The pendulum seems to be swinging back – server farms do what mainframes used to And we get a wide range of terminals – phones, netbooks, PCs, … And we get a wide range of terminals – phones, netbooks, PCs, … How should we make sense of all this? How should we make sense of all this?

4. Krakow May 13th 2009 Economics and Security About 2000, we realised that engineering analysis alone didn’t explain all that goes wrong About 2000, we realised that engineering analysis alone didn’t explain all that goes wrong Economic analysis often explains failure better! Economic analysis often explains failure better! Electronic banking: UK banks were less liable for fraud, so became careless and ended up suffering more internal fraud and errors Electronic banking: UK banks were less liable for fraud, so became careless and ended up suffering more internal fraud and errors Distributed denial of service: viruses now don’t attack the infected machine so much as use it to attack others Distributed denial of service: viruses now don’t attack the infected machine so much as use it to attack others Why is Microsoft software so insecure, despite market dominance? Why is Microsoft software so insecure, despite market dominance?

5. Krakow May 13th 2009 New View of Infosec Systems are often insecure because the people who guard them, or who could fix them, have insufficient incentives Systems are often insecure because the people who guard them, or who could fix them, have insufficient incentives Medical record systems bought by research or finance directors, not patients – so failed to protect privacy Medical record systems bought by research or finance directors, not patients – so failed to protect privacy Casino websites suffer when infected PCs run DDoS attacks on them Casino websites suffer when infected PCs run DDoS attacks on them Insecurity is often what economists call an ‘externality’ – a side-effect, like environmental pollution Insecurity is often what economists call an ‘externality’ – a side-effect, like environmental pollution

6. Krakow May 13th 2009 IT Economics (1) The first distinguishing characteristic of many IT product and service markets is network effects The first distinguishing characteristic of many IT product and service markets is network effects Metcalfe’s law – the value of a network is the square of the number of users Metcalfe’s law – the value of a network is the square of the number of users Real networks – phones, fax, email Real networks – phones, fax, email Virtual networks – PC architecture versus MAC, or Symbian versus WinCE Virtual networks – PC architecture versus MAC, or Symbian versus WinCE Network effects tend to lead to dominant-firm markets where the winner takes all Network effects tend to lead to dominant-firm markets where the winner takes all

7. Krakow May 13th 2009 IT Economics (2) Second common feature of IT product and service markets is high fixed costs and low marginal costs Second common feature of IT product and service markets is high fixed costs and low marginal costs Competition can drive down prices to marginal cost of production Competition can drive down prices to marginal cost of production This can make it hard to recover capital investment, unless stopped by patent, brand, compatibility … This can make it hard to recover capital investment, unless stopped by patent, brand, compatibility … These effects can also lead to dominant-firm market structures These effects can also lead to dominant-firm market structures

8. Krakow May 13th 2009 IT Economics (3) Third common feature of IT markets is that switching from one product or service to another is expensive Third common feature of IT markets is that switching from one product or service to another is expensive E.g. switching from Windows to Linux means retraining staff, rewriting apps E.g. switching from Windows to Linux means retraining staff, rewriting apps Shapiro-Varian theorem: the net present value of a software company is the total switching costs Shapiro-Varian theorem: the net present value of a software company is the total switching costs So major effort goes into managing switching costs – once you have $3000 worth of songs on a $300 iPod, you’re locked into iPods So major effort goes into managing switching costs – once you have $3000 worth of songs on a $300 iPod, you’re locked into iPods

9. Krakow May 13th 2009 IT Economics and Security High fixed/low marginal costs, network effects and switching costs all tend to lead to dominant- firm markets with big first-mover advantage High fixed/low marginal costs, network effects and switching costs all tend to lead to dominant- firm markets with big first-mover advantage So time-to-market is critical So time-to-market is critical Microsoft philosophy of ‘we’ll ship it Tuesday and get it right by version 3’ was quite rational Microsoft philosophy of ‘we’ll ship it Tuesday and get it right by version 3’ was quite rational Whichever company had won in the PC OS business would have done the same Whichever company had won in the PC OS business would have done the same “Growth is primary, revenue is secondary” – Mark Zuckerberg “Growth is primary, revenue is secondary” – Mark Zuckerberg

10. Krakow May 13th 2009 IT Economics and Security (2) When building a network monopoly, you must appeal to vendors of complementary products When building a network monopoly, you must appeal to vendors of complementary products That’s application software developers in the case of PC versus Apple, then of Symbian versus Windows/Palm, now Facebook That’s application software developers in the case of PC versus Apple, then of Symbian versus Windows/Palm, now Facebook Lack of security in early Windows / Symbian / Facebook made life easier for them Lack of security in early Windows / Symbian / Facebook made life easier for them So did the choice of security technologies that dump costs on the user (SSL, not SET) So did the choice of security technologies that dump costs on the user (SSL, not SET) Once you’ve a monopoly, lock it all down! Once you’ve a monopoly, lock it all down!

11. Krakow May 13th 2009 Security Economics and Web Applications The big security economics problem is aligning incentives The big security economics problem is aligning incentives The big system engineering problem is managing complexity. You want architecture, i.e. interfaces, to divide up systems sensibly The big system engineering problem is managing complexity. You want architecture, i.e. interfaces, to divide up systems sensibly Consider a travel agent, buying services from airlines, hotels etc. It pretty much all lines up Consider a travel agent, buying services from airlines, hotels etc. It pretty much all lines up Open interfaces, defined by contract Open interfaces, defined by contract Competition drives costs down, usability up Competition drives costs down, usability up

12. Krakow May 13th 2009 Security Economics and Web Applications (2) However, some web apps are platforms, so operate under the same forces as Windows or Symbian or S/360 However, some web apps are platforms, so operate under the same forces as Windows or Symbian or S/360 E.g. Facebook – huge network effects E.g. Facebook – huge network effects Incentives on its developers: Incentives on its developers: grab the market now, fix privacy later grab the market now, fix privacy later appeal to complementers (app writers) appeal to complementers (app writers) But does social context change anything? But does social context change anything?

13. Krakow May 13th 2009 How Fraud Adapts to SNS The old scams are still there – 419, spam, phishing, XSS, malware, click fraud, … The old scams are still there – 419, spam, phishing, XSS, malware, click fraud, … Social context makes phishing more effective (72% in controlled study – Jagatic) not to mention targeted attacks / scams Social context makes phishing more effective (72% in controlled study – Jagatic) not to mention targeted attacks / scams Facebook now 7th biggest phishing target (after PayPal, top banks, eBay) Facebook now 7th biggest phishing target (after PayPal, top banks, eBay) Frequent genuine emails with login links Frequent genuine emails with login links Some incentive on operator to fight it (spam caused decline of MySpace, Friendster) Some incentive on operator to fight it (spam caused decline of MySpace, Friendster)

14. Krakow May 13th 2009 Privacy Most people say they value privacy, but act otherwise. Most privacy ventures failed. Why? Most people say they value privacy, but act otherwise. Most privacy ventures failed. Why? Odlyzko – technology makes price discrimination both easier and more attractive Odlyzko – technology makes price discrimination both easier and more attractive Acquisti – people care about privacy when buying clothes, but not cameras Acquisti – people care about privacy when buying clothes, but not cameras Loewenstein – privacy is heavily context sensitive. People only really worry if salient Loewenstein – privacy is heavily context sensitive. People only really worry if salient Facebook viruses ‘worse’ than PC viruses (as more personal) or not (as less salient)? Facebook viruses ‘worse’ than PC viruses (as more personal) or not (as less salient)?

15. Krakow May 13th 2009 Privacy and SNS Conflict of interest Conflict of interest Facebook wants to sell user data Facebook wants to sell user data Users want feeling of intimacy, small group, social control Users want feeling of intimacy, small group, social control Very complex access controls – over 60 settings on 7 pages Very complex access controls – over 60 settings on 7 pages Over 90% of users never change defaults Over 90% of users never change defaults The complexity lets Facebook blame the customer when things go wrong The complexity lets Facebook blame the customer when things go wrong

16. Krakow May 13th 2009 Privacy and SNS (2)

17. Krakow May 13th 2009 Privacy and SNS (3) See our paper ‘Eight friends are enough’ Given the eight published friends, an outsider can run all the usual network analysis Including covert community detection as used by the spooks

18. Krakow May 13th 2009 Security Economics and Web Applications (3) As you’d expect from the incentives, Facebook provides the appearance of security, not reality – ‘security theatre’ As you’d expect from the incentives, Facebook provides the appearance of security, not reality – ‘security theatre’ Abd it deals with the occasional outrage using ‘democracy theatre’ (see our blog, www.lightbluetouchpaper.org for more) Abd it deals with the occasional outrage using ‘democracy theatre’ (see our blog, www.lightbluetouchpaper.org for more) www.lightbluetouchpaper.org Is this sustainable? Is this sustainable? Long-term problem: European regulators Long-term problem: European regulators

19. Krakow May 13th 2009 Security Economics and Web Applications (4) Sometimes the monopoly doesn’t come from platform dynamics but exogenously Sometimes the monopoly doesn’t come from platform dynamics but exogenously Example: UK attempt to centralize all medical records, children’s records Example: UK attempt to centralize all medical records, children’s records Records at GPs, hospitals being moved to ‘hosted’ systems Records at GPs, hospitals being moved to ‘hosted’ systems Sales pitch: benefits of research Sales pitch: benefits of research Driver: bureaucratic centralization Driver: bureaucratic centralization Gotcha: I v Finland Gotcha: I v Finland

20. Krakow May 13th 2009 Security Economics and Web Applications (5) Thankfully the UK TG programme is failing; see our report “Database State” for more Thankfully the UK TG programme is failing; see our report “Database State” for more But might Google or Microsoft make a health- record web service work? But might Google or Microsoft make a health- record web service work? There are similar incentives on private and public sectors to collect data in order to price discriminate between clients / citizens There are similar incentives on private and public sectors to collect data in order to price discriminate between clients / citizens Are there any technical limits (systems complexity, microeconomics) or must we rely on our legislators and courts? Are there any technical limits (systems complexity, microeconomics) or must we rely on our legislators and courts?

21. Krakow May 13th 2009 The Gladman Principle “You can have security, or functionality, or scale. With good engineering you can have any two of these. But there’s no way you can get all three.” “You can have security, or functionality, or scale. With good engineering you can have any two of these. But there’s no way you can get all three.” Brian Gladman (formerly of UK Brian Gladman (formerly of UK Defence Science Advisory Board)

22. Krakow May 13th 2009 Compartmentation It’s OK to have 20 doctors and nurses having access to 10,000 patients’ records in a medical practice It’s OK to have 20 doctors and nurses having access to 10,000 patients’ records in a medical practice With some care, it’s just about OK to have 2000 doctors and nurses having access to 1,000,000 patients’ records in a hospital With some care, it’s just about OK to have 2000 doctors and nurses having access to 1,000,000 patients’ records in a hospital It’s not OK to have 580,000 health service staff having access to 50,000,000 citizens’ records on a national database It’s not OK to have 580,000 health service staff having access to 50,000,000 citizens’ records on a national database … as our Prime Minister has learned … … as our Prime Minister has learned …

23. Krakow May 13th 2009 Attack Trends One aspect of security economics is building models that explain how things go wrong One aspect of security economics is building models that explain how things go wrong Another is the econometrics – measuring what actually does go wrong Another is the econometrics – measuring what actually does go wrong We have a research project on collecting statistics on spam, phishing, malware (see my Google tech talk, for example) We have a research project on collecting statistics on spam, phishing, malware (see my Google tech talk, for example) Recent trends in malware are getting worrying! Recent trends in malware are getting worrying! If an attack can be industrialized, it will be … If an attack can be industrialized, it will be …

24. Krakow May 13th 2009 Case study – the Dalai Lama Simple attacks reported on the Office of His Holiness the Dalai Lama (OHHDL) since 2007 Simple attacks reported on the Office of His Holiness the Dalai Lama (OHHDL) since 2007 From directed spam to simple targeted attacks From directed spam to simple targeted attacks Compromise became obvious in July 2008 – foreign diplomats about to meet the Dalai Lama were warned off Compromise became obvious in July 2008 – foreign diplomats about to meet the Dalai Lama were warned off We got asked to investigate We got asked to investigate

25. Krakow May 13th 2009 Modus Operandi A sends email to B on topic X, archived publicly A sends email to B on topic X, archived publicly C sends email to A pretending to be B, on topic X, with toxic attachment C sends email to A pretending to be B, on topic X, with toxic attachment C pretending to be A takes over mail server C pretending to be A takes over mail server Internal mail attachments thereafter toxic Internal mail attachments thereafter toxic PCs then accessed remotely … PCs then accessed remotely … We call this ‘Social Malware’ We call this ‘Social Malware’ The typical company has no defence at all! The typical company has no defence at all!

26. Krakow May 13th 2009 A low grade sample

27. Krakow May 13th 2009 Malware Equilibrium? Big change in 2004: black market led to specialisation Big change in 2004: black market led to specialisation Malware now professionally written; most exploits are for money, not bragging rights Malware now professionally written; most exploits are for money, not bragging rights Most companies just don’t know how to block social malware (even Deloittes was among the victims of the Chinese) Most companies just don’t know how to block social malware (even Deloittes was among the victims of the Chinese) What will the world be like if 1%, or 5%, or machines are 0wned, and exploited? What will the world be like if 1%, or 5%, or machines are 0wned, and exploited?

28. Krakow May 13th 2009 Open versus Closed? Are open systems more dependable? It’s easier for the attackers to find vulnerabilities, but also easier for the defenders to find and fix them Are open systems more dependable? It’s easier for the attackers to find vulnerabilities, but also easier for the defenders to find and fix them This debate goes back to the 17th century! This debate goes back to the 17th century! Theorem (2002): openness helps both equally if bugs are random and standard dependability model assumptions apply Theorem (2002): openness helps both equally if bugs are random and standard dependability model assumptions apply So whether open is better than closed will depend on whether / how your system differs from the ideal So whether open is better than closed will depend on whether / how your system differs from the ideal

29. Krakow May 13th 2009 The Good, the Bad and the Ugly Travel agent: not a big deal if the bad guys occasionally go on holiday (the bank pays) Travel agent: not a big deal if the bad guys occasionally go on holiday (the bank pays) Facebook: there will be all sorts of platform exploits, and social exploits, with which they’ll have to cope. As for compromised user machines, my daughter’s view … Facebook: there will be all sorts of platform exploits, and social exploits, with which they’ll have to cope. As for compromised user machines, my daughter’s view … Government databases: you can’t make everyone’s medical records available to 500,000 doctors and nurses and still have privacy Government databases: you can’t make everyone’s medical records available to 500,000 doctors and nurses and still have privacy The insider (malware) threat sets limits here! The insider (malware) threat sets limits here!

30. Krakow May 13th 2009 An Opportunity If 1% of end-user machines will always be infected with malware, what can we do? If 1% of end-user machines will always be infected with malware, what can we do? Web services can offer a haven Web services can offer a haven But they need to assume some corrupt insiders But they need to assume some corrupt insiders Experience from defence – compartmentation Experience from defence – compartmentation And from accounting – dual control, audit, backup, … And from accounting – dual control, audit, backup, … How do you build these ideas into other apps? How do you build these ideas into other apps? What other limits on security, functionality and scale are there – and what’s the social angle? What other limits on security, functionality and scale are there – and what’s the social angle?

31. Krakow May 13th 2009 The Research Agenda The online world and the physical world are merging – many years of turbulence ahead! The online world and the physical world are merging – many years of turbulence ahead! If Web 2.0 is going to reinvent the world, expect it to reinvent the problems too If Web 2.0 is going to reinvent the world, expect it to reinvent the problems too The security world is changing, though The security world is changing, though The old paradigm was what might go wrong … The old paradigm was what might go wrong … Security economics gives us tools to think about what people might want things to go wrong, and metrics to measure what’s actually going wrong Security economics gives us tools to think about what people might want things to go wrong, and metrics to measure what’s actually going wrong

32. Krakow May 13th 2009 More … See www.ross-anderson.com for survey articles, our ENISA and Tibet reports, and my security economics resource page See www.ross-anderson.com for survey articles, our ENISA and Tibet reports, and my security economics resource pagewww.ross-anderson.com WEIS – Workshop on Economics and Information Security – UCL, June 24–5 WEIS – Workshop on Economics and Information Security – UCL, June 24–5 Workshop on Security and Human Behaviour – in Cambridge in 2010 Workshop on Security and Human Behaviour – in Cambridge in 2010 ‘Security Engineering – A Guide to Building Dependable Distributed Systems’ ‘Security Engineering – A Guide to Building Dependable Distributed Systems’

33. Krakow May 13th 2009