Presentation on theme: "Dinei Florencio and Cormac Herley WEIS’ 2011. We are curious about sex We are curious about politics We are curious about cybercrime And we satisfy some."— Presentation transcript:
We are curious about sex We are curious about politics We are curious about cybercrime And we satisfy some of these curiosities with surveys …
However… HOW BIG IS CYBERCRIME? “The spoils of cyber crime almost double in 2009. As a whole, losses totaled $560m," Patrick Peterson, Cisco Fellow . Cyber crime costs corporate America $10 billion every year!" . Damage caused by cyber-crime is estimated at $100 billion annually," Kilian Strauss, Organization for Security and Cooperation in Europe (OSCE) . Cyber-crime revenues are worth approximately $1 trillion," Edward Amoroso, CSO, AT&T (written testimony to the US Senate, March 17, 2009).
Surveys and errors… In a 1983 Federal Reserve survey, a single incorrectly recorded answer inflated the estimate of US household wealth by 1 trillion. In a 2006 FTC survey of ID theft, two respondents were excluded. Inclusion would have increased the estimate by 37 billion (3X)
Political Surveys Vote intention surveys seem reliable… Republican/Democrat What happens if a respondent lies?
Set membership vs Numeric surveys How many other lies to compensate a DEM/REP lie? How to compensate for a large, positive number, lie? How to keep the large number, when truthful?
Sex and Lies Particularly interesting sex survey: How many sexual partners of opposite sex have you had? SURVEY RESULTS: man report 2.5 to 9 times more lifetime partners than woman do. On a closed population, with M~W, the average should be similar for M/W. So, why the difference?
Sources of errors in Surveys Response error Non-response error Sampling error X Y R
Sex and lies SURVEY RESULTS: man report 2.5 to 9 times more lifetime partners than woman do. Possible explanations: Highly skewed distribution: (e.g., prostitution). Could explain some, but not all. Age difference of couple: Would increase the difference… Different definition of “partner”, or “sex”... Biased lies Men exaggerate Women reduce
Sex and Lies “the gender discrepancy appears to be driven primarily by [a few men reporting] large, “round” number of partners”. [Wiederman, 1997] On a prostitution survey, a respondent claimed 5401 partners in the last 6 months… [Brewer at al., 2000] So, could survey estimates of number of sexual partners be off by 5X due to a few liars??
The difficult of sampling heavytail distribution Cybercrime surveys Mean/median: 4 to 10 => highly skewed distribution. US Weatlh surveys US wealth mean/median: 4.3
Fraction of the wealth controlled by segments of the population . The top 1% and bottom 90% each control about one third of the wealth.
Instances of sample mean for a Pareto ( = 2:32; i.e., US wealth) distribution. There are 100 trials each with N = 1000 samples.
Cybercrime surveys Victims only a fraction of population (phishing: ~ 0.4%) On a 5000 people survey, 20 are victims. Mean/median higher than for US wealth. Need to sample more & better. Uniform sample may require well over 1 million respondents! Possibility of exaggeration on answers Risk of contamination from the 99.6% of non-victims
Conclusions “You should never trust user input”