1. Unintended Consequences of Data Sharing Laws and Rules Sam Weber Software Engineering Institute, CMU

2. Thesis Laws and regulations concerning data sharing and privacy often have unintended consequences

3. Problem Space Three dimensions of issue: – What is ethical? – What is legal? Usually, laws are an attempt to codify ethical rules. – Utility What are people trying to accomplish? Often dimensions conflict: “No useful database can ever be perfectly anonymous, and as the utility of data increases, the privacy decreases.” Paul Ohm, “Broken Promises of Privacy: Responding to The Surprising Failure of Anonymization”

4. Common Issues Sharing of cybersecurity data – Can defenders collect/share personal information about people “for good”? Health care data – Who has what rights over a patient’s medical data? Patient, doctors and insurance companies all have different interests – What rules should protect people whose data is used for medical research? General – What is personally identifiable information? What is privacy? – What about data that has already “gone out”? – What about data that can be inferred from already revealed information? – How do deal with changed laws/technology? When laws/technology changes, what happens to existing databases? What happens in cases of international data sharing?

5. Unintended Consequences Three real-world situations: 1.Medical research and skull-stripping 2.Botnets and intrusion data 3.Burglar/Girls Around Me applications

6. Medical Research Biomedical research strongly dependent on access to health data – ex: brain scans for alzheimer’s research Want to protect privacy of people whose data is used – Two general approaches: Privacy/USA: generally industry-specific (HIPAA, Driver’s Privacy Protection Act,…) – HIPAA identifies Personally Identifying Information Privacy/EU: global, Data Protection Directive – PII “anything that can be used to identify you” – First is often ineffective, second is unstable

7. Skull Stripping and Biomedical Research HIPAA Privacy Rule regulations permit use/disclosure of data that have been removed of patient identifiers w/o authorization – Informed consent otherwise difficult to obtain/manage Problem: from MRI of head, can reconstruct face Solution: “skull stripping”/“defacing” algorithms (ex: Bischoff-Grethe et. al. “A technique for the deidentification of structural brain MR image”) Notice: NO real threat! – Probably entirely wasted effort 7

8. Botnets Situation: Researcher discovers botnet C&C on university’s machine – Allows botnet to continue to run, but observes it – Discovers how botnet works and finds ways to defeat it First question: Is this ethical? – Con: Researcher-controlled machine is knowingly attacking innocent people – Pro: Researcher isn’t making existing situation worse, and is in long-run making people more secure

9. Consequences Current botnet strategy – Take over victim machine, then cause it to do illegal action. Only do “real” activity if illegal activity took place – Effectively disables defenders who are bound not to allow illegal activity Strategy if defenders aren’t allowed to violate victim’s privacy: – Bind all command-and-control activity to PII of victims – Inhibit all data sharing!

10. Burglary App Researchers asked IRB for permission to write app that – Collected public photos from Flickr, Facebook and other places – Automatically located people who Had address of home discoverable Lived close to researchers Were currently on vacation more than 1000 miles away IRB granted complete permission (to researchers’ surprise) – Information is already publically available – Okay even to publically release application “Girls Around Me” app – Used information from foursquare, Facebook, etc to: Locate girls/boys currently physically close to user Display photos and bios of said girls/boys Is there any difference between two applications? – We know that privacy attacks are currently being done, but unpublished. If research is prohibited, then defenders are hampered

11. Conclusions Need to consider implications of policies – Threat models useful: What threats are you intending to counter? How will attackers respond to policy? – Need to consider utility/social good

12. BACKUP SLIDES

13. Dead Sea Scrolls Scrolls found in 1940s – access controlled by owners, majority still unreleased by 1990 Concordance prepared in 1950s: – Used for linguistic analysis – Alphabetical listing of words in document, along with words immediately before and after 1991: Wacholder and Abegg reconstructed scrolls from concordance 13

14. Legal/Ethical Issues Laws aren’t logical rules, conflict, ambiguous, change over time Laws – Privacy/USA: generally industry-specific (HIPAA, Driver’s Privacy Protection Act,…) HIPAA identifies Personally Identifying Information – Privacy/EU: global, Data Protection Directive PII “anything that can be used to identify you” – Network data: variety of wiretapping laws, etc Even if action is legal, may not be ethical – Laws often lag technology Constrain big data solutions – Given data from multiple sources, what are the applicable laws? What happens when laws change? What exact purposes can the data be used for? What are the restrictions upon the analyses that can be performed? – Real issue: certain experiments can be conducted at some US universities but not others, because of different IRB rulings. 14

15. Data Creation/Storage What data do you store, and how? – Store anonymized data? Create synthetic data? – What about data gathered with different anonymization requirements? When laws/attacks change, how do you recover? – What meta-data is needed about origin of data? 15

16. Anonymization Long history of de-anonymization work Inherent tradeoff between usability and privacy/security – Attacker models often unclear Potential solution: keep track of information already disclosed – Result: self-destructing databases 16