Anonymizing Web Transaction Logs to Ensure Privacy and Increase Usability Paul A. Soderdahl University of Iowa Libraries ILA/ACRL Spring 2003, Dubuque, IA May 2, 2003

Description of the Problem Iowa Code §22.7 The following public records shall be kept confidential… 13. The records of a library which, by themselves or when examined with other public records, would reveal the identity of the library patron checking out or requesting an item or information from the library….

Description of the Problem University Libraries User Privacy Policy The Libraries will not reveal the identities of individual users nor reveal the information sources or services they consult unless required by law…. The Libraries from time to time may aggregate and retain user data for a reasonable period of time in order to investigate the use or value of resources and services. It will, however, neither collect nor retain information identifying individuals except during the period when and only for the purpose that such record is necessary to furnish a specific service.

Description of the Problem University Libraries User Privacy Policy Publicly Accessible Digital Information Systems The Libraries’ computer-based access systems (e.g., InfoHawk or various digital information systems) frequently track or "log" the actions performed by users of those systems. Transaction level logging that can be tied to individuals may be kept intact for a limited period of time for trouble-shooting and problem resolution related to system functions and service transactions. During the period this information is retained, it is held in confidence and is not shared with third parties unless required by law….

Description of the Problem University Libraries User Privacy Policy Publicly Accessible Digital Information Systems When the information is no longer useful, by a reasonable standard, for resolving problems, the Libraries may aggregate and retain anonymized user data in order to investigate the use or value of resources and services. Information regarding individual identities (or the source of the transaction) will be removed. Original transaction logging information that has been processed in this way will be destroyed and care taken to ensure that backups or other inadvertently stored forms of the data are not retained.

Description of the Problem Web transaction log data 1.Date and time 2.Client IP address 3.Client username (if logged in) 4.HTTP method used (usually GET/PUT) 5.URL requested 6.Parameters passed to URL (everything after the question mark) 7.HTTP status code 8.# of bytes server sends to client 9.# of bytes client sends to server 10.Length of transaction 11.Client software used 12.Any cookie client passed to server 13.URL of referring page

Sample Entry Server: Library Explorer :21: GET /ch1/subjectsearch/p_medicine.htm explorer.lib.uiowa.edu Mozilla/4.0+(compatible;+MSIE+5.01;+Windo ws+NT+5.0;+.NET+CLR ) - stand&hl=en&lr=&ie=UTF-8&start=10&sa=N

Sample Entry Server: PURL :26: GET /wiley/BioEssays purl.lib.uiowa.edu Mozilla/4.0+(compatible;+MSIE+6.0;+Window s+98) - www/ejrnl.html

Sample Entry Server: Intranet :50: soderdhl GET /infohawk/Staffdirectory.htm Mozilla/4.0+(compatible;+MSIE+5.01;+Windo ws+NT+5.0) -

Let’s Get Real How much can you tell from ? So what that a would-be terrorist looked at our overdue fines? Are web transaction logs that sexy?

Web Usage Reports Does the individual workstation information provide any valuable data? What do we really want to know about our web server usage?

Web Usage Reports H.I.T.S. – How Idiots Track Success Look for trends Document activity

Web Usage Reports Visitor profiles –On campus vs. off campus, etc. –Public workstation usage Reduce noise –Automated tests –Robots

Web Usage Reports proxy library public workstations residence halls computing labs dial-up ISU UNI ICPL AOL robots charlotte on campus UI affiliate off campus

IP Translation At end of each month, modify log file Strip out IP address information Replace with pseudo-DNS –Fake domain names based on interest

IP Translation Instead of DNS lookup:  mac107.civil.northwestern.edu  dhcp80ff991b.dynamic.uiowa.edu Replace with pseudo-DNS:  mnpub07.lib-public-uiowa.edu *  anonymous.itc-uiowa.edu *.*.*.*  anonymous.unknown.com

Examples Sample report before IP translationSample report before IP translation Sample report after IP translationSample report after IP translation

Examples

IP translation table Perl script

Weaknesses Lose distinctions:.edu,.gov,.com,.mil Lose foreign country usage

Improvements #1: Pseudo-DNS based on interest –lib-public-uiowa.edu –residence-rooms-uiowa.edu #2: DNS lookup and re-anonymize –anonymous.mil –anonymous.uk

Questions