1. EUBA: The Emory User Behavior Analysis System Eugene Agichtein, Qi Guo and Ryan Kelly Intelligent Information Access Lab http://ir.mathcs.emory.eduhttp://ir.mathcs.emory.edu Math & Computer Science Department Arthur Murphy, Selden Deemer, Kyle Fenton Emory Libraries

2. 2 Intelligent Information Access Lab http://ir.mathcs.emory.edu/ Goals/Motivation  Evaluate effectiveness of search and discovery with automatic behavioral metrics Perform aggregate and longitudinal studies  Develop tools for usability studies “in the wild” Scale (hundreds/thousands of “participants”) Realistic behavior and tasks On-demand playback of “interesting” sessions  Unified analysis/query framework for internal and external resource access and usage statistics Web-based query and statistics interface Access auditing, privacy, anonymity enforced

3. 3 Intelligent Information Access Lab http://ir.mathcs.emory.edu/ Approach: Client-side instrumentation  Implemented on top of the Emory Installation of the LibX Toolbar: (http://www.libx.org)http://www.libx.org  Extended LibX to track UI events: JavaScript patch to sample the mouse movements and other events on pre-specified web search pages. Events are encoded into a string and buffered, and periodically sent to the server (on internal library network).

4. 4 Intelligent Information Access Lab http://ir.mathcs.emory.edu/ Events captured (v0.5, Aug. 2008)  Button/link clicks/Url changes Name of the button, link, other meta-info  Mouse movements (x,y) coordinates sampled ~every 10ms  Scrolling Start, stop position, ~ every 10ms  Text entry, keypress (ctrl-c, ctrl-v) Query text, options changes  Menu item events Print, bookmark, save (all of them)  Hover over important elements  Mouse-in/out of browser

5. 5 Intelligent Information Access Lab http://ir.mathcs.emory.edu/ How it works  On login to Learning Commons, Firefox is started with http://irlib.library.emory.edu/consent.cgi?user=USERID If previously opted in (or out), goto homepage Else show consent form  Store user choice in database; if opted in, also store salted hash string for user login Can opted-in user behavior over “lifetime” No way to recover login id by dictionary attack Can be removed at any time by deleting mapping

6. 6 Intelligent Information Access Lab http://ir.mathcs.emory.edu/ How it works (2 of 3): Consent http://irlib.library.emory.edu/

7. 7 Intelligent Information Access Lab http://ir.mathcs.emory.edu/ How it works (3 or 3): which URLs?  For all visited URLs LibX notifies the server; information varies by type of site:  White list (search sites):  Black list (known private sites): Only domain name is saved All “https://” and “mail.*” URLs  White list (known search/discovery sites): EUCLID, Primo, Google, Google Scholar, Yahoo and Live search engines, Wikipedia All events captured  Gray list (search results and important public sites) Mouse moves and clicks (no keypress/text)  The rest: Only URL, button clicks, and menu items

8. 8 Intelligent Information Access Lab http://ir.mathcs.emory.edu/ Emory User Behavior Analysis System  Combines client side instrumentation, server-side caching, log management, querying, and analysis Client-side instrumentation, data mining/machine learning (Qi Guo) Log DB parsing, indexing, web-based interface for querying, playback, annotation (Ryan Kelly)  Plan: to release the system to research/library community (2009?)

9. 9 Intelligent Information Access Lab http://ir.mathcs.emory.edu/ EUBA Web-based analysis interface Prototype: http://ir.mathcs.emory.edu/library/private/index.pl user: test password: notsafe

10. 10 Intelligent Information Access Lab http://ir.mathcs.emory.edu/ Future Plans  Incorporate log data for ranking, discovery, query suggestion, collaborative filtering  Richer statistics and visualization  Streamline usability studies  Comments and suggestions welcome!