1. Studying SFX Logs to Better Understand User Behavior Bennett Claire Ponsford, Digital Services Librarian Anne L. Highsmith, Consortia Systems Coordinator Texas A&M University Libraries

2. Texas A&M University  46,000-plus undergraduate and 8,500 graduate students  250 degree programs in 10 colleges  2,800 faculty in a research-intensive environment  Branch campuses in Galveston, Texas and Doha, Qatar

3. University Libraries  Member of ARL  Main library with 5 branches  3 in College Station  1 in Galveston, TX  1 in Doha, Qatar  Spending over $7 million per year on electronic resources  Redesigning our website anyway

4. Our Current SFX Implementation  Went live with SFX 2 in May 2004  SFX 3 in January 2006  A-Z List in January 2006  A-Z List ver. 3 in January 2008  Current SFX menu design unchanged since we went live

8. Why study users?  To see how your users search when you’re not watching  To resolve internal disagreements over default features to include, etc.  To see whether changes to SFX menus really improved results  As a counterpoint to focus groups and task- based user testing

9. What do our users say?  Hated all the pop-up windows  Pop-up windows in general  Highjacking previous content  SFX menus too busy and confusing  Did not understand the 3 catalog links  Never used the FAQ or Ulrich’s link  All they want is full text anyway

11. What do the numbers say? Sample from the SFX logs – 14 days per year – 3 years, FY 2006 – as much of FY 2008 as available Sample from the apache logs – Data available only for FY 2008

12. Where are they coming from? Fiscal Year 08 CountPercent From Catalog to API58608.54% From AZ list to just SFX Menu12281.79% From AZ list to Full Text1599523.32% From MetaLib1409520.55% From Databases3113445.39% Total Sample Requests68585100.00%

13. Where do they go? FY08 OverallCountPercent Total Sample Requests68585100.00% Requests with Full Text3062644.65% - Clicked On2667987.11% - Not Clicked on594619.41%

14. What if there is no full text? FY08 OverallCountPercent ILL only243816.81% LibCat (main catalog) only229915.85% Search Google only4493.10% LibCat & ILL2871.98% LibCat & Chiron (medical catalog)2511.73% Chiron only1951.34% All 3 catalogs870.60% All 3 catalogs and ILL700.48% ILL for inadequate metadata570.39% LibCat & Search Google560.39% LibCat, Chiron, & ILL400.28%

16. Full Text from the Catalogs

18. Where do they go? FY08 from Catalog to APICountPercent Sample requests from Catalog to API5860 No Full Text Found398768.04% Full Text187331.96% - Clicked on Full Text (once or more)170190.82% - Clicked on Full Text (more than once)924.91% - Clicked on Full Text (total clicks)1802 - User Did Not Click on Full Text button47625.41%

19. Requests over Time

20. Full Text Availability and Actions

21. No Full Text Actions

22. Public/Library Usage CatalogSFX

23. Overall Requests: Public/Library

24. Clicking on Full Text: Public/Library

25. No Full Text Behavior: Public/Library

26. Requests from AZ List : Public/Library

27. Review of Apache Logs Sample of titlesType administrative science quarterlyJournal title adobeSubject? adolescenceJournal title adolescent literatureSubject? adolescentsSubject? Adopotion of biotechnologySubject? adult children homeSubject? adult children with parents at homeSubject? adult educationJournal title Adult LearningJournal title adv exp med biolJournal title

28. What Next: SFX Menus?  Redesign SFX Menus using simplified menus  Just display full text, if available, in basic section  Decrease all the verbiage  Reduce duplicate listings with display logic  Display catalog links only if holdings available  Experiment with direct link banner option  User test changes

29. What Next: Home page?  Search box from Libraries home page  Review apache logs re: size of problem  Wording changes/help text

31. Technical section -- outline Characteristics of stat tables How statistics are gathered and stored Characteristics of Apache logs Modifications to Apache logging to facilitate stats Statistical sample How you can do this too

32. Characteristics of stat tables (1) 3 stat tables (& offline equivalents) – stat_object – stat_target_service – stat_repeatables Request has 1 entry in stat_object table Tables join on request_id Request has multiple entries in stat_target_service table – 1 entry (row) for each link on menu

33. Characteristics of stat tables (2) Exceptions to “request in stat_object table has several corresponding records in stat_target_services” – API requests - 0 entries in stat_target_service – Click on any type of link where direct linking occurs – 1 entry in stat_target_service

35. Stat object data elements (1) Name ----------------------- REQUEST_ID ISSN ISBN LCCN LOCAL TITLE ATITLE JTITLE BTITLE CTITLE SERIES PUBLISHER PLACE_OF_PUBLICATION OBJECT

36. Stat object data elements (2) Name ----------------------- SUBCATEGORY STATUS DOI REQ_DATE TIME SOURCE IP OBJECT_TYPE INSTITUTE USER_GROUP FACULTY HAS_FULLTXT DATE_OF_PUBLICATION EPAGE SPAGE PRESENTATION_FORMAT SESSION_ID OPEN_URL

38. Rows in target service table Target Clicks Service INFORMAWORLD_JOURNALS 1 getFullTxt METAPRESS_ROUTLEDGE 1 getFullTxt AM_VOYAGER 1 getHolding MS_VOYAGER 0 getHolding GA_VOYAGER 0 getHolding WWW_SEARCH_ENGINES 0 getWebSearch AM_PROBLEM_REPORT 0 getWebService AM_SFX_FAQ 0 getWebService ULRICHSWEB_COM 1 getCitedJournal

39. How stats are gathered & stored (1) Run online to offline daily Run export_tab.pl monthly - Embedded in special perl script that copies monthly cumulations to report server Copy stat_object_offline & stat_target_service_offline, but not stat_repeatables_offline

40. How stats are gathered & stored (2) Copy these tables in their entirety, except for some open_urls in stat_object Perl script on report server loads data into Oracle tables Create separate tables by academic year because of size -- academic 2008 to date: – 1.3M+ requests – 8.4M+ target service links

41. Characteristics of Apache logs (1) “Real” ip associated with request available only from reverse Apache log Logs that span long time period – Beware of differences between v2 and v3 a-z lists – Note if you have changed display options, e.g. between brief and detail view

42. Characteristics of Apache logs (2) Certain data available only from logs, because it doesn’t generate a “request” or hasn’t generated a request yet. – Category search – Auxiliary functions Use of info button on az list Push to Metalib myspace from az list Opening SFX az list from within Metalib

43. Modifications to Apache logging Set up custom logging statement in httpd.conf – Write the data in a single record – Store tabs between data elements within record

44. Samples STAT table samples – 2 weeks worth of data across 3 years – Same day of week / same week of month 1 st Thursday in January, 2 nd Friday in March Apache logs – Academic 2008 was only year available with sufficient data elements – Selected representative days from that year

45. DIY instructions (1) Run online2offline job (server_admin_util) Run /exlibris/sfx_ver/sfx_version_3/[instance]/ad min/database/export_tab.pl – By default writes *.exp file to scratch directory – Must run once for each table Download *.exp files & load into MS Access

46. DIY instructions (2) Go to http://lib.tamu.edu/directory/ahighsmi and click on link for this presentation to download zip file.http://lib.tamu.edu/directory/ahighsmi Zip file contains – Copy of this presentation – Sample MS Access databases – Perl program to parse Apache log entries – Custom Apache log format for httpd.conf

47. Contact info Bennett Claire Ponsford – bennett.ponsford@tamu.edu bennett.ponsford@tamu.edu – http://lib.tamu.edu/directory/bponsfor http://lib.tamu.edu/directory/bponsfor Anne L. Highsmith – hismith@tamu.edu hismith@tamu.edu – http://lib.tamu.edu/directory/ahighsmi http://lib.tamu.edu/directory/ahighsmi