1. User Search Behaviors: Insights from Easy Search Transaction Log Analyses William Mischo, Mary Schlembach, Jason Heldreth, Avinash Kumar Grainger Engineering Library Information Center University of Illinois at Urbana-Champaign LIS Methods Round Table April14, 2015

2. Easy Search Custom Transaction Log Analysis Rich literature on TLA We record user actions, system suggestions, search reformulations, and clickthroughs in custom logs 2010-2011 study of 1.4 million searches and 1.5 million clickthroughs 2014 study of 1 million searches, 1.1 million clickthroughs over 10 month period May 2013 to March 2014 Initially supported by NSF and IMLS Background: OPAC TLAs have provided ambiguous results; web search engine behaviors different

3. Examples Competent Jerks, Lovable Fools, and the Formation of Social Networks The virtual supermarket: An innovative research tool to study consumer food purchasing Mothers and fathers : a study of the development and negotiation of parental behaviour Matouschek, Kellis, Serrano, Fersht Nature Kinsella and Phillips 2005 Hemenway, D. (2010). Why We don't Spend Enough on Public Health. New England Journal of Medicine, 362(18), 1657 university, need for money J. Quant. Spectrosc. Radiat. Transfer.

4. 2014 Transaction Log Analysis Breakdown of searches from the main Gateway: – Easy Search and associated tabs: 902,420 76.5% the gateway Easy Search default tab: (55.1%) – advanced search: 56,170 4.8% – departmental library page searches: 158,750 13.5% – myeasysearch: 18,159 1.5% – search suggestion searches: 42,708 3.6% Total: 1178207 4

5. 2014 Transaction Log Analysis Gateway searches total: 902,420 – Easy Search tab: 649,320 – Books search tab: 106,375 – Article tab: 76,375 – Journal Title tab: 70,350 2007: 49.4% of all searches were known-item (KI) 2011: 51.2% of searches in 54% of sessions were KI 2014: 57.7% of searches were KI 5

6. 2014 Transaction Log Analysis Sample of 974,137 user-entered searches: – average words per query is 5.11 words – This is up from earlier studies: 3.58 words per query in 2008; 3.76 words per query in 2009; 4.33 words per query in 2011 93,642 one word, 158,054 greater than 8 words 49.6% were 3 words or less, 29.4% are 6 words or more longest search was 299 words 6

7. 2014 Transaction Log Analysis 20.98% of all searches in 29.13% of all sessions used a search assistance suggestion or custom added search result link 57.2% of the 489,272 search sessions were single query sessions 6% of the sessions contain 6 or more queries, 1,157 sessions contain more than 20 queries, and 96 search sessions contain more than 50 queries 7

8. Search Assistance Features Spelling suggestions Redo as author search prompt Direct links to frequent searches and Libguides Links to matching e-journal title(s) Links to citation linker for full-text retrieval Limit results to exact phrase, title word(s), or title phrase Dark background searches

9. Additional Search Assistance Features Provide list of related terms System performs additional phrase and title searches within selected resources Displays e-journal and database list results Pass-through command language search Ask-a-Librarian live link Additional target searches are performed based on user query Links to e-book titles displayed on OPAC results line

10. Search Characteristics 2010 Characteristics of User SearchesPercentage of all Searches Boolean Operators (“and”, “or”, “not”) entered 13.03% AND operator searches (the vast majority of ANDs appear as conjunctions in known titles) 12.76% AND operator in all caps 0.41% OR operator-- lower and upper case entered 0.22% NOT operator-- lower and upper case 0.13% Commas entered 6.03% Parentheses entered 0.88% Quotes (single and double) entered 3.95% Prepositions entered 27.26% “+” sign entered 0.42% Truncation symbol(*) entered 0.12%

11. Clickthroughs by Category 2010 Clickthroughs into e-book content targets were 7.87% of all target clicks in 11.36% of all sessions User Clickthroughs by CategoryPercentages Article databases 47.65% All books 33.01% (OPACs 26.01%, E- Books 7%) Journal/Database titles 17.41% Web search engines 0.72% Newspapers and news sites 0.87% Reference titles 0.35%

12. Primo Clickthrough Study May-June 2014 Primo clicks 15,068 times between 05-01-2014 and 06-21-2014. Total of 147,326 clickthroughs during that time Looked at a sample of 478 of the 15,068 searches 245 known-item searches (51.8%) and 228 topical searches (48.2%) Of the 245 known-item searches, 159 were judged successful - they brought back the expected results in the Primo first page or as the top item 12

13. Primo Clickthrough Study (2) Of the 159 ‘successful’ searches, all of them were also successful in Scopus, Ebsco, CrossRef, WorldCat Discovery, Web of Knowledge, VuFind, IShare, Google, or Google Scholar searches Removing the Google results left 9 searches that were successful in Primo but not via Scopus, Ebsco, Web of Science, CrossRef, VuFind, IShare, or WorldCat Discovery. Removing CrossRef and WorldCat Discovery resulted in 35 searches successful in Primo but not successful in Scopus, Ebsco, Web of Science, VuFind, or IShare 13

14. Primo Clickthrough Study (3) Found that CrossRef and WorldCat Discovery good at retrieving successful search results for full citation searches 48 known-item searches that were unsuccessful in Primo but successful in one of the other Easy Search targets - Scopus, Ebsco, Web of Science, CrossRef, VuFind, IShare, WorldCat Discovery, or Google/GS 14

15. Primo Clickthrough Study (4) Evaluated sample against Arizona State’s Summon. In one case, a known-item search that was unsuccessful in Primo yielded a first page match in the AZU Summon. However, there were 50 known-item searches that were successful in our Primo that were unsuccessful (item not on top or first page) in the AZU Summon 15