NCSU Libraries Andrew Pace & Emily Lynema NCSU Libraries May 24, 2006

Technical Overview Endeca Information Access Platform co- exists with SirsiDynix Unicorn ILS and Web2 online catalog. Endeca Information Access Platform co- exists with SirsiDynix Unicorn ILS and Web2 online catalog. Endeca indexes MARC records exported from Unicorn. Endeca indexes MARC records exported from Unicorn. Index is refreshed nightly with records added/updated during previous day. Index is refreshed nightly with records added/updated during previous day.

Endeca IAP Overview Raw MARC data NCSU exports and reformats Flat text files Data Foundry Parse text files Indices MDEX Engine NCSU Web Application HTTP Client browser HTTP Endeca Information Access Platform

Endeca IAP Overview Raw MARC data NCSU exports and reformats Flat text files Data Foundry Parse text files Indices MDEX Engine NCSU Web Application HTTP Client browser HTTP Offline - Nightly

Endeca IAP Overview Raw MARC data NCSU exports and reformats Flat text files Data Foundry Parse text files Indices MDEX Engine NCSU Web Application HTTP Client browser HTTP Always Online

Integrating Endeca Endeca doesn’t understand MARC data / MARC-8 character encoding – translate to UTF-8 text files Endeca doesn’t understand MARC data / MARC-8 character encoding – translate to UTF-8 text files Each night a script updates the data indexed by Endeca: Each night a script updates the data indexed by Endeca: – Exports updated or new MARC records from Unicorn. – Reformats and merges these records with those already indexed. – Starts Endeca re-index – completely rebuilding index for the catalog. Process requires about 7 hours. Process requires about 7 hours. Retain Web2 OPAC for some functionality Retain Web2 OPAC for some functionality – Authority searching - known items and cross-references – Detailed record pages – how to make Endeca -> Web2 link?

Integrating Endeca - Future MarcAdapter plugin for raw MARC data. MarcAdapter plugin for raw MARC data. – Create local field mappings and special handlers in Java. – Eliminate need for external MARC 21 translation and file merging. Partial Updates Partial Updates – Update circulation data multiple times throughout the day.

Quick Demo

Some Search Statistics (March 2006)

Some Navigation Statistics (March 2006)

Navigation Statistics (II) (March 2006) DimensionRequestsOrder (on page) LC Classification Subject: Topic Library Format Author Subject: Genre Subject: Region Language86539 Subject: Era74518 Availability67901

Other interesting tidbits… (March 2006) Authority searching decreased 45% Authority searching decreased 45% Keyword searching increased 230%. Keyword searching increased 230%. – Caveat: default catalog search changed from title authority to keyword. ~ 6% of keyword searches offered spelling correction or suggestion ~ 6% of keyword searches offered spelling correction or suggestion – 3.6% - automatic spell correction – 2.6% - “Did you mean…” suggestion

Usability Testing 10 undergraduate students 10 undergraduate students – 5 with Endeca catalog – 5 with old Web2 OPAC Endeca performed as well as OPAC for known-item searching in usability test Endeca performed as well as OPAC for known-item searching in usability test – 89% Endeca tasks completed ‘easily’ (8/9) – 71% OPAC tasks completed ‘easily’ (15/21) Endeca performed better than OPAC for topical searching in usability test. Endeca performed better than OPAC for topical searching in usability test.

Topical Searching Tasks

Average Topical Task Duration

Usability Testing Trends Relevance *most* important Relevance *most* important – “Once I scroll through a page, I get pretty discouraged about the results...” Web2 OPAC participant looking for resources on cat health ‘Keyword’ term less intuitive / trusted than ‘Subject’ and ‘Title’ ‘Keyword’ term less intuitive / trusted than ‘Subject’ and ‘Title’ – “[I used] Keyword in Title because that’s what I want the book to be mainly referring to. But I also could’ve went Keyword in Subject. But if I’d have went Keyword Anywhere it would have had too big of a field to look through.” Web2 OPAC participant looking for resources on gene therapy When found, dimensions seem intuitive and useful When found, dimensions seem intuitive and useful ‘Did you mean’ seems intuitive ‘Did you mean’ seems intuitive

A study in relevance Are search results in Endeca more likely to be relevant to a user’s query than search results in Web2 OPAC? Are search results in Endeca more likely to be relevant to a user’s query than search results in Web2 OPAC? 100 topical user searches from 1 month in fall topical user searches from 1 month in fall 2005 How many of top 5 results relevant? How many of top 5 results relevant? – 40% relevant in Web2 OPAC – 68% relevant in Endeca catalog

Relevance defined Relevance ranking in Endeca – select from a variety of modules and order them based on importance. Relevance ranking in Endeca – select from a variety of modules and order them based on importance. Relevance most important in Keyword Anywhere - searches all fields. Relevance most important in Keyword Anywhere - searches all fields. At NCSU… At NCSU… 1.Original query term(s) (no thesaurus, stemming, spell correction) 2.Exact phrase match 3.Field ranking (Title higher than Author higher than Table of Contents) 4.Number of fields that contain term(s) …

Future Plans Ongoing tweaks: Ongoing tweaks: – Continued usability testing – Relevance ranking algorithms & spell correction thresholds – Additional browsing options Endeca 2.0 ideas Endeca 2.0 ideas – FRBR-ized display – Discussions with OCLC regarding FAST (Faceted Access to Subject Terms) and FRBR – Patron-generated refinements (folksonomies?) – Enrich records with supplemental Web Services content – more usable TOCs, book reviews, etc. – The death of authority searching (?) – More integration with QuickSearch, other data repositories, and third-party discovery tools

Thanks Andrew Pace, Head, IT Emily Lynema, Systems Librarian for Digital Projects