Presentation is loading. Please wait.

Presentation is loading. Please wait.

Implementing a Faceted Search Framework Emily Lynema & Andrew K. Pace NC State University Libraries ASIS&T Seminar April 9, 2007.

Published byBaldwin Ward Modified over 8 years ago

Similar presentations

Presentation on theme: "Implementing a Faceted Search Framework Emily Lynema & Andrew K. Pace NC State University Libraries ASIS&T Seminar April 9, 2007."— Presentation transcript:

1 Implementing a Faceted Search Framework Emily Lynema & Andrew K. Pace NC State University Libraries ASIS&T Seminar April 9, 2007

2 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Agenda The Context: Problem & motivation Local Implementation What and How? Challenges Encountered Outcomes Usage Statistics Future Opportunities

3 UNC School of Information & Library Science ASIS&T Student Chapter Seminar The Context

4 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Online Catalogs "Most integrated library systems, as they are currently configured and used, should be removed from public view." - Roy Tennant, CDL

5 UNC School of Information & Library Science ASIS&T Student Chapter Seminar What was the problem? Existing catalogs are hard to use: known item searching works pretty well, but … users often do keyword searching on topics and get large result sets returned in system sort order catalogs are unforgiving on spelling errors, stemming

6 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Catalog value is buried Subject headings are not leveraged in searching they should be browsed or linked from, not searched Data from the item record is not leveraged should be able to filter by item type, location, circulation status, popularity

7 UNC School of Information & Library Science ASIS&T Student Chapter Seminar What was the motivation? Unresponsive vendors (2001-2006) Some reading and writing SUNY Buffalo XML OPAC (2004) “My Kingdom for an OPAC” (Feb 2005) Some casual conversation (Jan 2005) Some formal conversation (Feb-June 2005) Organizational culture (all along) Fast implementation (July 2005-Jan 2006)

8 UNC School of Information & Library Science ASIS&T Student Chapter Seminar What’s the big picture? Improve the quality of the library catalog user experience Exploit our existing authority infrastructure (aka make MARC data work harder) Build a more flexible catalog tool that can be integrated with discovery tools of the future.

9 UNC School of Information & Library Science ASIS&T Student Chapter Seminar What is Endeca? Software company based in Cambridge, MA Search and information access technology provider for a number of major e-commerce websites Developers of the Endeca Information Access Platform

10 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Why Endeca? Customized relevance ranking of results Better subject access by leveraging available metadata (including item level data!) through facets Improved response time Enhanced natural language searching through spell correction, etc. Browse

11 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Local Implementation

12 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Demo

13 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Relevance ranking Based on locally customizable algorithm: Most relevant: query as entered For multi-term searches: phrase match Field match title match more relevant than notes match Other factors: number of fields matched weighted frequency (tf/idf) static ordering (publication date, circulation stats)

14 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Faceted navigation Combine search and browse in single interface (Guided Navigation™) Filter results across multiple facets Remove facets in any order

15 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Facet refinements Availability Author Library Format Language New LC Classification Subject: Topic Subject: Genre Subject: Region Subject: Era

16 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Added search tools Automatic spell correction “Did you mean…” suggestions Automatic stemming

17 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Implementation team Information Technology Team chair and project manager Technical lead ILS Librarian Technical manager Research and Information Services Reference librarian Metadata and Cataloging Cataloging librarian Digital Library Initiatives Interface development

18 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Implementation timeline License / negotiation: Spring 2005 Acquire: Summer 2005 Implementation: August 2005 : vendor training September 2005 : finalize requirements October 2005 – January 2006 : design and development January 12, 2006 : go-live date It doesn’t have to be perfect!

19 UNC School of Information & Library Science ASIS&T Student Chapter Seminar The nitty gritty Endeca co-exists with SirsiDynix Unicorn ILS and Web2 online catalog Endeca handles keyword search Web2 handles authority search and detail page display Endeca indexes MARC records exported nightly from Unicorn Index is refreshed nightly with records added/updated during previous day

20 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Technical overview Raw MARC data NCSU exports and reformats Flat text files Data Foundry Parse text files Indices MDEX Engine NCSU Web Application HTTP Information Access Platform

21 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Technical overview Raw MARC data NCSU exports and reformats Flat text files Data Foundry Parse text files Indices MDEX Engine NCSU Web Application HTTP Offline - Nightly

22 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Technical overview Raw MARC data NCSU exports and reformats Flat text files Data Foundry Parse text files Indices MDEX Engine NCSU Web Application HTTP Always Online

23 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Challenges – System design Identifying appropriate facets Integrating 2 independent data systems Unique identifiers are important! Designing the user interface Search page Results page

24 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Too many boxes, lines, and shaded areas. Elements for a single record not visually grouped.

25 UNC School of Information & Library Science ASIS&T Student Chapter Seminar First version of results page wireframe (~8 total iterations). Ideas drawn from OPAC, RedLightGreen, Amazon, etc.

26 UNC School of Information & Library Science ASIS&T Student Chapter Seminar 8 th (and Final) Revision: Aggregate holdings information by library. Reduces complexity of continuing and online resources. Brief view vs. Full view gives user choice about displaying holdings.

27 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Challenges - Data MARC data with MARC-8 encoding => Text data with UTF-8 encoding

28 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Fun with MARC MARC  flat text file(s) for ingest by Endeca. Transformation accomplished with MARC4J. Opportunity to manipulate data on the back-end.

29 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Transformed data

30 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Challenges - Data MARC data with MARC-8 encoding => Text data with UTF-8 encoding Data issues revealed by exposing metadata in facets Relevance ranking for bibliographic data

31 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Maintenance Little ongoing work required after deployment Quarterly data refresh from ILS Version upgrades 6 member product team meets monthly Lots of development ideas (as time / library priorities afford)! Loosely coupled = making changes twice

32 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Outcomes

33 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Relevance Are search results in Endeca more likely to be relevant to a user’s query than search results in old OPAC? 100 topical user searches from 1 month in Fall 2005 How many of top 5 results relevant? 40% relevant in Web2 OPAC; 31 no hits 68% relevant in Endeca catalog; 12 no hits

34 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Usage statistics

35 UNC School of Information & Library Science ASIS&T Student Chapter Seminar July 06 – Jan 07

36 UNC School of Information & Library Science ASIS&T Student Chapter Seminar July 06 – Jan 07

37 UNC School of Information & Library Science ASIS&T Student Chapter Seminar 19.4% Subj./Class July 06 – Jan 07

38 UNC School of Information & Library Science ASIS&T Student Chapter Seminar July 06 – Jan 07

39 UNC School of Information & Library Science ASIS&T Student Chapter Seminar July 06 – Jan 07

40 UNC School of Information & Library Science ASIS&T Student Chapter Seminar The Future

41 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Future opportunities Integrate catalog w/other tools through web services Enrich catalog through external web services: book jackets, reviews, etc. – Amazon/OCLC Build cross-application shopping cart functionality

42 UNC School of Information & Library Science ASIS&T Student Chapter Seminar The catalog & web services Initial impetus – 2 requests Can we have RSS feeds for the catalog? Can we integrate catalog results into library website QuickSearch? Initial plan Build RSS feeds and extend with OpenSearch for integration. Where did we end up?

43 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Introducing CatalogWS “A Web API for dynamically querying information from the NCSU Libraries Catalog” http://www.lib.ncsu.edu/catalog/ws/ Generic XML layer provides same functionality as HTML interface REST web API: define HTTP GET requests via URL parameters Enables server-side user-defined XSL transformations

44 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Why go there? More open access to the data available in our library catalog Core XML schema can be re-used and modified via stylesheets Enable other developers in the library to build applications using catalog data Reduce bottleneck (I don’t have to do everything)

45 UNC School of Information & Library Science ASIS&T Student Chapter Seminar RSS

46 UNC School of Information & Library Science ASIS&T Student Chapter Seminar QuickSearch

47 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Mobile device searching

48 UNC School of Information & Library Science ASIS&T Student Chapter Seminar Thanks NCSU project site: http://www.lib.ncsu.edu/endeca Andrew K. Pace Head, Information Technology andrew_pace@ncsu.edu Emily Lynema Systems Librarian for Digital Projects emily_lynema@ncsu.edu

Download ppt "Implementing a Faceted Search Framework Emily Lynema & Andrew K. Pace NC State University Libraries ASIS&T Seminar April 9, 2007."

Similar presentations

Metasearching: The Problem, Promise, Principles, Possibilities & Perils Roy Tennant California Digital Library.

Metasearching: The Problem, Promise, Principles, Possibilities & Perils Roy Tennant California Digital Library.
Endeca Taking a different path Cindi Holt Information Services Manager September, 2007.

Endeca Taking a different path Cindi Holt Information Services Manager September, 2007.
Getting Started with MarcEdit

Getting Started with MarcEdit
ICOLC October 4, 2001 OCLC Services. Purpose Libraries’ web-based information portal needs –Maximize consortia’s role in their members’ use of database.

ICOLC October 4, 2001 OCLC Services. Purpose Libraries’ web-based information portal needs –Maximize consortia’s role in their members’ use of database.
Chapel Hill 03-Mar-2006 Using Endeca for a Catalog Interface “So, yeah, the catalog sucks, but what are you going to do about it?” Andrew K. Pace Head,

Chapel Hill 03-Mar-2006 Using Endeca for a Catalog Interface “So, yeah, the catalog sucks, but what are you going to do about it?” Andrew K. Pace Head,
Summon: Web-scale discovery. Agenda Web-scale Discovery Defined How Summon Works Summon User Experience (live demonstration) Additional Resources.

Summon: Web-scale discovery. Agenda Web-scale Discovery Defined How Summon Works Summon User Experience (live demonstration) Additional Resources.
Endeca Statistics: An Online Set of Tools for UWF Stephen Fluharty Digital & Learning Technologies.

Endeca Statistics: An Online Set of Tools for UWF Stephen Fluharty Digital & Learning Technologies.
Catalogs for the Future Andrew K. Pace NCSU Libraries March 24, 2006 Library Automation: Yesterday’s Technology Tomorrow ILS Vendors: Squandering our money.

Catalogs for the Future Andrew K. Pace NCSU Libraries March 24, 2006 Library Automation: Yesterday’s Technology Tomorrow ILS Vendors: Squandering our money.
Opening the Door: using Endeca for a faceted catalog Emily Lynema NCSU Libraries MLC: Discovery & Access March 2, 2007.

Opening the Door: using Endeca for a faceted catalog Emily Lynema NCSU Libraries MLC: Discovery & Access March 2, 2007.
WWW Challenges : Supporting Users in Search and Navigation Natasa Milic-Frayling Microsoft Research, Cambridge UK SOFSEM 2004 January 28, 2004.

WWW Challenges : Supporting Users in Search and Navigation Natasa Milic-Frayling Microsoft Research, Cambridge UK SOFSEM 2004 January 28, 2004.
PRIMO AT THE ROYAL LIBRARY OF DENMARK Integrated search – Google of the library? Helsinki, October 28 2010.

PRIMO AT THE ROYAL LIBRARY OF DENMARK Integrated search – Google of the library? Helsinki, October
BC Integration of Systems and Resources MetaLib at Boston College Theresa Lyman Digital Resources Reference Librarian Boston College Libraries.

BC Integration of Systems and Resources MetaLib at Boston College Theresa Lyman Digital Resources Reference Librarian Boston College Libraries.
Information Retrieval in Practice

Information Retrieval in Practice
River Campus Libraries CUIPID Catalog User-Interface Platform for Iterative Development David Lindahl Director of Digital Library Initiatives River Campus.

River Campus Libraries CUIPID Catalog User-Interface Platform for Iterative Development David Lindahl Director of Digital Library Initiatives River Campus.
Searching TAL Online Developed by Northern Lights Internet Solutions Ltd. Advanced Searching.

Searching TAL Online Developed by Northern Lights Internet Solutions Ltd. Advanced Searching.
River Campus Libraries Metadata That Supports Real User Needs David Lindahl Director of Digital Library Initiatives University of Rochester Libraries.

River Campus Libraries Metadata That Supports Real User Needs David Lindahl Director of Digital Library Initiatives University of Rochester Libraries.
River Campus Libraries Metadata That Supports Real User Needs Jennifer Bowen Head of Cataloging University of Rochester Libraries David Lindahl Director.

River Campus Libraries Metadata That Supports Real User Needs Jennifer Bowen Head of Cataloging University of Rochester Libraries David Lindahl Director.
River Campus Libraries Metadata That Supports Real User Needs Jennifer Bowen Head of Cataloging University of Rochester Libraries David Lindahl Director.

River Campus Libraries Metadata That Supports Real User Needs Jennifer Bowen Head of Cataloging University of Rochester Libraries David Lindahl Director.
Relevance ranking of results from MARC-based catalogues: from guidelines to implementation exploiting structured metadata Tony Boston and Alison Dellit.

Relevance ranking of results from MARC-based catalogues: from guidelines to implementation exploiting structured metadata Tony Boston and Alison Dellit.
Overview of Search Engines

Overview of Search Engines

Similar presentations

Ads by Google