Federated Searching Pre-Conference Workshop - The federated searching cookbook Qin Zhu HP Labs Research Library February 18, 2007

The Basics: what you need to know about federated searching

Outline 1.What is federated searching? 2.Why to use federated search? 3.Research on federated searching 4.Federated search tools in the market place 5.How to choose a federated search tool 6.Our approach

What is federated searching? Federated searching consists of transforming a query and broadcasting it to a group of disparate databases with the appropriate syntax, merging the results collected from the databases, presenting them in a succinct and unified format with minimal duplication, and allowing the library patron to sort the merged result set by various criteria. –As described by Peter Jacso, (2004), WikipediaPeter JacsoWikipedia

Federated search in a nutshell – User experience -- From Serials Solutions Central Search PageCentral Search Page

Federated search in a nutshell – System components Federated search engine Authentication and Access Resource 1 Resource 2 Resource 3 Resource 4 Resource 5 OpenURL – Link resolver ILL/ Document delivery

Why to use federated search? -- What is good about federated search… Google like, “one stop searching” A good starting point for unknown research Users don’t have to learn multiple resource interfaces Increase awareness of available resources Increase library resource usage statistics Preselected resources Can be used for subject page Can be used for initial user education

Federated search engine What you may not know about federated search Search results download time Resource licensing issues Resource special search methods Resource native search Search complexity

Research on federated searching A lot of research has been done in this area –Information seeking –User behavior –Usability –Information service –Information retrieval –Information system –System computing –Metadata management

Research on federated searching Search results from a federated search tool: hits Search results from Google: hits Search results from Google Scholar: 918 hits

Federated search tools in the market place Library of Congress Portals Applications Issues Group –Federated Search Products and Vendors

Federated search tools in the market place ILS vendors Electronic resource solution providers Federated search specialized firms Library resource providers Search solution firms Search engines

How to choose a federated search tool Develop a list of criteria –Features –UI –Technology –Hosting solutions –Available connectors –Authentication methods –OpenURL standard compliance –Implementation time –Develop and support custom connectors –API tools –Vendor staff knowledge about federated searching

Our Approach We did information seeking study for our library users. –Users like Google type search We did research of federated searching and federated search tools. –Identified candidates We developed a list of criteria for federated search tool. –Federated search tool metric –A list of questions to vendor We contacted vendors and had product demos. –Remote and on site demos

Federated search tool metric

Our Approach We checked with vendors for connector availability for our resources. We did pilot test for federated search tools. –Hosting solutions –Authentication and Access –UI –Customization –System stability

Implementation: Serials Solutions Central Search

Outline 1.Resources for Central Search 2.Authentication and access 3.Customization 4.UI

Resources for Central Search Configure resources for Central Search –Connector availability –Description for each resource –Subject category for each resource –Default URL for each resource

Authentication and Access Serials Solutions will assign a specific IP for Central Search instance For IP authenticated resources, the resource vendors need to be notified with the additional IP address.

Customization Customize label –Database  Resource Default search method –Title  Any Customize message

UI Customization - Header and footer

The Advanced: Integration

Outline 1.Article Linker 2.Vivisimo clustering 3.Internal authentication and tracking system 4.Library website

Integrating with Article Linker

Clustering and federated searching What is clustering? Why to have results clustering under federated search?

Vivisimo Clustering and Central Search Authentic- ation and Access Resource 1 Resource 2 Resource 3 Resource 4 Resource 5 Result 1 Result 2 Result 3 Result 4 Result 5 Result 6 Result 7 Result 8 Result 9 … Category 1 >> sub-class 1 Category 2 >> sub-class 1 >> sub-class 2 Category 3 Category 4 …

Vivisimo Clustering Configure Vivisimo clustering Customization –Label –UI Custom CSS file for Central Search and clustering look & feel

UI Customization - Vivisimo Clustering

Internal authentication and tracking system Link to user data, such as user base Check and register user upon user base Grant permission to the resource

Integrating with Internal authentication and tracking system Central Search search result URL (with DB metadata) Pass to a cgi-bin program to check the DB code Map the DB code to our internal system service code Check user information Check restriction for the service Record usage Grant access to user

Integrating with library web site Add combination search box to library home page –Google/Yahoo style search box –Combination search for Central Search or Site Search –Passing search query between Central Search and Site Search Add combination search box to starting point page

Integrating with library web site

Let’s take a look at how things work together.

Further Developments

More connectors for resources Able to add custom links Central Search API Federated alerts Usage A way to search resources inside the firewall

Questions & Answers

Thank You!