Taking the Library Back from Google Abe Lederman, President and CTO October 18-20, 2007

Abe Lederman Background Earned B.S. and M.S. Computer Science degrees, MIT Began work in information retrieval in 1988 Co-founded Verity Developed one of the earliest web-based search & retrieval applications, 1994 Pioneered “Deep Web” searching in 1999 Founded Deep Web Technologies, 2002

The “Google Myth” If you can’t find it on Google, the information doesn’t exist.

Why Do Students Like Google and Google Scholar? Quickly searches accessible information Easy to search Multi-Lingual support FREE

What Students Don’t know Links for many articles on Google Scholar require subscriber login or an access fee. Many libraries have already paid for subscriptions. Other databases may be more relevant to the topic. Google Scholar advanced search is not very powerful.

So Why NOT Google Scholar? “Scholarly content” intermixed with “non- scholarly” content Spotty coverage, requires publisher cooperation Poor relevance (citation counts don’t seem to help) Unable to limit searching to specific sources Difficult to find the needles in the haystack

Bringing Relevance Back to the Library Checking the Library Website… Reference Libraries Deep Web Databases Specialized Search Directories Specialized Search Engines Subscription Databases Library Catalogs “Ask A Librarian!”

Federated Search… Allows you to search, retrieve, and merge de-duped and relevantly ranked results from multiple high-quality information sources.

In Other Words… One Search, Many Sources Library Catalog Wikis Subscription Sources Public Web Sources Blogs Subscription Sources

Federated search by any other name…. Metasearch 6,690,000 Meta search 1,840,000 Federated Search339,000 Distributed search237,000 Broadcast search106,000 Distributed information retrieval 70,200 Deep Web Search24,600 * Number of occurences on Google (Sept. 21, 2007)

Tennant’s Tenets Roy Tennant: Internationally known speaker and writer on library and information technology issues. Only librarians like to search, everyone else likes to find. All things being equal, one place to search is better than two or more. Services should be placed as close to the user as possible.

Tennant’s Tenets (cont.) “ Good enough” is often just that. Our ability to create effective one-stop searching is dependent on our ability to appropriately target user needs. The size of the result set doesn’t matter as much as how the results are presented. (‘the Google Lesson’)

Increase efficiency of information search information search Gather the most relevant, important information important information Make better decisions & fewer mistakes More Users, Find More Information How can Federated Search help your Patrons?

Benefits of Federated Search One-Stop Access Support of More Efficient Research Consolidated, Formatted Results Authentication and Access Control

One-Stop Access Single query search across multiple disparate sources in real-time: Subscription services (journals, bibliographic databases, etc.) In-house databases Unstructured data (white papers, theses, reports, etc.) Catalogs (web-based OPAC) Publicly available websites Information Discovery

Support of More Efficient Research Results across all selected sources are: Merged Presented in relevance-ranked order Sorted Clustering

Consolidated, Formatted Results Filtering and removal of duplicates Ranking by relevance Displaying results by source Permitting results to be Downloaded Printed Automatically ed using Alerts Flagging of results from subscribed sources

Authentication and Access Control Authorization based on: IP address User ID and password Implementation using: Proxy server Management of session cookies Different access levels for different user groups Virtual server support

#1 Benefit to Libraries Improve your ROI on Licensed Content!

Integrating Trusted Science + Technology Research What does Federated Search look like? 15 leaders in science and technology research 3 million documents 150 years of knowledge

Bringing Federated Search to Your Patrons

Clearly Define Requirements Compile your list of sources to federate Access to Internal Sources Licensed Product vs. Managed Solution Staff Resources Timelines Determine features important to users

Will the Real Federated Search Please Step Forward… Easy-to-Use, Customizable Interface Incremental display of real-time and indexed results Clustering or Visualization Relevance ranking of results on-the-fly Support Boolean and fielded searching Access sources via multiple protocols (XML gateway, HTTP, SR/U, SR/W, Z39.50)

Sample Features Checklist Integration with URL Resolver Integration with EndNote or other export capabilities Spell Checker Alerts Options Export to an RSS Feed Advanced Customizability

Vendor Evaluation Demonstrations and Pilots User reaction Ease of search Fulfillment of requirements and features Relationship Future Relationship Flexibility

The Future of Federated Search Multi-Lingual Searching Personal Libraries Automated Source Selection

Conclusion Help Patrons to… Improve their search Find what they need Access scholarly sources Return to your library for information

Thank You! Abe Lederman