Abe Lederman, President and CTO Deep Web Technologies 2008 STIP Working Meeting, April 23, 2008 Federated Search: The Technology For Making Global Discovery a Reality

Who We Are… Founded in 2002 Headquartered in Santa Fe, New Mexico 20 Employees Over $2 million in R&D through DOE SBIR program DWT is focused on providing state-of-the-art federated search products and solutions which search, retrieve, aggregate and analyze content from web-based databases.

Abe Lederman-Background Earned B.S. and M.S. Computer Science degrees, MIT Began work in information retrieval in 1988, Verity Consulted to LANL, Developed some of the first web-based applications that searched text-based content, 1994 Pioneered “Deep Web” searching in 1999, OSTI

Some of Deep Web’s Customers DOE Office of Scientific and Technical Information Department of Defense National Agricultural Library Scitopia.org The Intel Corporate Library Stanford University

DWT-OSTI Partnership Present

What is Federated Search? Federated Search is an application or service that allows a user to submit a search in parallel to multiple, distributed information sources and retrieve aggregated, ranked and de-duped results.

In Other Words… One Search, Many Sources Library Resources Technical Reports Subscription Sources Public Web Sources Conference Proceedings Journal Articles

Benefits of Federated Search One-stop access to multiple information sources Users don’t need to know where/how to search Saves researcher time and money Improves utilization of information sources Consolidated, ranked and de-duped results Important results are not missed

Speeding up the Diffusion of New Scientific Knowledge Global Discovery 2

Knowledge Diffusion in Action Math Community Biology Community Physics Community Global Discovery Search Portal Mathematician’s Scientific Discovery Biology Researcher’s Scientific Discovery Physics Scientific Discovery Math Databases: Research Papers Correspondence Conferences Physics Databases: Research Papers Correspondence Conferences Biology Databases: Research Papers Correspondence Conferences

Challenges of Global Discovery Identify, configure and maintain access to reliable information sources Automatically select sources to search Perform hundreds or thousands of searches in parallel Rank and organize results retrieved Goal: Search 1000 sources by February 2009

Global Discovery Search Portal NASA TRS USGS Agricola Pub Med SPIE IOP IEEE APS Vascoda SciELO DEFF ETOC Eureka Science News Eureka Alert! Science News Online Science Daily Science News Portal Nuclear fission

Nuclear fission

Using Federated Search at your DOE Lab Internal Sources Public sources including Science.gov worldwidescience.org Subscription Sources Biology & Medicine Math Engineering Computer Science Nuclear Physics Geosciences Chemistry Renewable Energy Workstation

Polar bears

Turning Vision into Reality Thank you! Abe Lederman 11