Week 3: MetaSearch Engines Click here for Word handout Tom Johnson Boston University - Dept. of Journalism

2 JO807 © J.T.Johnson 2000 ________________________________ MetaSearch Engines: Definition  You submit keywords in search box  Engine transmits search simultaneously to several individual search engines and their databases of web pages.  Results back from all the search engines queried, usually deleting duplicates  Meta-search engines do not own a database of Web pages; they send your search terms to the databases maintained for other search engines.

3 JO807 © J.T.Johnson 2000 ________________________________ How do you know if your search terms will "work"?  Search protocol (the way you enter search keywords) is far from standardized  Almost all accept " " as causing a phrasephrase  A few accept Boolean AND, OR, and NOT.Boolean  Fewer accept ( ) to group terms.  Some only accept + or -.  Some default to OR, some to AND  Some take * to truncate. Other stem automaticallytruncatestem

4 JO807 © J.T.Johnson 2000 ________________________________ Determining the usefulness of any meta-search engine?  Which search engines they send your search terms to (size, content, number of search engines, you ability to choose the search engines you prefer. All of them search subject directories as well as search engines and intermix results from all.)  How they handle your search terms and search syntax (Boolean operators, phrases, and defaults they impose);  How they display results (ranking; aggregated into one list, or with each search engine's results reported separately)

5 JO807 © J.T.Johnson 2000 ________________________________ Use metasearch, but cautiously  Meta-search engines only spend a short time in each database; often retrieve <10% of any of the results in any of the databases queried.  Most meta-searchers simply pass your search terms along; if search contains more than one or two words or very complex logic, most of that will be lost.  Quantity in results does not equal satisfaction.  Look for meta-search engines that also send your terms to selective or odd databases like WebCrawler, Thunderstone, Direct Hit, and WhatUSeek. WebCrawlerThunderstoneDirect Hit WhatUSeek

6 JO807 © J.T.Johnson 2000 ________________________________ Types of Search Engines  Search Engines & Meta-Search Engines  Subject Directories  Subject Guides  Specialized Databases  The Invisible Web The Invisible Web  Edgar Edgar

7 JO807 © J.T.Johnson 2000 ________________________________ Types of Search Engines: Search Engines & Meta-Search Engines  Full-text of selected Web pages  Search by keyword, trying to match words in pages  No browsing, no subject categories  Databases compiled by "spiders" (computer-robot programs) with minimal human oversight  Search-Engine size: from small and specialized to 90+ percent of the indexable Web  Meta-Search Engines quickly, superficially search several individual search engines at once. Return results compiled into a sometimes convenient format.  CAVEAT: Only catch about 10% of search results in any of the search engines they visit.

8 JO807 © J.T.Johnson 2000 ________________________________ Types of Search Engines Search Engines & Meta-Search Engines Examples:  Alta Vista, Northern Light, Infoseek, Fast Search  Other search engines:  Hotbot, Lycos, Excite  Meta-Search Engines:  Metacrawler, Inference Find, Dogpile, Metafind

9 JO807 © J.T.Johnson 2000 ________________________________ Types of Search Engines: Subject Directories Characteristics:  Hand-selected sites picked by editors, more or less carefully  Organized into hierarchical subject categories  Often annotated with descriptions (not in Yahoo!)  Browse subject categories or search using broad, general terms  NO full-text of documents. Can search only the subject categories and descriptions

10 JO807 © J.T.Johnson 2000 ________________________________ Types of Search Engines: Subject Directories Examples:  Librarians' Index, Infomine, Britannica's Internet Guide, Yahoo!, Galaxy Other subject directories:  Scout Report Signpost, Looksmart, Lycos's A2Z and Top 5%  Built into Infoseek, Excite  Most university libraries maintain subject directories

11 JO807 © J.T.Johnson 2000 ________________________________ Types of Search Engines: Subject Guides Characteristics:  Web pages of collections of hypertext links on a subject  Compiled by "expert" subject specialists, agencies, associations, and hobbyists  Locate through special guides to guides or subject directories or sometimes among search engine keyword search results

12 JO807 © J.T.Johnson 2000 ________________________________ Types of Search Engines: Subject Guides Characteristics:  Web pages of collections of hypertext links on a subject  Compiled by "expert" subject specialists, agencies, associations, and hobbyists  Locate through special guides to guides or subject directories or sometimes among search engine keyword search results

13 JO807 © J.T.Johnson 2000 ________________________________ Types of Search Engines: Subject Guides Examples:  Guides to guides:  Argus Clearinghouse Argus Clearinghouse  WWW Virtual Library WWW Virtual Library

14 JO807 © J.T.Johnson 2000 ________________________________ Types of Search Engines: Specialized Databases Characteristics:  The Web provides access through a search box into the contents of a database in a computer somewhere  Can be on any topic, can be trivial, commercial, task-specific, or a rich treasure devoted to your topic  Locate by special guides to databases, in Librarian's Index, Yahoo!, and sometimes by keyword searching general search engines

15 JO807 © J.T.Johnson 2000 ________________________________ Types of Search Engines: Specialized Databases Examples:  Sites listing lots of Databases  Search.com Search.com  The Invisible Web The Invisible Web  Edgar Edgar  FECInfo FECInfo  Adobe PDF files searchpdf.adobe.com Adobe PDF files

16 JO807 © J.T.Johnson 2000 ________________________________ A good place to start  BEST General Web Page SEARCH ENGINES et/SearchEngines.html BEST General Web Page SEARCH ENGINES et/SearchEngines.html  A Graduated Approach in 5 Steps A Graduated Approach in 5 Steps  Search Engine Guide 3,600+ search engines in its index. Will take you to search engines that target specific industries or topics, including 67 for "News and Media." This homepage also has a link to its mailing list, a good listserv that helps you stay on top of new developments in the metasearch world. Search Engine Guide