Internet Searches n User Communication Connection Vendor Large Computer

The Internet as a Resource n No review of information, generally n You decide what is junk n Hard to find really useful information (dilution problem)

Search Concepts n Precision (or relevance) the fraction of the documents retrieved in a search that are useful All documents Useful

Search Concepts n Recall - the fraction of useful documents retrieved out of all useful documents in the database

Problem n A search that is too narrow may have high relevance (precision) but low recall n High purity but low percent yield

Problem n A search that is too broad may have high recall but low precision (most of the useful documents are retrieved but many irrelevant ones are retrieved also) n Low purity with high percent yield

Searching On The Internet n If you can find it here, you can find it anywhere n Lots of dross, little gold for scientists

Search Engines n “Spiders” create massive indexes of words on the pages

Search Engines n Searching occurs on the index Word 1 on page A Word 2 on page A Word 2 on page B … Word 1 on page K... A search on word 1 gives references to pages A and K

Search Results n Summary information from some source n link to the page n relevance rating

Summary Information n Excite - from the page but may be from several sentences deep using a keyword algorithm n may be irrelevant

Summary Information n HotBot - from the description tags or from the top headings n may produce gibberish if page was poorly done

Relevance Rating n count the total number of times the search word occurs on the page n higher frequency gives higher relevance

Relevance Rating n count the occurrence of the search word in the first “n” words on the page n assumption is that the word occurs early in the document if it is the main subject of the document

Advanced Searches n search on phrases requiring that EVERY word of the phrase be present on the page in the EXACT ORDER entered in the phrase

Advanced Searches n Boolean u and (+) u or u not (-)

General Tips n Avoid single search terms n For information on the Whitesox use +baseball+Whitesox+”batting average”

General Tips n Avoid common words as search terms

General Tips n use and (+) to require words be on the page n use not (-) to omit pages with specified word on them

General Tips n require exact matching of all words in a phrase

Evaluating Search Engines

Choosing the Best Search Tool to Start With n Directories and Virtual Libraries u Contain topical lists of selected resources, hierarchically arranged u Many directories rate and review Web resources u Meant to be browsed, but can also be searched by keyword u Depend on people for selection and control of the included resources u Usually best for broad topics u Updated less frequently than search engine databases u Tend to have small databases u Contain links to subject guides and specialized databases

Best tool (cont) n Search Engines u Attempt to index as much of the Web as possible u Most are full-text databases u Require knowledge of search techniques to guarantee good results u Databases are created by computer programs called spiders or robots u Most often used for multifaceted topics and obscure subjects u Search very large databases that are updated frequently

Search Tools (cont) n Meta-search tools u Some allow you to search several search engines simultaneously u Some supply lists of databases that can be searched directly from their pages u Provide a good way to keep up with new search engines u May not fully exploit the features of individual search engines, so you must keep your search simple

Search Tools (cont) n Intelligent agents u Software entities that perform tasks on your behalf u Some can send the information requested on a schedule that you define u Sometimes referred to as bots or robots u Agents work with some degree of autonomy

Web directory vs. search engine n Directory u Has a structure that can be browsed u Editors generally decide where to list each site within the categories n Search engine u You must search a search engine on any subject by using keywords u Usually run an automated program that continually scans the web content, page by page, storing all the information found along the way. u The results are returned to you based on relevance (the number of times your search words appear on any given page)

Different uses of directories vs. search engines n If you are looking for a specific site (know by name) use a directory n If looking for a list of sites on the same subject, use a directory. n If looking for a specific page within a site, use a search engine

Yahoo n n Use help to find about how to search n Has a directory which is an online guide to the WWW. It is created by editors who visit and evaluate web sites. n Yahoo’s directory can also be browsed.

Lycos n n Different pages to search for different information

AltaVista n

Video Help

Google n

Google Advanced Images Search

Google –Groups search

Google--directory

Search.com n What is new about Search.com? u Search.com uses SavvySearch metasearch technology to give you the most comprehensive search results on the Internet. Now you only have to search once for results from over 1,000 search engines, Web directories, auctions, storefronts, news sources, discussion groups, reference sites, and more. In addition, Search.com's metasearch channels give you the power to get specialized results from dozens of search engines dedicated to popular topics, like Music or Travel.MusicTravel n How does Search.com work? u Search.com sends your search query to several search engines at one time and integrates the results into one list. That means you don't have to spend time going to each individual site to search — you can get all the Web's top results at once from Search.com

Search.com n Can I search in specific categories? u Of course — just choose a metasearch channel from our main page. Each of our channels searches specialized engines related to that particular topic or genre. For example, our Music channel gives you the option to search for MP3s, lyrics, reviews, prices, software, concert tickets, and more. You can select as many categories as you like on a particular channel. The more checkboxes you select, the more engines we search.main pageMusic

Search.com n Understanding search results u At the top of search results pages, you'll see a list of the sites we searched to get your results. (A green bullet means we successfully contacted the site; a gray bullet means we did not get a response in time.) Below that list your results appear, organized into groups of similar information. For example, if you do a search on earthquakes, you'll see links to the earthquake category pages found on human-built Web guides like Yahoo! and Snap under the heading Directories, earthquake sites under Web Pages, earthquake news under Headlines, and prices on earthquake books, videos, and other products under Latest Prices. If you'd like more results within a certain grouping — Web pages, for example — just click the link that says See more Web pages results. On that page, your results will be grouped according to which site they came from. If you want to see more results from a particular site, click the link that says See all matching results from the site you're interested in. And if you didn't find what you were looking for, you can always perform another search from the search box at the bottom of the page, or choose another channel to search from. Results you get on our search channels are organized similarly, based on which checkboxes you select before you perform your search.earthquakes

Search.com n Advanced search tips u Some engines that Search.com queries support the following advanced search options: double-quoted phrases: " " (e.g., "john lennon" not john lennon) enforced term operators: +/- (e.g., music +"john lennon" - beatles) boolean language: and, or, not (e.g. music and "john lennon" and not beatles) You may improve your results through these advanced search techniques, but many engines do not support these options, so sometimes your results may appear to be exactly the same as they would be without them. Try out a few different approaches and see what works best for you.

Virtual Libraries n Directories that contain collections of resources that librarians and information specialists have carefully chosen and have organized in a logical way n Smaller than directories since resources are carefully selected n Examples u The Argus Clearinghouse F u INFOMINE F u The Internet Public Library F u Librarians’ Index to the Internet F u WWW Virtual Library F

Three Major Types of Information in a Virtual Library n Subject Guides (meta-pages) u WWW resource that is devoted to including hyperlinks of most Web pages on a particular subject n Reference Works u A full-text document with self-contained information n Specialized Databases u An index that catalogs certain material. They are searched by keyword

More Information on Search Engines n Tools that use computer programs called spiders or robots to automatically gather information on the Internet to create databases. n Search Engines u AltaVista F u Excite F u Google F u HotBot F u Infoseek F u Lycos F u Northern Light F u Webcrawler F n Types of searching u Boolean –AND, OR, NOT u Phrase—use “ “ around the phrase—words appear adjacent to each other in the document

Meta-Search Tools n Allow you to search several search engines simultaneously n Some Meta-Search Tools u All-in-One Search Page F u Dogpile F u Fossick.com F u MetaCrawler F u ProFusion F u SavvySearch F u SEARCH.COM F

Intelligent Agents n Software entities that gather information or accomplish tasks without your immediate presence. n You tell the agent what action you want performed and the agent then executes them on your behalf. n They can perform repetitive tasks of searching databases, retrieving and filtering information, and delivering information back to you. n Example u MyPage from Go Network F You access services, personalize it and use your username and password you create to get into information F You set up the agent that search continuously for information on your topic and it alerts you when it finds something. F You can leave it working and come back a week later to see what it has turned up.

Domain types n.edu u Educational (can be anything from university research to a student’s or faculty member’s home page n.gov u Governmental (usually contains reliable data) n.com u Commercial (may be trying to sell a product) n.net u Network (may provide services to commercial or individual customers) n.org u Organization (usually created by a nonprofit institution; may be trying to persuade the reader; may be biased n.mil u United States military sites, agencies, and some academics

10 Basic Steps to a Good Search Strategy n Identify the important concepts of your search n Choose the keywords that describe these concepts n Determine whether there are synonyms, related terms, or other variations of the keywords that should be included. n Determine which search features may apply, including truncation, phrase searching, Boolean operators, etc. n Choose a search engine

10 steps (cont) n Read the search instructions on the search engine’s home page. Look for sections entitle “Help”, “Advanced Search”, “Frequently Asked Questions (FAQ)”, etc. n Create a search expression using syntax that is appropriate for the search engine n Evaluate the results. How many hits were returned? Were the results relevant to your query? n Modify your search if needed. Go back to Steps 2 through 4 and revise your query accordingly. n Try the same search in a different search engine, following steps 5 through 9.

Guidelines for Evaluation of Internet Resources n Who is the author or institution u If the resource was written by an individual, does it offer or give links to biographical information about the author u If an institution produced the resource, does it provide links to information about itself, including its purpose, history, and street address u Have you seen the author’s or institution’s name cited in other sources or bibliographies u If the page is part of a larger institution’s Web site, does the institution appear to filter the information that appears at its site u What clues does the URL give you about the source’s authority. F A ~ indicates a personal page F Pay attention to the domain name

Guidelines (cont) n Who is the audience u Is the Web page intended for the general public, or is it meant for scholars, practitioners, children, and so forth u Is the audience clearly stated u Does the Web page meet the needs of its stated audience n What is the purpose of the information u Is the purpose of the information to inform, explain, persuade, market a product, or advocate a cause u Is the purpose clearly stated u Does the resource fulfill the stated purpose

Guidelines (cont) n Is the content accurate, objective, and supported by other sources u Are there political, ideological, cultural, religious, or institutional biases u Is the content intended to be a brief overview of the topic or an in-depth analysis? u If the information is opinion, is this clearly stated u If there are facts and statistics included, are they properly cited u If it clear how the data was collected, and is it presented in a logical, organized way u Is there a bibliography at the end of the document.

Guidelines (cont) n How current is the information u Does the Web page have a date that indicates when it was placed on the Web u Is it clear when the page was last updated

Citing a Web Reference n There are three majors styles u APA (American Psychological Associates) F u MLA(Modern Language Association) F u Chicago Manual of Style n Agreements of citation u Author’s name u Document title u Title of larger or complete work; if relevant u Date of publication or last revision u Date page was accessed u URL