Laying the Foundation Mining the Web Fr. Jomar Legaspi

Learning Milestones Internet / World Wide Web Search Engines Fundamentals of Search Mathematics Search Strategies Evaluating Web Resources Citing Web Resources Web Search Exercise

Internet How it happened? –need to connect scientists / experts from diverse locations to fast track space exploration project – ARPANET –explosive growth – browser How big is the Internet? –approximate – 40 million networks –200 M users connected to it –5 M websites – quadruples by 2005 –1 billion web documents (IDC – Internet Data Corporation – 1998) Internet revolution: –democratization of informationdemocratization of information –convergence of technology

Search Engines – Mining the Internet Individual Search Engines: compile their own searchable databases –Index words or terms in web based documents –Directories – classify web documents or locations in arbitrary classifications or taxonomy e.g. Yahoo, Google, Altavista Metasearch engines – gateway to databases from multiple search engines –Advantages: fast, more relevant but not that comprehensive vs individual search engines e. g. MetacrawlerMetacrawler

Mining the Internet – Search Engines Subject Directories –maintained by human editors rather than by spiders or web robots –Types: General Academic Commercial Portals = Gateway Vortals – subject specific –Strengths and weaknesses Cumbersome – process entails going through several layers of categories / steps High quality content – less instances of out of context search results Active links –When to use: General search / general topic –Examples: Yahoo LookSmart Magellan

Mining the Internet – Search Engines Gateways and Vortals –Gateways / portals: collection of databases and information websites categorized by subjects assembled, reviewed, recommended by content specialists or experts. Excellent for academic research Internet Public Library: Argus Clearinghouse: WWW Virtual Library: –Vortals (vertical portals) – dedicated to a single subject Eric Clearinghouse: The Big Hub: Complete Planet:

Mining the Internet – Search Engines Deep Web or the “Invisible Web” –approximately 60% - 80% of the web remains invisible to search spiders / robots. –Information in secured private networks / databases –Gateways and vortals = the best way to gain access and exploit the Deep Web / Invisible Web

Mathematics of Search Engines Use + or – signs before a keyword to force their inclusion / inclusion in the search. “” – keywords are searched in exact order / sequence –“information technology strategies” Combination of all the symbols –“information technology strategies”-business- government +schools

Search Strategies Articulate what you need to search. Formulate the key concepts as specific at they could be. Critical success factor: KEYWORDS Keywords = use NOUNS / OBJECTS rather than verbs and adjectives Avoid use of propositions, conjunctions, or common verbs – most search engines will disregard them Most powerful keywords = “phrase”

Separating diamonds from dirt… Tool – CARS by Robert Harris Credibility –Trustworthiness of the author = authority and credibility Author’s name Qualification Affiliations Publisher / Sponsor Address, tel. Nos. address Accuracy –Objective, correct, up-to-date, comprehensive, exact. The information is appropriate to the audience it was intended for. Date of publication Last date when the site was updated address Link to questions and comments Reasonableness –Balance, objectivity, and consistent; tone of the language – moderate / absence of motherhood statements / grandstanding –Watch out who is the sponsor Support –Sources of information / knowledge –Corroboration Citations of sources: bibliography

Resources Ellen Chamberlain, Bare Bones 101: A Basic Tutorial on Searching the Web, University of Southern California Beufort Library, January 2000, February 10, Craig Branham, A Student’s Guide to Research in the WWW, St. Louis University, Illinois, March 27, 1997, February 10, BrightPlanet Corp., Guide to Effective Searching of the Internet, – 2002, March 1,

Your school recently subscribed to the services of a local Internet Service Provider. Initially it was decided that Internet access will be available in the library where 15 computers were installed. Your school principal understood that the Internet can exponentially increase the number of learning resources available to the students which before where simply limited to print media. The principal wrote a memo asking all teachers to develop an online resource center as a way to assist students to search for quality information in the web. Your task: 1. define your audience 2. define the subject area / content / discipline 3. search the web for at least 10 online resources 4. give a brief description of each site