2. Conquering the Presented by Bonnie Shucha University of Wisconsin Law Library bjshucha@wisc.edu August 10, 2005 Invisible Web

3. The Problem Most searchers only locate 0.03% - 1 in 3,000 - of the Web pages available to them Even advanced searchers, using largest search engines, can only access about 16% of Web content Diagrams from http://brightplanet.com/technology/deepweb.asp

4. Why? Because 84% of the information available on the Internet is found only on the “invisible Web,” a.k.a. “deep Web,” and is not searchable using a general search engine such as Google Statistics from The Deep Web: Surfacing Hidden Value, http://www.press.umich.edu/jep/07-01/bergman.html

5. The Invisible Web

6. The Visible Web Visible Web page exists in “static” or unchanging form Exists as a “physical” file on a computer Most in.htm or.html format Similar to a word processed document in.doc or.wpd format

7. The Visible Web Static Web pages considered “visible” because standard search engines can index them and display them as search results

8. Indexing & the Visible Web Search engine spider crawls Web starting with already indexed static pages Spider encounters link to a new static Web page Spider follows link Spider adds new Web page to search engine’s index Content rendered “visible” Webmaster registers new static Web page with search engine

9. The Invisible Web Invisible Web content is “dynamic” or changing Contains bits of information stored in a database and pulled together on-the-fly into a Web page at your request Page doesn’t exist until you request it Similar to a mail merged document

10. The Invisible Web Dynamic Web Page Database Your search results Author Title Publication Searching Smarter Wisconsin Lawyer J. Doe Common Law Marquette Law Review B. Shucha 1. B. Shucha, “Searching Smarter,” Wisconsin Lawyer. 2. J.Q. Public, “Legal Tech Tips,” ABA Journal. J.Q. Public Legal Tech Tips ABA Journal

11. The Invisible Web Because this content is dynamic, or “physically” nonexistent, most search engines are unable to retrieve it, thereby rendering it “invisible”

12. Indexing & the Invisible Web Spider encounters database Query is required to access “dynamic” data Spider incapable of generating query Spider stops and cannot index data in database Content rendered “invisible” Spider crawls Web starting with already indexed static pages

13. The Invisible Web Other types of Invisible Web Content Very recent static pages which haven’t yet been indexed Password protected data

14. Invisible Web Content 95% of invisible Web content is free and available to the public Quality of content often exceeds that of visible Web content From The Deep Web: Surfacing Hidden Value, http://www.press.umich.edu/jep/07-01/bergman.html

15. Invisible Web Content Legal & Governmental Materials Available in the Public Domain Case law Statutes Bills Regulations Patents Briefs Census Data Government Reports

16. Invisible Web Content Business Data SEC filings Stock quotes Company profiles

17. Invisible Web Content General Information Address & phone directories Flight schedules Dictionaries Maps

18. Invisible Web Content NOT freely available on Web (usually) For Profit Publications Public domain documents with editorial enhancements Other material that is someone’s intellectual property

19. Finding Invisible Web Content To find ANY information, consider where an authoritative source might be found Print? Visible Web? Invisible Web? Subscription Database? Phone Call? Next, consider the quickest, most cost- effective way to get the information

20. Finding Invisible Web Content If you determine that it may be available on the invisible Web, how do you find it? By knowing where to look!

21. Finding Invisible Web Content A great deal of excellent legal and business information is freely available on the Internet Much of it is contained within databases and is, therefore, invisible to most conventional search engines

22. Finding Invisible Web Content The most effective way to access this information is using the database’s own search box The search box is usually found on a static, visible Web page that is accessible using a conventional search engine

23. Finding Invisible Web Content Search Strategy DON’T search for specific information using a conventional search engine DO use a conventional search engine to search for a database that may contain the information you seek THEN use the search box for that database to search for the specific information

24. Finding Invisible Web Content “ T he point is that often the key to the answer is not locating the answer itself as the first step, but locating the right database in which to search for it.” Diana Botluk, Mining Deeper into the Invisible Web, http://www.llrx.com/features/mining.htm

25. Search Exercises Attempt to locate the following: 1. Wisconsin Statute 758.01 2. My email address 3. Brief from WI Court of Appeals case, Docket 99-2588 4. The name of the person next to you 5. Subtitle C of the U.S. Internal Revenue Code 6. “Contract Law in Wisconsin” (State Bar of Wisconsin CLE Book)

26. Reviewing Search Exercises Were you able you find the information? Was it from an authoritative source? What was your search strategy? Why did you choose this strategy? What were the costs associated with your search? Did you choose quickest, most cost effective method to locate the information?

27. Group Exercise - #3 Locate brief from WI Court of Appeals case, Docket 99-2588 One strategy: Open Google or another search engine Search for database that may contain the brief “Wisconsin briefs” or “Wisconsin court of appeals brief” In the Wisconsin Briefs database, search for the brief (follow instructions) “992588”

28. Group Exercise - #5 Locate Subtitle C of U.S. Internal Revenue Code One strategy: Open Google or another search engine Search for database that may contain the code “Internal Revenue Code” On the Internal Revenue Code page, browse to Subtitle C

29. Group Exercise - #5 Locate Subtitle C of U.S. Internal Revenue Code Another strategy: Open Google or another search engine Search for database that may contain the code “United States Code” Search the GPO Access U.S. Code database, “Internal Revenue Code subtitle C” In the IRC section, note citation to subtitle C Go back to search box, and enter citation “26USC3101”

30. Invisible Web Resources Federal Law American Factfinder, http://factfinder.census.gov American Factfinder Population, housing, economic, and geographic data from the U.S. Census FedStats, http://www.fedstats.gov FedStats Statistics from United States government agencies FindLaw's Cases & Codes, http://findlaw.com/casecode/ FindLaw's Cases & Codes Links to databases of federal and state cases and legislation

31. Invisible Web Resources Federal Law FindLaw's Supreme Court Center, http://supreme.lp.findlaw.com/supreme_court/resources.html FindLaw's Supreme Court Center Recent U.S. Supreme Court opinions, orders, briefs, docket, and more GPO Access, http://www.gpoaccess.gov GPO Access U.S. Code, C.F.R., and so on, from the Government Printing Office Thomas, http://thomas.loc.gov Thomas U.S. legislation and other congressional information

32. Invisible Web Resources Wisconsin Law WisBar State and Federal Legal Resources, http://www.wisbar.org/AM/Template.cfm?Section=Legal_Research WisBar State and Federal Legal Resources Links to Wisconsin and federal resources Wisconsin Briefs, http://library.law.wisc.edu/elecresources/databases/wb/index.php Wisconsin Briefs Supreme Court & Court of Appeals briefs Wisconsin Legislative Drafting Records, http://library.law.wisc.edu/~draftingrecords Wisconsin Legislative Drafting Records Written materials, letters, and memoranda given to or created by the legislative drafting attorney

33. Invisible Web Resources Wisconsin Law Wisconsin Legislature Infobases, http://folio.legis.state.wi.us/ Wisconsin Legislature Infobases Wisconsin statutes, acts, bills, and more Wisconsin Online Court Records, http://www.wicourts.gov/casesearch.htm Wisconsin Online Court Records Status information for Wisconsin cases (WSCCA.i & CCAP)

34. Invisible Web Resources Journals, News, and More Badgerlink, http://www.badgerlink.net Badgerlink Scholarly & popular journals and newspapers Available to Wisconsinites Legaltrac, http://wsll.state.wi.us/enterlt.html Legaltrac Index of legal periodicals Available to WSLL card holders

35. Invisible Web Resources Journals, News, and More MPL Database for Remote Use, http://www.mpl.org/files/great/bookmark.cfm?Category=82 MPL Database for Remote Use D&B Million Dollar Database, CQ Researcher, etc. Available to Milwaukee PL card holders Yahoo Search Subscriptions, http://search.yahoo.com/subscriptions Yahoo Search Subscriptions WSJ, LexisNexis, Consumer Reports, etc. Text available by subscription

36. Invisible Web Resources General Invisible Web Directories CompletePlanet, http://www.completeplanet.com CompletePlanet Direct Search, http://www.freepint.com/gary/direct.htm Direct Search ProFusion, http://www.profusion.com ProFusion Librarian's Index to the Internet, http://lii.org Librarian's Index to the Internet

37. Presentation based on the article: Bonnie Shucha, Searching Smarter: Finding Legal Resources on the Invisible Web, Wisconsin Lawyer, September 2004, at 19, at http://tinyurl.com/dthen. http://tinyurl.com/dthen © Bonnie Shucha Reference & Electronic Services Librarian University of Wisconsin Law Library bjshucha@wisc.edu http://wisblawg.blogspot.com