1. MODULE THREE: Demand-side Grey Literature GreyWorks 2010 Transparency Governs the Grey Landscape

2. Measures Demand-side Citations References Website logs User statistics Links Etcetera Farace – General

3. Applications and Uses Examples: Problem solving Decision making Policy development Scholarly communication Research and innovation Product enhancement Technology transfer Etcetera Farace – General

4. Grey Literature Usage Respondents were asked to name 2 documents used in the last month that were important to their work. Participants provided document titles and sources which we then located. 59% of documents listed were Grey Literature. Many thought they could find all Grey Literature via traditional online services. Liddy et al., GL6 - Usage

5. On usage statistics The big silence. Confirms the DRIVER study 2007: 70% of the repositories do log the statistical data on access but analysis and interpretation « in development » or « problematic ». No public access to statistics but in some cases, information to authors (CCSD). Schopfel et al., GL10 – Usage Stats

6. Advantages of including ‘grey’ in a literature review Access to wider variety of information Reduces publication bias Contain more local information- important for planning Contain process information- important for policy makers Knight et al., GL6 - Use

7. Advantages of including ‘grey’ in a literature review Reports not published in peer reviewed journals (e.g. UN, WHO, HTA) Access to new research World Wide Web (4-6 Billion web pages) Contain valuable and unique information, which is not found elsewhere Knight et al., GL6 - Use

8. Conclusion The identification of relevant studies in the grey literature and their inclusion in systematic reviews can be particularly time-consuming and difficult. However, the inclusion of grey literature in systematic reviews will help to overcome some of the problems of publication bias and evidence for process and context Aberdeen University is taking forward the development of the tools which can be used to review the grey literature in terms of best available evidence and evaluation mechanisms Knight et al., GL6 - Use

9. 96.6% Use Grey Information n=58 86.2% 82.8% 81.0% Proceedings Trip/Cave Reports Theses/ Dissertations + Maps Top Four Grey Information Sources To the statement, “Grey literature is a very important source of information to karst researchers …” 70.9 % agree Karst Information Portal Chavez et al., GL8 - Use

10. Preservation & Access 75.5% Personal Archiving Space Where do you archive the grey information that you produce? No Formal Archiving 28.3% Information Type% Theses/Dissertations51.0 Conference Proceedings/Papers46.9 Governmental Reports42.9 Non-Governmental Reports40.8 Cave Entrance Databases36.7 Trip & Cave Reports32.7 Foreign-Language Translations28.6 Association/Organization Publications28.6 Maps26.5 Pre-1923 Karst Papers26.5 Most Difficult Grey Information Types to Access n=49 Karst Information Portal Chavez et al., GL8 - Preservation

11. 96.6% Use Grey Information n=58 86.2% Proceedings 82.8% Trip/Cave Reports 81.0% Theses/Dissertations & Maps 89.3% Produce Grey Information n=56 80.8% Proceedings 76.9% Trip/Cave Reports 65.4% Images & Speeches/Invited Talked Chavez et al., GL9 – Use and Production

12. KIP & Grey Literature A core function of KIP is to facilitate access to grey literature such as:  Maps  Measurements of cave notes  Notes on subterranean wildlife Audience for KIP:  Geologists & geographers  Insurance companies & home buyers  Biologists & cave enthusiasts  Land managers & land planners Chavez et al., GL9 – Access and Use

13. report negative results discuss studies that concluded prematurely offer a much more comprehensive picture of the state of knowledge for any topic Analysis of GL in the changing context (2) Why is it important to raise the profile of GL? Because grey literature is still useful… GL is more likely to: IN FACT De Castro et al., GL8 - Use

14. … ? Which HEP information system do you use the most? High Energy Physics Gentil, GL9 - Use

15. Variations according to search Gentil, GL9 - Use

16. What feature is important in a HEP information system? NOT IMPORTANT VERY IMPORTANT Gentil, GL9 – User Demands

17. Conclusions What did we learn? –Community-driven systems answer the needs of the community. –Listen to the users – they know what they want! –Increased importance of a diversifying grey literature… Full analysis still in progress – more coming soon! What’s next: –Create a single information service for the entire community –Include new features –Survey your users! Gentil, GL9 – User Needs

18. Evidence Based Practice 5 Essential Steps in EBP Convert information needs into answerable questions Track down best evidence to answer questions Critically appraise evidence performance for its validity and usefulness Apply results of the appraisal in clinical practice Evaluate performance and outcomes Gelfand, GL7 – Evidence based Practice

19. Hierarchy of Evidence Meta-Analysis Systemic Review Randomized Controlled Trial Cohort Studies Case Control Studies Case Series / Reports Basic Laboratory & Animal Research Gelfand, GL7 – Evidence based Practice

20. AdjectiveFrequency Interesting 16 Stimulating 10 Tiring 8 Useful 7 Demanding 4 Formative 3 Frustrating 3 Complex 2 Comprehensive 2 Table 8. Expressions used by respondents to describe their experience in GL, ranking by frequency Di Cesare et al., GL8 - Users

21. (13) The Average Net User should at least recognize the term Grey Literature Leading Questions Boekhorst et al., GL6 – Users

22. Comparative Data Survey Data 2004 with a control population from the Author Survey 2005 Farace et al., GL7 – Access

23. The Surface Web is accessible to popular search engines such as Google. But less than 1% of government R&D results are currently accessible to crawlers. Getting to the R&D Results Slide compliments of Walt Warnick Carroll, GL8 - Access

24. What is missing? Governmental reports Funding body reports Subject population materials Rudasill, GL10 – Access (Development Studies)

25. Possible reasons for lack of disclosure Competition for funding dollars Confidentiality of client records Fear of reprisal Lack of expertise in web programming Rudasill, GL10 – Access (Development Studies)

26. Web access to grey literature Schopfel, GL9 – Access

27. Access to full text Schopfel et al., GL10 – Access (full-text)

28. Freedom of Information Mandatory FOI publication schemes provide an inventory of grey content Police work requires well-organised records and this should permit retrieval in response to enquiries from the public FOI enquiries received are in addition to high volumes of press enquiries FOI enquiries have produced some ‘unwilling’ revelations from police, but also some information that is less revealing. Sturges et al., GL9 – Access and Retrieval

29. Some preliminary results CISTI Decrease of supply requests Open access: CNRC serials Open access: institutional repository Integrated, partnership-based approach Change of acquisition policy? Boukacem et al., GL7 – Document Delivery

30. Some preliminary results INIST Decrease of supply requests Open access: CNRS strategic policy Open access: institutional repository, portals Open access: LARA, e-publishing, conferences Change of acquisition policy? Boukacem et al., GL7 – Document Delivery

31. Conclusions Search engines for grey literature are being widely deployed and represent a central information service in scholarly communication Institutional repositories gain momentum and become dominant over disciplinary repositories Standardized frameworks for distributed and federated document processing have been established Vesely et al., (CERN) GL5 - Repository

32. (14) GL Collections are beter managed by centralized repositories… The Document Level Boekhorst et al., GL6 – Repositories

33. More Definitions Institutional Repositories Set of services that a university offers to the members of its community for the management and dissemination of digital materials created by the institutions and its community members.” Gelfand, GL6 - Repositories

34. Faculty Benefits Diminished Isolation Shared Purpose & Cooperation Increased Curricular Circulation Fresh Approach to One's Discipline Increased Satisfaction with Students' Learning Gelfand, GL6 - Repositories

35. Selective Repository Benefits Free to contribute for all UC affiliates Promising alternative Increased visibility Usage reports eMail notification Permanence Global accessibility Ability to upload associated content Gelfand, GL6 - Repositories

36. Long Standing and Ongoing Issues Transcience of grey literature Maturation of the repository Timely publishing Access Standards Multiple formats Gelfand, GL6 - Repositories

37. Institutional Repositories: Challenges Difficult to convince researchers of benefits of institutional repositories –Consequent lack of interest in depositing materials, even if they will be managed by others Will take further evaluation to determine if institutional repositories reach full potential For now, coverage of grey literature in institutional repositories is encouraging Banks, GL7 - Repositories

38. Importance of Grey Lit from Public Health NGOs NGOs and their information products are key to public health in developing countries Grey lit from NGOs is often difficult to find Pakistan Healthcare Facilities Facility Type Public Sector Private Sector Total s Hospitals9066261,532 Dispensaries4,5903404,930 Rural Health Centers550 Basic Health Units5,308 MCH Centers (Maternal/Child Centers) 862300 (maternity homes) 1,162 TB Clinics285 Totals12,5011,26613,76 7 Crowe et al., GL7 – Repository (NGO Publications)

39. Possible Solution: Repository Digital collection that captures and preserves the intellectual output of an institution, agency or organization Benefits –Data collection and coordination –Could support the building of health capacity in developing countries –Knowledge management tool for NGOs Crowe et al., GL7 - Repository

40. Conclusion Grey lit from NGOs is a vital component of public health and medical care information, particularly in developing countries Public health NGOs would benefit by having one or more repositories of grey literature Relevant repository models exist There are significant barriers and challenges There are significant benefits to public health research community and to people world-wide Crowe et al., GL7 - Repository

41. Role of the institutional repository The IR is a support for integration –A central point within the organisation for literature and data –Stores multiple manifestations for the different parts of the process –Brings clarity to work and version relationships –GL can be recorded and retrieved according to accurate and up to date personnel, project and business unit structure IR integrated as part of the institutional memory Lambert et al., GL7 - Repository

42. A Little Less Grey? 1990: 67% of health sciences journals cite grey literature Alberani V, De Castro Pietrangeli P, Mazza AM. The use of grey literature in health sciences: a preliminary survey. Bull Med Libr Assoc. 1990 Oct;78(4):358-63. 2004: 85% of health policy journal articles cite grey literature 2004: 61% of think tank reports cite journal articles Dunn, GL6 - Citations

43. Triangle of Stakeholders Schopfel et al., GL6 - Citations

44. Grey versus Commercial A Comparison of Standard and Hyperlinked Citations Schopfel et al., GL6 - Citations

45. Language of Cited Documents and Citing Authors Schopfel et al., GL6 - Citations

46. Top Five “Types of Publications” Schopfel et al., GL6 - Citations

47. Guidelines for GL Citations General i.All conference papers should contain references ii.Standardization should be maintained among the citations provided iii.The more complete and accurate a citation, the more guarantee of a paper´s content and subsequent review Specific iv.Endnotes are preferred and should be numbered v.Hyperlink need the accompanying name of resource and date; a simple URL is not acceptable vi.If the citation is to a corporate author, the acronym takes precedence vii.If the document type is known, it should be stated at the close of a citation viii.If a citation is revised and refers to an edited and/or abridged work, the original (re)source should also be mentioned. Schopfel et al., GL6 - Citations

48. Citations to GOMC Publications Gulf of Maine Council on the Marine Environment OnlinePrint Gulf of Maine Region151164 United States (outside Gulf region) 6130 Canada (outside of Gulf region)2435 Europe231 Rest of world137 Total272237 MacDonald et al., GL8 - Citations

49. MacDonald et al., GL8 - Citations

50. MacDonald et al., GL8 - Citations

51. Conclusions visibility of GL both as cited and citing documents GL is evidently part of core papers possibility of reconstructing the document life-cycle: from GL to conventional literature Google Scholar Citation analysis Citation counts, citations over time, immediacy impact, self-citations Differences between GL and conventional literature tend to disappear …. Some typical characteristics of GL can be still noticed Highly cited GL receives more citations by GL documents and viceversa GL receives a higher number of citations in the first 5 years Language Di Cesare et al., GL9 - Citations