MODULE THREE: Demand-side Grey Literature GreyWorks 2010 Transparency Governs the Grey Landscape

Measures Demand-side Citations References Website logs User statistics Links Etcetera Farace – General

Applications and Uses Examples: Problem solving Decision making Policy development Scholarly communication Research and innovation Product enhancement Technology transfer Etcetera Farace – General

Grey Literature Usage Respondents were asked to name 2 documents used in the last month that were important to their work. Participants provided document titles and sources which we then located. 59% of documents listed were Grey Literature. Many thought they could find all Grey Literature via traditional online services. Liddy et al., GL6 - Usage

On usage statistics The big silence. Confirms the DRIVER study 2007: 70% of the repositories do log the statistical data on access but analysis and interpretation « in development » or « problematic ». No public access to statistics but in some cases, information to authors (CCSD). Schopfel et al., GL10 – Usage Stats

Advantages of including ‘grey’ in a literature review Access to wider variety of information Reduces publication bias Contain more local information- important for planning Contain process information- important for policy makers Knight et al., GL6 - Use

Advantages of including ‘grey’ in a literature review Reports not published in peer reviewed journals (e.g. UN, WHO, HTA) Access to new research World Wide Web (4-6 Billion web pages) Contain valuable and unique information, which is not found elsewhere Knight et al., GL6 - Use

Conclusion The identification of relevant studies in the grey literature and their inclusion in systematic reviews can be particularly time-consuming and difficult. However, the inclusion of grey literature in systematic reviews will help to overcome some of the problems of publication bias and evidence for process and context Aberdeen University is taking forward the development of the tools which can be used to review the grey literature in terms of best available evidence and evaluation mechanisms Knight et al., GL6 - Use

96.6% Use Grey Information n= % 82.8% 81.0% Proceedings Trip/Cave Reports Theses/ Dissertations + Maps Top Four Grey Information Sources To the statement, “Grey literature is a very important source of information to karst researchers …” 70.9 % agree Karst Information Portal Chavez et al., GL8 - Use

Preservation & Access 75.5% Personal Archiving Space Where do you archive the grey information that you produce? No Formal Archiving 28.3% Information Type% Theses/Dissertations51.0 Conference Proceedings/Papers46.9 Governmental Reports42.9 Non-Governmental Reports40.8 Cave Entrance Databases36.7 Trip & Cave Reports32.7 Foreign-Language Translations28.6 Association/Organization Publications28.6 Maps26.5 Pre-1923 Karst Papers26.5 Most Difficult Grey Information Types to Access n=49 Karst Information Portal Chavez et al., GL8 - Preservation

96.6% Use Grey Information n= % Proceedings 82.8% Trip/Cave Reports 81.0% Theses/Dissertations & Maps 89.3% Produce Grey Information n= % Proceedings 76.9% Trip/Cave Reports 65.4% Images & Speeches/Invited Talked Chavez et al., GL9 – Use and Production

KIP & Grey Literature A core function of KIP is to facilitate access to grey literature such as:  Maps  Measurements of cave notes  Notes on subterranean wildlife Audience for KIP:  Geologists & geographers  Insurance companies & home buyers  Biologists & cave enthusiasts  Land managers & land planners Chavez et al., GL9 – Access and Use

report negative results discuss studies that concluded prematurely offer a much more comprehensive picture of the state of knowledge for any topic Analysis of GL in the changing context (2) Why is it important to raise the profile of GL? Because grey literature is still useful… GL is more likely to: IN FACT De Castro et al., GL8 - Use

… ? Which HEP information system do you use the most? High Energy Physics Gentil, GL9 - Use

Variations according to search Gentil, GL9 - Use

What feature is important in a HEP information system? NOT IMPORTANT VERY IMPORTANT Gentil, GL9 – User Demands

Conclusions What did we learn? –Community-driven systems answer the needs of the community. –Listen to the users – they know what they want! –Increased importance of a diversifying grey literature… Full analysis still in progress – more coming soon! What’s next: –Create a single information service for the entire community –Include new features –Survey your users! Gentil, GL9 – User Needs

Evidence Based Practice 5 Essential Steps in EBP Convert information needs into answerable questions Track down best evidence to answer questions Critically appraise evidence performance for its validity and usefulness Apply results of the appraisal in clinical practice Evaluate performance and outcomes Gelfand, GL7 – Evidence based Practice

Hierarchy of Evidence Meta-Analysis Systemic Review Randomized Controlled Trial Cohort Studies Case Control Studies Case Series / Reports Basic Laboratory & Animal Research Gelfand, GL7 – Evidence based Practice

AdjectiveFrequency Interesting 16 Stimulating 10 Tiring 8 Useful 7 Demanding 4 Formative 3 Frustrating 3 Complex 2 Comprehensive 2 Table 8. Expressions used by respondents to describe their experience in GL, ranking by frequency Di Cesare et al., GL8 - Users

(13) The Average Net User should at least recognize the term Grey Literature Leading Questions Boekhorst et al., GL6 – Users

Comparative Data Survey Data 2004 with a control population from the Author Survey 2005 Farace et al., GL7 – Access

The Surface Web is accessible to popular search engines such as Google. But less than 1% of government R&D results are currently accessible to crawlers. Getting to the R&D Results Slide compliments of Walt Warnick Carroll, GL8 - Access

What is missing? Governmental reports Funding body reports Subject population materials Rudasill, GL10 – Access (Development Studies)

Possible reasons for lack of disclosure Competition for funding dollars Confidentiality of client records Fear of reprisal Lack of expertise in web programming Rudasill, GL10 – Access (Development Studies)

Web access to grey literature Schopfel, GL9 – Access

Access to full text Schopfel et al., GL10 – Access (full-text)

Freedom of Information Mandatory FOI publication schemes provide an inventory of grey content Police work requires well-organised records and this should permit retrieval in response to enquiries from the public FOI enquiries received are in addition to high volumes of press enquiries FOI enquiries have produced some ‘unwilling’ revelations from police, but also some information that is less revealing. Sturges et al., GL9 – Access and Retrieval

Some preliminary results CISTI Decrease of supply requests Open access: CNRC serials Open access: institutional repository Integrated, partnership-based approach Change of acquisition policy? Boukacem et al., GL7 – Document Delivery

Some preliminary results INIST Decrease of supply requests Open access: CNRS strategic policy Open access: institutional repository, portals Open access: LARA, e-publishing, conferences Change of acquisition policy? Boukacem et al., GL7 – Document Delivery

Conclusions Search engines for grey literature are being widely deployed and represent a central information service in scholarly communication Institutional repositories gain momentum and become dominant over disciplinary repositories Standardized frameworks for distributed and federated document processing have been established Vesely et al., (CERN) GL5 - Repository

(14) GL Collections are beter managed by centralized repositories… The Document Level Boekhorst et al., GL6 – Repositories

More Definitions Institutional Repositories Set of services that a university offers to the members of its community for the management and dissemination of digital materials created by the institutions and its community members.” Gelfand, GL6 - Repositories

Faculty Benefits Diminished Isolation Shared Purpose & Cooperation Increased Curricular Circulation Fresh Approach to One's Discipline Increased Satisfaction with Students' Learning Gelfand, GL6 - Repositories

Selective Repository Benefits Free to contribute for all UC affiliates Promising alternative Increased visibility Usage reports notification Permanence Global accessibility Ability to upload associated content Gelfand, GL6 - Repositories

Long Standing and Ongoing Issues Transcience of grey literature Maturation of the repository Timely publishing Access Standards Multiple formats Gelfand, GL6 - Repositories

Institutional Repositories: Challenges Difficult to convince researchers of benefits of institutional repositories –Consequent lack of interest in depositing materials, even if they will be managed by others Will take further evaluation to determine if institutional repositories reach full potential For now, coverage of grey literature in institutional repositories is encouraging Banks, GL7 - Repositories

Importance of Grey Lit from Public Health NGOs NGOs and their information products are key to public health in developing countries Grey lit from NGOs is often difficult to find Pakistan Healthcare Facilities Facility Type Public Sector Private Sector Total s Hospitals ,532 Dispensaries4, ,930 Rural Health Centers550 Basic Health Units5,308 MCH Centers (Maternal/Child Centers) (maternity homes) 1,162 TB Clinics285 Totals12,5011,26613,76 7 Crowe et al., GL7 – Repository (NGO Publications)

Possible Solution: Repository Digital collection that captures and preserves the intellectual output of an institution, agency or organization Benefits –Data collection and coordination –Could support the building of health capacity in developing countries –Knowledge management tool for NGOs Crowe et al., GL7 - Repository

Conclusion Grey lit from NGOs is a vital component of public health and medical care information, particularly in developing countries Public health NGOs would benefit by having one or more repositories of grey literature Relevant repository models exist There are significant barriers and challenges There are significant benefits to public health research community and to people world-wide Crowe et al., GL7 - Repository

Role of the institutional repository The IR is a support for integration –A central point within the organisation for literature and data –Stores multiple manifestations for the different parts of the process –Brings clarity to work and version relationships –GL can be recorded and retrieved according to accurate and up to date personnel, project and business unit structure IR integrated as part of the institutional memory Lambert et al., GL7 - Repository

A Little Less Grey? 1990: 67% of health sciences journals cite grey literature Alberani V, De Castro Pietrangeli P, Mazza AM. The use of grey literature in health sciences: a preliminary survey. Bull Med Libr Assoc Oct;78(4): : 85% of health policy journal articles cite grey literature 2004: 61% of think tank reports cite journal articles Dunn, GL6 - Citations

Triangle of Stakeholders Schopfel et al., GL6 - Citations

Grey versus Commercial A Comparison of Standard and Hyperlinked Citations Schopfel et al., GL6 - Citations

Language of Cited Documents and Citing Authors Schopfel et al., GL6 - Citations

Top Five “Types of Publications” Schopfel et al., GL6 - Citations

Guidelines for GL Citations General i.All conference papers should contain references ii.Standardization should be maintained among the citations provided iii.The more complete and accurate a citation, the more guarantee of a paper´s content and subsequent review Specific iv.Endnotes are preferred and should be numbered v.Hyperlink need the accompanying name of resource and date; a simple URL is not acceptable vi.If the citation is to a corporate author, the acronym takes precedence vii.If the document type is known, it should be stated at the close of a citation viii.If a citation is revised and refers to an edited and/or abridged work, the original (re)source should also be mentioned. Schopfel et al., GL6 - Citations

Citations to GOMC Publications Gulf of Maine Council on the Marine Environment OnlinePrint Gulf of Maine Region United States (outside Gulf region) 6130 Canada (outside of Gulf region)2435 Europe231 Rest of world137 Total MacDonald et al., GL8 - Citations

MacDonald et al., GL8 - Citations

MacDonald et al., GL8 - Citations

Conclusions visibility of GL both as cited and citing documents GL is evidently part of core papers possibility of reconstructing the document life-cycle: from GL to conventional literature Google Scholar Citation analysis Citation counts, citations over time, immediacy impact, self-citations Differences between GL and conventional literature tend to disappear …. Some typical characteristics of GL can be still noticed Highly cited GL receives more citations by GL documents and viceversa GL receives a higher number of citations in the first 5 years Language Di Cesare et al., GL9 - Citations