Presentation on theme: "The Role of COUNTER David Sommer JIBS User Group 2009."— Presentation transcript:
The Role of COUNTER David Sommer JIBS User Group 2009
Background Understanding usage Different approaches Role of usage statistics Usage statistics Should enlighten rather than obscure Should be practical Should be reliable Are only part of the story Should be used in context COUNTER Current status Future directions
Why COUNTER? Goal: credible, compatible, consistent publisher/vendor-generated statistics for the global information community Libraries and consortia need online usage statistics To assess the value of different online products/services To support collection development To plan infrastructure Publishers need online usage statistics To experiment with new pricing models To assess the relative importance of the different channels by which information reaches the market To provide editorial support To plan infrastructure
COUNTER Codes of Practice Definitions of terms used Specifications for Usage Reports What they should include What they should look like How and when they should be delivered Data processing guidelines Auditing Compliance Maintenance and development of the Code of Practice Governance of COUNTER
COUNTER: current Codes of Practice 1) Journals and databases Release 1 Code of Practice launched January 2003 Release 2 published April 2005 replacing Release 1 in January 2006 Now a widely adopted standard by publishers and librarians 100+ vendors now compliant 15000+ journals now covered Librarians use it in collection development decisions Publishers use it in marketing to prove value Release 3 published August 2008, implemented by 31 August 2009 2 vendors already compliant: HighWire Press, MPS Technologies 2) Books and reference works Release 1 Code of Practice launched March 2006 13 vendors now compliant Relevant usage metrics less clear than for journals Different issues than for journals Direct comparisons between books less relevant Understanding how different categories of book are used is more relevant
Code of Practice for Journals and Databases: Release 3 – main features SUSHI protocol incorporated into R3 Usage Reports must be provided in XML as well as in Excel Vendors providing journal archives as a separate acquisition must report usage of these in a separate report Search and Session activity generated by federated search engines and automated search agents must be reported separately New library consortium usage reports, in XML only Activity generated by internet robots, crawlers, etc. must be excluded form the COUNTER reports A new optional additional report that enables usage of journals and books available on the same platform to be included in the same COUNTER report Release 3 is available at http://www.projectcounter.org/code_practice.html
Journal and Database Code of Practice: Release 3 Usage Reports Consortium Report 1 (XML only) Successful full-text journal article or book chapter requests by month Consortium Report 2 (XML only) Total searches by month and database Journal Report 1 Full text article requests by month and journal Journal Report 1a Full text article requests from an archive by month and journal Journal Report 5 Full-text article requests by year-of-publication and journal Journal Report 2 Turnaways by month and journal Database Report 1 Total searches and sessions by month and database Database Report 2 Turnaways by month and database Database Report 3 Searches and sessions by month and service
SUSHI Standardized Usage Statistics Harvesting Initiative (SUSHI) No mechanism existed for automatically retrieving, combining, and storing COUNTER usage data from different sources NISO-sponsored XML-based SUSHI provides a means to do just this, via a standard model for machine to machine automation of statistics harvesting. COUNTER and NISO have worked together to develop the SUSHI protocol. More details of SUSHI can be found at:- http://www.niso.org/committees/SUSHI/SUSHI_comm.html
SUSHI: Objectives COUNTER statistics provides an excellent model and rules for usage statistics counting Librarians needed: A more efficient data exchange model Current model is file-by-file spreadsheet download Background query and response model is more efficient and scalable
SUSHI: What it is and Isnt What it is: A web-services model for requesting data Replaces the users need to download files from vendors website A request for data where the response includes COUNTER data Using COUNTERs schema What it isnt: A model for counting usage statistics A usage consolidation application
Why COUNTER and SUSHI are important For libraries and publishers Usage statistics are being used to inform decisions They need to be consistent, credible and comparable = And, easy to obtain = SUSHI
Summary COUNTER provides the consistency and credibility ERM Systems/Usage Consolidation service provide the tools for more effective consolidation and reporting of usage data SUSHI acts as an enabling technology by allowing Usage Consolidation modules to automate the harvesting of COUNTER reports Release 3 of COUNTER Codes of Practice, will turn the theory and promise of these systems into reality by making make SUSHI support a requirement for compliance.
Future challenges Improving/extending the Codes of Practice Reliability ( audit, federated searches,) Usability (number of compliant vendors, XML format, additional usage reports) Database usage reports Additional data (year of publication, individual article usage data) Usage statistics for Institutional Repositories Categories of content/activity Online publications are more than collections of articles How to reflect the value provided by other features Deriving metrics from the Codes of Practice Journals (cost per use, Usage Factor) Databases? Books?
Deriving metrics from COUNTER- compliant usage statistics Local metrics For libraries and library consortia At journal, collection and publisher level To compare the cost-effectiveness of journal subscriptions To assess the value of Big Deals Global metrics For authors, funding agencies, libraries and publishers At journal, collection and publisher level To compare quality and value Usage Factor project PIRUS project
COUNTER: global metrics Impact Factor Well-established, easily understood and accepted Endorsed by funding agencies and researchers Does not cover all fields of scholarship Reflects value of journals to researchers Over-emphasis on IF distorts the behaviour of authors Over-used, mis-used and over-interpreted Usage Factor Usage-based alternative perspective Would cover all online journals Would reflect value of journals to all categories of user Would be easy to understood
Basis for Usage Factor A possible basis for the calculation of Usage factor could be COUNTER Journal Report 1: Number of Successful Full-text Article Requests by Month and Journal Usage Factor = Total usage (COUNTER JR1 data for a specified period) Total articles published online (for a specified period)
Usage Factor Objective: Assess the feasibility of developing and implementing journal Usage Factors Stage 1 Level of support from author, librarian and publisher communities Data from which UF would be derived COUNTER Journal Report 1? Article numbers Process for consolidation, calculation and reporting of UFs Factors in the calculation Level of reporting Total usage Articles? Report submitted in May 2007 Find at http://www.uksg.org/usagefactorshttp://www.uksg.org/usagefactors Stage 2: Test approach and methodology on real publisher usage data RFP to be circulated in mid-2009
Usage Factor: Stage 1 feedback Are the COUNTER usage statistics sufficiently robust? Frustration at lack of comparable, quantitative data on journals Should items covered by restricted to articles? Many journals still have significant usage in print Diversity of views on the factors in the calculation Specified usage period Specified publication period Usage data is more susceptible to manipulation Will the journal be a meaningful concept in the future? Two measures with different limitations are better than one, and UF will be derived from a set of credible, understandable data Usage data will be used as a measure of value, whether publishers like it or not
Stage 2:Preparing and harmonising data for analysis Journal usage data from multiple publishers to be ingested and analysed by expert third party 6 publishers 1 aggregator 1 hosting service All interested in the insights to be gained from contributing data to the project
Key Data Issues 1. Consistency – numerator/denominator 2. Defining article usage year 3. Defining article publication date 4. Different usage patterns by subject
Report specification With key data issues addressed, we have developed a specification for the report via which participating publishers will transfer their usage data to the appointed third party for analysis
Data Analysis and Modelling Using the data provided, the appointed contractor will produce a report which will:- Outline the various metrics assessed Recommend which of them prove consistent and robust enough to be adopted for scaled up onward monitoring Suggest any ways in which data providers might amend the way they capture, structure, label, and maintain their data which would make the measurement of Usage Factors:- Easier More reliable Propose ways to audit Usage Factors for accuracy
Usage Factor Project More information at:- http://www.uksg.org/usagefactors Or contact:- Richard Gedye email@example.com
PIRUS Publisher and Institutional Repository Usage Statistics A COUNTER Project Sponsored by JISC through the PALS Metadata and Interoperability programme (phase 3) Further information and full report: http://www.jisc.ac.uk/whatwedo/programmes/pals3/pir us.aspx
Increasing interest in article level usage More journal articles hosted by Institutional and other Repositories Authors and funding agencies are increasingly interested in a reliable, global overview of usage of individual articles Online usage becoming an alternative, accepted measure of article and journal value Knowledge Exchange report recommends developing standards for usage reporting at the individual article level Usage-based metrics being considered as a tool for use in the UK Research Excellence Framework and elsewhere.
Article usage metrics now more practical Implementation by COUNTER of XML-based usage reports makes more granular reporting of usage a practical proposition Implementation by COUNTER of the SUSHI protocol facilitates the automated consolidation of usage data from different sources.
The challenge An article may be available from:- The main journal web site Ovid ProQuest PubMed Central Authors local Institutional Repositories If we want to assess article impact by counting usage, how can we maximise the actual usage that we capture?
PIRUS Project Mission To develop a global standard to enable the recording, reporting and consolidation of online usage statistics for individual journal articles hosted by Institutional Repositories, Publishers and other entities
PIRUS Project Aims Develop COUNTER-compliant usage reports at the individual article level Create guidelines which, if implemented, would enable any entity that hosts online journal articles to produce these reports Propose ways in which these reports might be consolidated at a global level in a standard way.
PIRUS Outputs 1. A proof-of-concept COUNTER- compliant XML prototype for an individual article usage report Can be used by both repositories and publishers
PIRUS outputs 2. A tracker code, to be implemented by repositories that sends a message to either: an external party that is responsible for creating and consolidating usage statistics and for forwarding them to the relevant publisher for consolidation The local repository server
PIRUS outputs 3. Step 1: a fulltext article is downloaded Step 2: tracker code invoked, generating an OpenURL log entry Step A1: OpenURL log entries sent to external party responsible for creating and consolidating the usage statistics Step B1: OpenURL log entry sent to local server Step A3: COUNTER-compliant usage statistics collected and collated per article (DOI) in XML format Step B2: OpenURL log entries harvested by external party responsible for creating and consolidating usage statistics Step B5: COUNTER compliant usage statistics available from central organization to authorized parties Step A4: COUNTER compliant usage statistics available from central organization to authorized parties Step C1: OpenURL log entry sent to local server Step A2: logs filtered by COUNTER rules Step B4: COUNTER-compliant usage statistics collected and collated per article (DOI) in XML format Step C2: logs filtered by COUNTER rules Step C3: COUNTER-compliant usage statistics collected and collated per article (DOI) in XML format Scenario A Scenario B Scenario C Step B3: logs filtered by COUNTER rules Step C4: COUNTER compliant usage statistics available from IR to authorized parties
PIRUS outputs 4. Specification for the criteria for a a central facility that will create the usage statistics where required or collect and consolidate the usage statistics
COUNTER Membership Member Categories and Annual Fees (2009) Publishers/intermediaries: $850 Library Consortia: $560 Libraries: $425 Industry organization: $425 Library affiliate: $170 (non-voting member) Benefits of full membership Owner of COUNTER with voting rights at annual general meeting, etc. Regular bulletins on progress Opportunity to receive advice on implementation Vendors: no compliance fee; reduced price audit fees
http://www.projectcounter.org Apply for COUNTER membership
For more information………. http://www.projectcounter.org Thank you! Peter Shepherd, COUNTER firstname.lastname@example.org