1. Proposed Policy Improvements on Policy and Procedures for Measurement of Research Outputs of Public Higher Education Institutions, 2003 University of Pretoria

2. Outline 1.Subsidy-eligible Research Outputs : Current and Proposed Policy Improvements Published Conference Proceedings Books Journals 2.Performance Indicators for Research The system Performance by types of publications The Institution 3.Conclusions

3. Subsidy-Eligible Outputs  The policy recognises the following research outputs for purposes of subsidy: Research Articles in approved Journals (1 unit) Scholarly Books and Chapters in Books (up to 5 units depending on the length) Published Conference Proceedings (0.5 units)

4. The Policy on Journals  Journals are peer-reviewed periodical publications devoted to disseminating original research and new developments.  All research articles published in approved journals are subsidised.  Book reviews, abstracts, news articles and similar publications are not subsidized.

5. The Policy on Journals (2)  There are 3 lists of approved journals (ISI, IBSS, DHET)  The Thomson Reuters Institute of Scientific Information (ISI) Web of Science Lists contains just over 11400 journals. The majority of these (almost 8300) are in the sciences, although all subject fields are covered.

6. The Policy on Journals (3)  The IBSS list contains just under 3000 journals and focuses on the social sciences, particularly anthropology, economics, political science, and sociology.  The DHET list contains 199 South African journals in various subject fields.

7. Policy Review  On encountering challenges with the approved indices and interacting with institutions, the DHET embarked on a process of policy improvements.  The process included: – Consideration of other journal indices – Survey of Non-Approved journal articles – International practices

8. Indices Considered  The DHET did background research on the following indices: SCOPUS ERIH ULRICH’S SCIENCE DIRECT NORWEGIAN LIST LILACs

9. Survey on non-accredited journal articles  In December 2009 the DHET requested that all institutions which have information on publications in non-accredited journals share this with the DHET.  22 HEIs responded with some information dating back to 2005.  Institutions indicated the number of articles (without author affiliation).  Institutions also made suggestions regarding indices which the Department could consider.

10. Non-Accredited Output (Articles)/ Accredited Output (Units)

11. Non-accredited articles on Scopus & Norwegian Lists

12. SCOPUS Number of Journals: 16,500 Fields of study: Life Sciences, Physical Sciences, Social Sciences, Arts & Humanities and Health Sciences. Coverage: Broad global coverage (i.e. Europe, the Middle East and Africa). Languages: English and ‘non-English’. Quality: Content Selection and Advisory Board (CSAB) was established to ensure quality. Selection criteria: English language abstract, Editorial policy, Peer-review, Diversity in provenance of editors and authors.

13. International Ranking & Journal Lists  The Shanghai Ranking only uses two of the Thomson Reuters ISI Web of Science Lists – the Science Citation Index Expanded and Social Sciences Citation Index and not the Arts & Humanities Citation Index.  For its ranking, the Times Higher Education Supplement initially made use of the Thomson Reuters ISI Web of Science lists. In 2007 it changed to Scopus only. In 2010 a new agreement was reached with Thomson’s and Scopus was dropped again in favour of ISI.

14. Norwegian List  Number of Journals: 19923  Criteria for inclusion: A scientific or scholarly publication must Present new insight Be in a form that allows the research findings to be verified and /or used in new research activity Be in a language and with a distribution that makes the publication accessible for a relevant audience be in a publication channel with peer review Authors must come from more than one institution

15. Survey on SA Journals on International Lists

16. Policy on Books  Books, book chapters, monographs and edited collections are evaluated for subsidy;  Books must have an ISBN number, be peer-reviewed, and at least 60 pages long;  Introductions, dissertations, theses, text books, study guides, speeches, contract work and fiction are not subsidized.

17. Policy on Conference Proceedings  This is a published record of a conference, congress or symposium which is aimed at the dissemination of original research and new developments.  Proceedings must meet specific criteria such as being published with an ISBN number, scholarly and peer-reviewed.  Abstracts, book reviews and similar are not subsidized.

18. Total Units Allocated to HEIs 2005 -2011

22. Some Significant Additions Research Integrity – This policy aims to support and encourage scholarship. Institutions and academics must remember the importance of the research integrity and ethics when implementing this policy and are urged to focus on quality research and not maximum accrual of subsidy. This should be taken into consideration when recycling previously published work and dividing research outcomes between articles. Similarly, when moving between institutions, author affiliation should reflect the institution where research was conducted, supported and funded. At its discretion, the Department may consider punitive measures against an institution which consistently makes incorrect or unethical claims.

23. SOUTH AFRICAN JOURNALS The Department of Higher Education and Training maintains a list of South African journals that meet the criteria set out in this policy. South African journals not appearing on the approved lists of journals which meet the policy’s minimum criteria, can apply for accreditation. The Department’s list is for developmental purposes, and for this reason a journal will only be listed for five years. This provides the time necessary to develop the journal to international standards and to apply for listing on any of the accredited indices. A South African journal which has been dropped from an approved index may apply for indexing on the Approved List of South African journals. However, reasons for their removal must be included in their application.

24. SOUTH AFRICAN JOURNALS Proposals must be accompanied by a detailed explanation of the reasons for the removal of such journals, in particular, stating which criteria the journal ceases to fulfil. Such proposals must be submitted to the Department not later than 15 June. South African journals no longer fulfilling the above criteria will be removed from the list. Removed journals may re-apply for listing after a minimum of two years using the above procedure. For purposes of transparency, the reasons for removal of journals will be made public.

25. BOOKS: UNITS ALLOCATION A chapter in a book = 1 unit A book of 60 pages = 2 units 120 pages = 4 units 180 pages = 6 units 240 pages = 8 units 300 pages (and above) = 10 units

26. BOOKS: PEER REVIEW AND RESEARCH EXPLANATION The peer-review evidence must be clear and unambiguous. A justification (maximum 500 words) from the author of the book or chapter explaining the contribution that the book makes to scholarship must be attached to each publication claim. This justification should not be an abstract of the contents, but should, rather, describe the methodology used as well as the unique contribution made. It should be clear that the book or chapter against which subsidy is being claimed disseminates original research and new developments within the specific discipline. The target audience should be mentioned. If such information is already provided in the actual publication, a marker or reference to this must be made rather than providing the justification.

27. BOOKS: PEER REVIEW AND RESEARCH EXPLANATION In the case of second or later editions being submitted for subsidy, clear evidence of new research must be provided. It is necessary that at least 50% of the publication being claimed must have not been published previously. A statement from the auditors indicating that they have checked both the previous and current edition and affirm that at least 50% of the work was not previously published is required for every claim. The DHET may request additional information at its discretion.

28. BOOKS: PEER REVIEW AND RESEARCH EXPLANATION Dissertations and Theses that have been converted into books must be clearly identified as such and there must be evidence of substantial reworking and additional research carried out. A statement from the auditors indicating that they have checked both the original thesis and published book and that there has been substantial reworking is required for every claim. The DHET may request additional information at its discretion. Proof of peer review (and not supervisors reports) for such publications is still a requirement.

29. RECOGNITION OF CONFERENCE PROCEEDINGS Where proceedings are published in an approved journal, the output will be treated as a journal article. Conference Proceedings published in non-accredited journals will not be subsidised, although those published in conference proceeding series can be submitted for consideration if they meet the stated criteria. Articles published in approved conference proceedings will be allocated a maximum of 0.5 units. In the case where authors are affiliated with two or more institutions, the subsidy is shared between the claiming institutions.

30. INDEX OF CONFERENCE PROCEEDINGS The Department will establish a list of South African approved conference proceedings. This list will be compiled from data collected by the Department through the normal submission of conference proceedings for subsidy purposes. The Department may request additional information, such as the editorial policy. The Department’s list will only include South African conferences not included in the international lists. The list will be made available on an annual basis together with the international lists. Institutions must submit to the Department, on or before 15 May of each reporting year, audited subsidy claims for research output appearing in approved indexed conference proceedings.

31. THE END

32. European Reference Index for the Humanities (ERIH) Fields of study: Humanities Coverage: Europe Languages: English, French, German, Spanish and Russian. Quality: ERIH expert panels Selection criteria: International cohort of contributions, High- quality scholarly content, Selectivity in articles published. Three journal categories: – A ( High-ranking international publications with a very strong reputation) – B (Standard international publications with a good reputation) – C (Journals with an important local/regional significance)

33. ULRICH’S Number of publications: 300,000 periodicals Fields of study: Broad Coverage: Broad global coverage Languages: Over 200 languages Criteria for inclusion: Covers publications that meet the definition of a serial except administrative publications. Refereed Serial: Ulrich assigns this status to a journal that has been peer-reviewed. Australia makes use of this index (Refereed only) for its Research Outputs Assessment.

34. Other Indices Considered SCIENCE DIRECT 2,500 Journals Fields: Broad Languages: English and other European Languages The list only covers journals published by Elsevier. LILACS 807 Journals Fields: Health Sciences Languages: English, French & Spanish Very few (less than 10%) English language journals.

35. Problems with Journal Indices ERIH – Too narrow subject coverage – Are all covered in Scopus Ulrich’s – Hard to distinguish between refereed and non-refereed.

36. International Practice: Australia Australia follows a similar research funding model to South Africa. In order for a journal article to be recognized it must: – Meet the definition of research – Be published in a scholarly journal – Be peer-reviewed – Have an International Standard Serial Number (ISSN) The following lists of journals are used: – ISI – Ulrich’s International Periodicals Directory – Department of Education Science and Training (DEST) Register of Refereed Journals (consists of 1238 journals) – Scopus (I think this is what we were told today??)

41. Accredited lists