1. W HERE IS O PEN A CCESS IN THE ARC' S ERA? John Lamp School of Information Systems

2. ERA assessment measures ERA ranking context ERA ranking analysis Information Systems journals Business journals OA journals (DOAJ data) Conclusions The next round O UTLINE

3. A CHEQUERED CAREER Projectionist Printer Public sector Pundit Provocateur Programmer Pedagogy Pontificator

4. N EW TECHNOLOGY AND THE A USTRALIAN P RINTING I NDUSTRY Produced by the PKIU in 1979 Arguably the first examination by a union of the impact of information technology on jobs

5. G OODBYE GUTENBERG Produced in 1980 Anthony Smith was Director of the British Film Institute

6. A VIEW FROM 1980 … it is into this empty terrain that the new electronic information systems will make their appearance during their pioneering period.

7. Peer review Books Book chapters Journal articles Conference publications Non-traditional research output Esteem Editor prestigious works of reference Membership of Learned Academy Category 1 research fellowships Membership of statutory committees Australia Council grants or fellowships ERA MEASURES

8. Applied Patents Registered designs Plant breeder’s rights NHMRC endorsed guidelines Research commercialisation income Citation Analysis Relative Citation Impact Centile analysis Relative Citation Impact classes ERA MEASURES

9. HERDC Research Income Category 1 Category 2 Category 3 Category 4 Volume and Activity Analysis Eligible researchers profiled by level Traditional research outputs by type Non-traditional research outputs by type ERA MEASURES

10. Ranked Outlets Journals Conferences ERA 2010 Discipline Matrix, http://www.arc.gov.au/era/key_docs10.htm ERA MEASURES

11. What is it? Quality? Influence? Prestige? Papers determine journal rankings, not vice versa Approaches Metrics Didactic pronouncements Consensus O UTLET R ANKING

12. “A journal ranking index is currently being developed, and the list will be finalised in consultation with the sector. More than 17,000 journals have been ranked across 100 disciplines. This is a larger number of journals than either the Thompson ISI or Scopus index use in their respective databases which contain approximately 15,000 journals.” Consultation Paper, June 2008, p11 W HY HAVE AN ERA R ANKING ?

13. Organic Chemistry Seven (12%) journals at A* Publish fortnightly - 182 issues/year Many people have many A* papers Information Systems Nine (5%) journals at A* 0806, less HCI, CS journals – see ACPHIS list Publish quarterly - 36 issues/year Few people have A* papers R ANKING IS DISCIPLINE SPECIFIC …

14. ERA W O S RANKING International focus required for ranking > B?

15. It’s the mix of publications that matter Remember all disciplines are different Looking at a marker for local relevance Peer reviews override mathematical formulas ERA VIEWS C A* A B

16. “Journal articles are defined as either indexed or non-indexed. An ‘indexed journal article’ is an article that has been published in a journal indexed by Scopus and has an electronic identifier (EID). For disciplines where citation analysis is used … a ‘non-indexed journal article’ is an article that has been published in a journal which is not indexed by Scopus and does not have a Scopus EID.” ERA 2010 Submission Guidelines, December 2009, p37 I NDEXED ARTICLE

17. “For disciplines where citation analysis is used, the low volume threshold is 50 indexed journal articles. This means that no evaluation will be conducted for the relevant FoR for a given institution if the number of indexed journal articles over the six year research outputs reference period is fewer than 50 in any four- or two-digit FoR.” ERA 2010 Submission Guidelines, December 2009, p12 V OLUME THRESHOLD

18. ACPHIS J OURNALS IN ERA & S COPUS Scopus InOut A*260 A352 B4210 C3431 Total13743

19. ERA JOURNALS Total number of journals ranked: 20,712 Total number of DOAJ journals in ERA: 1466 http://lamp.infosys.deakin.edu.au/era/ Access via ANZFoR Access via journal/conference title Access via conference acronym Draft listings also available 70,000 hits per month ARMS Excellence in Research Management Award, 2010

20. ERA DISTRIBUTION OF OA JOURNALS (N=1466) RankAllDOAJ A*5%1% A15%5% B28%20% C52%74%

21. ERA RANKING AND AGE : I NFORMATION S YSTEMS (N=541)

22. ERA RANKING AND AGE : B USINESS (N=1045)

23. OA JOURNALS BY START DECADE

24. ERA RANKING AND AGE OA JOURNALS (N=1453)

25. T EN C HALLENGES FOR OA J OURNALS Peter Suber – the Scholarly Publishing and Academic Resources Coalition SPARC OA Newsletter, November 2009 1. Measuring up – journal metrics 2. Opening up – use of CC license 3. Closing up – prestige vs quality 4. Doubts about quality – peer review used? 5. Doubts about preservation – dead OA journals 6. Doubts about honesty – especially fee based OA 7. Doubts about publication fees – corrupting peer review 8. Doubts about sustainability – bills vs workload vs quality 9. Doubts about redirection – of funds from trad. journals 10. Doubts about strategy – how to make them a success

26. C ONCLUSIONS Remember to take the issue of journal ranking in context Rankings aren’t everything Their interpretation is questionable The profiles are discipline specific Publication frequency is hidden No evidence of discrimination Darwin vs Machiavelli Natural selection assumes open choice Social Darwinism Ranking Eugenics!

27. Assuming no change to ERA program Assessment in 2010 Call for comments etc in 2011 Next ERA Ranking list in 2012 More open/transparent process Blogging comments on journals? T HE NEXT ROUND