2. 2 Journals in the arts and humanities: their role and evaluation Professor Geoffrey Crossick Warden Goldsmiths, University of London

3. 3

4. 4 Distinctive place of journals in arts & humanities research  Diversity of output monographs, edited collections edited texts, journal articles practice outputs  No clear hierarchy of esteem amongst as well as within each challenge for RAE and promotion panels esp problems for practice outputs

5. 5 Why do we want to know about journals? Explore the terms  Use of journals to indicate: activity and productivity impact - relevance and use quality - peer judgment

6. 6 Measurements and proxies Activity & ProductivityImpactQuality Quantitative Tangible & Qualitative methods & intangible methods & data dataevidence & data Increasing use of proxy metrics to infer impact and quality Especially in context of changes to RAE

7. 7 Why is it so hard to rank journals or use citation data?  Rank according to what criteria?  Impact factors very difficult  Citation behaviour is very different not cumulative cf science old work remains highly cited [See ISI list] critical discourses as mode of research citation not clear sign of quality/influence the culture of the footnote  Diverse outputs - not just paper-based  Print output different arts & humanities  Lower % paper outputs in ISI-Journals

8. 8 Highest citations in ISI humanities journals 2000 1 Karl Marx 2 Lenin 3 Shakespeare 4 Aristotle 5 The Bible 6 Plato 7 Freud 8 Chomsky 9 Hegel 10 Cicero

9. 9 RAE submissions 1996 & 2001 Humanities, Languages and Arts  Journal articles? 1996 - 33%2001 - 37% sciences 90% 96% engineering 57%78% social sciences 42% 54%  Books? 1996 - 51% 2001 - 52% sciences 6% 3% engineering 8% 5% social sciences32% 28%  Book chapters 1996 - 4% 2001 - 3%  Other 1996 - 11%2001 – 9%

10. 10 Journal publication & RAE quality Journal articles as % outputs to RAE2001 by discipline UoA3b3a455* Law8371615343 Asian studies4356554229 History373633 34 Art & design 7 710 9 Music1115 2327

11. 11 ISI journals & the arts & humanities  Publication in ISI journals often small % overall outputs RAE 2001  Philosophy highest at 52%  Library & information management 40%  Most other subjects in 20%-29% range  English & French just 21%  Below 20% in Italian, Theology, Art & Design and Middle Eastern & African Studies

12. 12 Current projects on quality & journals  Issue of evaluating quality very current  So too is assessing standing of journals  But they’re not the same thing at all  And decreasingly so arts & humanities  Reflect on these:  RAE and metrics  European Science Foundation’s ERIH  Humanities Indicators Project

13. 13 RAE metrics  Metrics-driven RAE  ‘Neither citations nor RC income’  AHRC expert group  outputs but no proxies for their quality  Post-2008: STEM cf rest of disciplines  ‘ robust indicators’ being sought – for STEM  primarily HESA and bibliometric data  arts & humanities 2013+? earlier impact  consultation non-science 2009-10  national bibliometrics consortium

14. 14 European Reference Index for the Humanities  European Science Foundation “ The ERIH lists will help to identify excellence in Humanities scholarship & should prove useful for the aggregate bench-marking of national research systems…in determining the international standing of the research activity in a given field in a country. However, as they stand, the lists are not a bibliometric tool”

15. 15 Methods & goals of ERIH  Expert Panels in 15 disciplinary areas  ESF member lists, iterative consultation  Categorisation (lists emerging this year)  A = high-ranking international level  B = standard international level  C = important local/regional level  Resistance to hierarchy: why?  Say above all to strengthen peer review  How, if at all, will it be used?

16. 16 Humanities Indicators Project  American Academy of Arts & Sciences  many variables on state of humanities  Publications element:  Focus is monograph publications data no quality indicators sought  No plans to look at journal publishing  Making humanities count: the importance of data

17. 17 Arts & humanities journals: the challenge of bibliometrics  Is there a challenge?  Does anyone want to do bibliometrics with arts & humanities journals?  Little interest UK or elsewhere  In many ways for good reasons set out here  Yet looming is RAE post-2013…. can we build ‘robust measures’ without them? can it be done with them?