1. The Research Assessment Exercise in the United Kingdom Paul Hubbard International colloquium “Ranking and Research Assessment in Higher Education” 13 December 2007

2. The Research Assessment Exercise (RAE) Background: research funding in HE in England The RAE 1986-2008 – why and how Impact of the RAE (positive and negative) Moving on: the Research Excellence Framework from 2009

3. Background: research funding for HE in England Funding from multiple sources Government funding for infrastructure and projects Funding allocated selectively based on excellence: some Universities funded for research in many disciplines, others in few

4. English HEIs’ research income by source, 2003-04 Total = £3213M

5. Government funding for infrastructure and projects The Dual Support System Funding for projects and programmes through the Research Councils Block grant to support the research infrastructure, to underpin research of public benefit funded by other sources, and for “blue skies” research at the discretion of the University – through HEFCE using RAE

6. The 'top' four HEIs get 29 per cent of HEFCE research funds The 'top' ten HEIs get 50 per cent of HEFCE research funds The 'top' 23 HEIs get 75 per cent of HEFCE research funds

7. RAE – why? Accountability – to show what public money is paying for Funding – drives the allocation of funding councils’ research grant Public information and benchmarking – where is the best research and how good is it?

8. RAE: How? (1/4) Periodic assessment of quality in all disciplines in all UK Universities and colleges at department level Expert review based on written submissions Review by panels – mainly practising academic researchers but also including “users” Outcome expressed as a 7 point scale (2001) but now as quality profiles (2008) for each unit submitted

9. RAE: How? (2/4) RAE Statistics for 2001(UK): 2,600 departments submitted by 173 Universities and colleges 47,000 research active staff employed by submitting institutions

10. RAE: How? (3/4) Written submissions for assessment: Identify research active staff to be considered and up to 4 published outputs for each Information about research income and research students Other contextual information including research plans

11. RAE: How? (4/4) Submissions assessed by 67 subject panels, moderated by 15 main panels, looking at: Quality of cited outputs (panels will read a varying proportion of these) Evidence of peer esteem Research environment The primary criterion is research excellence measured against common quality descriptors expressed in terms of international standards.

12. Quality profile: definitions 4* Quality that is world leading in terms of originality, rigour and significance 3* Quality that is internationally excellent in these terms but falls short of the highest standards 2* Quality that is recognised internationally in these terms 1* Quality that is recognised nationally in these terms Unclassified: below 1* or not recognised as research within the exercise

13. Quality profile (2008): examples Unit of assessment A FTE staff submitted for assessment Percentage of research activity in the submission judged to meet the standard for: four star three star two star one star un- classified highest lowest quality University X 50152540155 University Y 2005404510

14. RAE impact: positive (1/2) Funding informed by quality judgement Strategic management of research Performance management Focus on quality wherever it is found – leading departments, not just leading universities Quality of UK research (as measured by both RAE and citation indices) has risen steadily since late 1980s

15. Improved ratings for total submitted staff over time (UK)

16. HEIs achieving a maximum rating of 5 or 5* in one or more UOA (UK)

17. RAE impact: positive (2/2) Critical mass – particularly in hard sciences Benchmarking data nationally and internationally Broad confidence in RAE judgements

18. RAE impact: negative Distorting institutional missions: chasing the funding Encouraging more volume Games playing Administrative burden

19. RAE impact: myths and perceptions Publish or perish Destroying careers Discouraging applied research Interdisciplinary research

20. Moving on: the Research Excellence Framework (REF) RAE has achieved its objectives Increasing concern about bureaucracy and behavioural impact Increasing acceptance that quality can be measured by quantitative indicators and indirect peer review

21. REF: Main features Science based disciplines: assessment and funding by “metrics” – citation indices, research income and research students – in 6 broad subject groups Non science disciplines – radically reduced peer review informed by “metrics”

22. REF: implementation Phased implementation: for sciences gradually from 2010, for other disciplines from 2013-14 We are developing citation indices that will produce robust quality indicators at broad subject group level for each HEI expressed as a quality profile – pilot exercise in 2008

23. For further information On RAE 2008: www.rae.ac.uk On RAE 2001 and earlier: www.hero.ac.uk/uk/research On HEFCE research funding: www.hefce.ac.uk/research