2 Contents What is the REF? Elements of Assessment: Outputs, Environment, ImpactREF TimetableStaff eligibilityCode of PracticeStaff circumstances
3 What is the REF?Part of the system of funding HEIs Funding formula requires a measure of research quality Previously called Research Assessment Exercise (1986 – 2008), now Research Excellence Framework (2014 onwards) Quality judgements by discipline-based panels of expert peer reviewers Each unit of assessment awarded quality profile (0 to 4*) Results translated into funding formula multiplier
4 Selectivity and Performance Funding is increasingly selective% of QR funds to 26 HEIs% of QR funds to 23 HEIs, 27% QR funds to ‘big four’ HEIs– 30% to big fourSelective funding drives sector performance“in the period since the first RAE in 1986, the effectiveness and productivity of the UK research base has significantly increased”The Role of Selectivity and the Characteristics of Excellence, Evidence Ltd, 2000.Big four – Oxford, Cambridge, UCL and Imperial. Science helps! Also biggest for RG&C income
5 Quality ProfilesBy percentage of research activity in the submission judged to reach quality standardSub-Panel xCat A FTE Staff submitted for assessment4*3*2*1*UnclassifiedUniversity A501525405University B204510Sub-panel = subject
6 Overall quality profile: education 204*10153025u/c1*2*3*Quality Level% of Research Activity10%70%20%Research EnvironmentEsteem IndicatorsThe overall quality profile comprises the aggregate of the weighted profiles produced for outputs, research environment and esteem indicators – rounded to nearest 5%Research Outputs4*3*2*1*u/c152535104*3*2*1*u/c2515204*3*2*1*u/c3020106
7 Funding the RAE2008Funding Councils take profiles and devise a funding formula (each FC uses a different formula and it can change each year)Formula is adjusted to fit overall funds availableFurther performance improvement across the sectorPrioritisation of STEM subjectsFurther concentration of resources – islands of excellencevolume changesrelative quality changes
8 IOE Funding from RAE2008FTE entered 35% 4*, 30% 3*, 25% 2*, 10% 1* Funding allocated 3:1 in favour of 4*:3*, no funds for 2* and lower Funding in 2012/13 of £7,854,108
9 Research Excellence Framework Selection of outputs from members of “research active” staff Research environment: grant funding, PGR students, research degrees awarded New element of Impact Profiles will be 65% outputs, 15% environment, 20% impact
10 REF Timetable Intention to submit: December 2012 Submit: November 2013 Results: December 2014Funding decision: March 2015Funding: August 2015Census date October 2013Outputs 1 January 2008 to 31 December 2013Environment and Impact 1 January 2008 to 31 July 2013Data August 2008 to 31 July 2013
11 Elements of Assessment Research outputs 65%Originality, significance and rigourEnvironment 15%Sustainability and vitalityImpact 20%Significance and reach
12 OutputsThe sub-panels will assess the quality of submitted research outputs in terms of their ‘originality, significance and rigour’, with reference to international research quality standardsAll forms of outputs: articles, books, chapters, exhibitions, performances, compositions, media, reports, patents, software, etc.Outputs can be double-weighted if they consist of a substantial amount of work, but a reserve publication can also be submitted in case the panel don’t accept it as a double output.The deadline for publication of outputs is December 31st That means that the output must be in the public domain by that date. As submission is 29th November, it means that you mare allowed to submit details of outputs which you know will be available by 31st December.
14 Environment Criteria: Vitality will be understood as the extent to which a unit provides an encouraging environment for research, has an effective strategy, is engaged with the national and international research and user communities, and is able to attract excellent postgraduate and postdoctoral researchersSustainability will be assessed by considering leadership, vision for the future and investment in people and infrastructure and, where appropriate for the subject area, the extent to which activity is supported by a portfolio of research funding
15 Environment: elements Data relating to research income and doctoral degrees awardedOverview – 0%, Panel C gives equal weighting to all other sectionsEnvironment template gives us seven pages to talk about how the institute supports and develops researchers and research. Institutional, faculty and departmental level.
16 Impact Two criteria for impact: how significant or transformative the impacts have beentheir reach (how widely the impacts have been felt)For the REF, HEFCE have defined two main criteria for assessing impact: how significant or transformative the impacts have been and their reach or how widely the impacts have been felt. To understand how these can be assessed, think of two examples of impact. The first details the development of a drug which completely cures a disease. But the number of people suffering from the disease is relatively small. So whilst the significance is very high, the reach is limited. The second details the development of a drug which helps everybody suffering from minor aches and pains feel slightly better slightly faster than competitor drugs. For this example, the significance is relatively low, whilst the reach is potentially enormous. For the purposes of the REF assessment we would expect both of these cases to receive a similar rating.
17 Impact: general guidance Impact will count for 20% of overall profileResearch conducted at institutionUnderpinning research has to be of high quality (deemed ≥2* or originating from a peer-reviewed competitive project)2 case-studies up to FTE, 1 more every 10 FTEResearch between January 1993 and December 2013Impact occurring between January 2008 and July 2013Impact statement to describe approach to impact and future strategies (20%)Assessors from user communitiesWith impact making up 20% of an institution’s overall profile, it’s obviously an important part of an institution’s preparations for the REF. Imagine you’re an institution that receives £10m a year from QR funding. Impact is worth £2m and your case studies are worth £1.6m. That’s a lot of 4* publications!The research underpinning your case studies has to have been conducted at the submitting institution. Whilst academics take their outputs with them when they move institutions, impact stays with the institution where the research was carried out. There’s a lot of debate about whether it’s a good idea to submit case-studies based on research by staff who’ve left the institution - if the member of staff left under less-than-ideal circumstances for example, would they co-operate with the preparation of a case study? Can one be prepared without their input? Likewise, an institution with a new member of staff whose previous work has had impact would not be able to submit that case study unless further research had been carried out that generated additional impact – all within the REF period, of course!Research needs to be 2* or above – internationally recognised for its originality, significance and rigour. Or originating from a peer-reviewed competitive project.Each unit of assessment submits 2 case studies for up to FTE and one additional case study for every 10 FTE thereafter. This could mean some difficult decisions, if a UoA with 24 staff suddenly adds a couple of members of staff in the lead-up to the REF, that will mean an additional case study will be needed. And at this stage it’s unlikely that the new members of staff will be able to submit a case study (unless they are a former member of staff returning to the institution, which does happen). If an additional, strong case study isn’t available, what does the UoA do?Research needs to have appeared in the public domain between 1 January 1993 and 31 December 2013 – so outputs which are definitely in press and will be available after the submission deadline are allowed. Some UoAs (i.e. Architecture) allow a longer period for the research to recognise the long time lag between research and impact in that area. Impact has to have occurred between 1 January 2008 and 31 July So impact can have occurred before the research is published. This recognises that not all impact stories are linear. There can be occurrences when impact occurs during the research period, for example a project carried out in a local hospital which identified gaps in service and the NHS trust funded additional services whilst the research was still going on.The case studies count for 80% of the impact assessment, and we also submit a template which describes what the institution has been doing to support and encourage impact and what it plans to do in the future, drawing on the submitted case studies and the lessons learned from how their impact was developed.It’s important to note that there will be representatives from user communities for each UoA on the assessment panels. Some of these will be recruited to the panels following the Survey of Submission Intentions in December, which will identify the areas of impact that each Institution plans to submit, allowing the panels to recruit the appropriate numbers of assessors with the appropriate expertise. These were invaluable in the Impact Pilot in and enabled the panels to make distinctions between similar examples of impact through their knowledge of the specific circumstances.
18 Staff eligibility Contract of at least 0.2 FTE All staff on ‘teaching & research’ or ‘research only’ contracts as per HESA definitionsIndependent researchers – usually Grade 8 and above, but some Grade 7 staff will be eligible (i.e. John Adams Fellows)Not eligible if employed to carry out another individual’s research programme rather than as independent researchers in their own rightUsually 4 outputs for each staff member submittedYou are deemed to have started your career as an independent researcher when you have a contract of 0.2FTe or above which includes a primary employment function of undertaking “research” or “teaching and research” with any HEI and undertook independent research, leading or acting as principal investigator or equivalent on a research grant or significant piece of research work. It’s not enough to be named on research outputs. Research assistants are ineligible. (paras of Guidance on submissions)
19 Code of PracticeInstitutional practices that ensure the REF submission is transparent, inclusive and consistentStaff circumstances survey – December 2012Standard circumstances (maternity/early career/part-time)Complex circumstances (case by case)Inclusion threshold – not yet decided
20 Clearly defined circumstances: ECR Date at which the individual first met the REF definition of an early career researcher:Number of outputs may be reduced by up to:On or before 31 July 2009Between 1 August 2009 and 31 July 2010 inclusive1Between 1 August 2010 and 31 July 2011 inclusive2On or after 1 August 20113
21 Clearly defined circumstances: part-time working, secondment, career break Total months absent between 1 January 2008 and 31 October 2013 due to working part-time, secondment or career break:Number of outputs may be reduced by up to:1246 or more3
22 Clearly defined circumstances: maternity, paternity or adoption leave Individuals may reduce the number of outputs by one, for each discrete period of: Statutory maternity leave or statutory adoption leave taken substantially during the period 1 January 2008 to 31 October 2013, regardless of the length of the leave Additional paternity or adoption leave lasting for four months or more, taken substantially during the period 1 January 2008 to 31 October 2013.
23 Complex circumstances Circumstances which have impacted on an individual’s ability to work productively throughout the assessment period Confidential submission via the staff survey Panel will make decisions about the number of outputs each member of staff needs to submit Panel will consist of HR, RCS and academic representatives Appeals panel
24 IOE PreparationsInternal REF exercise Reviews over Spring term with feedback April 2013 Decisions about threshold for inclusion made by Director, advised by Pro-Director, Research and Development and theme leaders Combined with results of staff survey to determine which staff will be submitted
25 Any questions? Institute of Education University of London 20 Bedford WayLondon WC1H 0ALTel +44 (0)Fax +44 (0)Web ioe.ac.uk