REF2014 HODOMS Birmingham 8 th April Ann Dowling: Chairman of REF Main Panel B John Toland: Chairman of REF Sub-Panel B10: Mathematical Sciences

The REF assessment framework First meeting of Sub-panels Graeme Rosenberg, REF Manager

The assessment framework Overall assessment framework Main panel criteria and working methods Sub-panel specific input The REF is a single assessment framework, within which panels have flexibility to develop aspects of the criteria and working methods

Key documents During 2011, our task is to produce two key documents: Guidance on submissions Panel criteria and working methods These will set in stone what HEIs need to include in their submissions, and how the panels will assess them

Timetable 2011 Impact decisions (Feb) Panel membership (Feb) Guidance on submissions (Jul) Draft panel criteria (Jul) 2012 Final panel criteria and methods (Jan) Test submissions system 2013 Submissions deadline(Nov ) Recruit additional assessors 2014 Assessment Publish outcomes (Dec)

The overall assessment framework Overall quality Outputs ‘rigour, originality and significance’ Maximum of 4 outputs per researcher Environment ‘vitality and sustainability’ Template and data Impact ‘reach and significance’ Template and case studies 65% 15% 20%

Outputs EnvironmentImpact 4*3*2*1*u/c *3*2*1*u/c % Overall Quality Profile 12 4* u/c1*2*3* 4*3*2*1*u/c %20% Quality Level % of the submission

Submissions Each submission to a UOA will include staff selected by the HEI, their research outputs and associated environment and impact: Normally one submission by an HEI to a UOA Multiple submissions in exceptional cases Joint submissions Sub-panels to produce UOA descriptors and boundaries

Research staff Submissions to include: Details of eligible staff selected by the HEI Up to 4 outputs per selected researcher The number of outputs may be reduced without penalty for staff with circumstances that constrained their productivity: – Early career researchers – Clearly defined periods of absence – Complex circumstances

Research outputs All research outputs placed in the public domain during the publication period (1 Jan 2008 to 31 Dec 2013) will be eligible Outputs from all types of research to be treated equally Main panels to expand on: – The diversity of expected output types – How co-authorship will be handled – Requirements for additional information – Criteria for ‘double-weighting’ outputs of exceptional scale and scope

Citation data We expect some panels will use citation information to support their review of outputs: We will identify a standard approach to making data available to panels and provide guidance: – Judgements based on expert review; citations only inform ‘academic influence’ – All types of research to be treated equally – Clear limitations for recently published outputs Main panels to: – Decide whether or not citation data will be used – Be explicit about any use of citation data

Research environment A structured template to describe, for example: – Overview – Strategy – People – Income, infrastructure and facilities – Collaboration and contribution Standard data on research income and research students Main panels to explain: – What kinds of evidence or information are required – The relative importance of each section – The approach to forming an environment ‘sub-profile’

Research impact A broad generic definition of (non academic) impact Sub-panels will make judgements based on: – Case studies detailing specific impacts – Information about how the unit has supported and enabled impact Generic templates and scale for the number of cases Main panels to expand on: – How the criteria (reach and significance) will be applied – Types of impacts and evidence appropriate to their disciplines – Suitable evidence of underpinning research quality – How the ‘impact template’ will influence the sub-profile

Equalities and diversity We have built on the measures taken in the 2008 RAE: A consistent approach to enabling staff with constraining circumstances to submit fewer outputs All institutions to adhere to a code of practice on the fair and transparent selection of staff Evidence of support for equalities and diversity (within the research environment) Analysis of selection rates at sector level Panels will be briefed on equalities Ongoing advice from an equalities expert panel

Panel working methods Working methods to explain: How the main panels will in practice carry out their roles in providing oversight and governance How the sub-panels will in practice carry out the assessment, including: – The approach to making use of assessors – The approach to allocating and reviewing submissions – Procedural matters

A note on assessors For the criteria phase, the funding bodies have appointed panels with broad expertise to develop the criteria; and with numbers in proportion to each panel’s remit Additional assessors will be appointed in 2013 where the breadth or depth of a panel’s expertise needs to be extended, to undertake the assessment: – Assessors (either academic or user) will play a full and equal role to panel members in developing the sub- profiles (either outputs or impact) – They will be trained and briefed and attend meetings as necessary during the assessment phase