Research Impact Impact and the REF Dr Lucy Davies Research Impact & Quality Officer, Research & Innovation Plymouth University Impact and the REF by Dr Lucy Davies is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.Dr Lucy DaviesCreative Commons Attribution-NonCommercial-ShareAlike 4.0 International License

Research Impact What is research impact? “Research impact is a recorded or otherwise auditable occasion of influence from academic research on another actor or organization” LSE Knowledge exchange/co-production “Impact is the good that researchers can do in the world” Professor Mark Reed (Newcastle University, Fast Track impact)

Research Impact What is research impact: Economic and Societal Impacts “An effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia.’’ HEFCE “The demonstrable contribution that excellent research makes to society and the economy. Economic and societal impacts embrace all the extremely diverse ways in which research-related knowledge and skills benefit individuals, organisations and nations by: fostering global economic performance, and specifically the economic competitiveness of the United Kingdom, increasing the effectiveness of public services and policy, enhancing quality of life, health and creative output.” RCUK

Research Impact Why is it important? Accountability Public dialogue and engagement Funding – REF (retrospective) – RCUK – H2020 Quality & reach of research Resource management and assessment

Research Impact Impact and REF 2014 “An effect on, change or benefit on the activity, attitude, awareness, behaviour, capacity, opportunity, performance, policy, practice, process or understanding of an audience, beneficiary, community, constituency, organisation or individuals in any geographic location whether locally, regionally, nationally or internationally.’’ HEFCE 20% weighting (80% case study/20% template) 2 studies per UoA + 1 for every 10 FTE staff above 14.9 Research conducted at HEI/ at least 2*/20yr time period Impact generated within REF period, link to research Fit criteria/priority of Panels Evidence of reach and significance

Research Impact Impact and REF 202? Review - Lord Stern “We will explore ways in which a simpler, lighter-touch, system for the REF might be developed…that uses data and metrics more effectively while retaining the benefits of peer review” Impact still on the agenda “…strengthen the focus on research excellence and impact while reducing administrative burden on the sector” Consensus: Very little will change Follow REF 2014 Guidelines Timeline: Report July 2016, HEFCE consultation, guidelines Spring or 2019?

Research Impact REF2021? No guidelines! Assume Impact Case studies will remain Identify potential case studies now Consider research, impact and evidence Aim to have good mix of quantitative and qualitative evidence – try to adhere to standard measurements Use REF 2014 as Guide

Research Impact REF 202?: Underpinning research Eligibility – Time period: from 2000 onwards (in case of 2021 submission) – Quality: Majority 2-3* outputs – HEI: Research (that leads to impact) has to be conducted in Plymouth; other research can be used as background – Time line: Impacts can happen before publication, so long as can be justified Identify any complications now!

Research Impact Underpinning research: Complications Complications: – Where significant research has not been published yet, will it be published before the impact? If not, are there non 2* outputs (presentation of initial results, conference proceedings etc.) that could be used to demonstrate a linear narrative? – Are there any current or potential anomalies that will need explaining in a way that won’t invalidate the eligibility of the research researchers arriving/leaving, lead authors outside of the HEI etc. PhD students – Where there are collaborations, is the contribution of Plymouth to research explained clearly? – Is there a clear relationship between the impact and research? Will the case study be able to demonstrate that the impact could NOT have happened without the research?

Research Impact REF 202?: Impact Points to consider: – Eligibility Has impact been generated within REF period (from 2014) Is the impact generated outside the HEI – What are the key impact types (Economic, Political, Environmental, Health, Technological, Legal, Cultural and Societal)? – How well does the claimed impact meet with Panel guidance and priorities? – How did the research lead to impact?

Research Impact REF 202?: Impact Points to consider: – Who has benefited? Do stakeholders fit with Panel priorities? – Is the (potential) significance articulated clearly? Is the context made clear (remember even a £1,000,000 can be peanuts in some contexts!)? – What is the expected reach? Is it likely to be maximised (local impacts can count as much international ones, so long as that is where you expect to have impact and it is fully realised)? – Are there multiple impacts? Are they all of equal quality? – Are all the impact claims likely to be fully realised within the timeframe?

Research Impact Impact across the Panels Each panel produced guidance on impact – worth reading! – Panel A: UoA 1-6 Impacts: Health and welfare, Society, culture and creativity, Economy, Commercial, Public policy and services, Production, Practitioners and services, Environment, International development – Panel B: UoA 7-15 Impacts: Economy, Public policy and services, Society, culture and creativity, Health, Practitioners and professional services, Environment – Panel C: UoA Impacts: Creativity, culture and society, Economy, commerce or organisations, Environment, Health and welfare, Practitioners and professional services, Public policy, law and services – Panel D: UoA Impacts: Civil society, Cultural life, Economic prosperity, Education, Policy making, Public discourse, Public services

Research Impact Impact across the Panels Things to consider: – Emphasis of impact, e.g. economic impact Panel A: cost-savings, efficiencies in health industry/ NHS Panel B: risk assessment & management Panel C: management, regulation/governance/ social responsibility – Medical/Clinical education: needs to be outside HEI – Assessing research for 2* threshold

Research Impact Evidence Do you have evidence of impact and link to research? Will you be able to get it? What indicators are you using? Mix of qualitative and quantitative? Quality/Credibility? Can it be easily incorporated into narrative Is evidence best measure for claiming impact? Is it standard/validated? Baseline data/context (significance) Can you demonstrate reach adequately

Research Impact Citation in a public discussion, consultation document or judgement. Citation by journalists, broadcasters or social media. Citation by international bodies such as the United Nations, UNESCO, IMF and so on. Evidence of citation in policy, regulatory, strategy, practice or other documents. Evidence of debate among practitioners, leading to developments in attitudes or behaviours. Public debate in the media. Parliamentary or other democratic debate. Quantitative data relating, for example, to cost-effectiveness or organisational performance Visitor or audience numbers, or number of participants (for example, in the uptake of CPD). Media reviews. Measures of improved inclusion, welfare or equality. Independent documentary evidence of links between research and claimed impact(s). Documented evidence of influence on guidelines, legislation, regulation, policy or standards. Documented change to professional standards or behaviour. Satisfaction measures (for example, with services). Use in scrutiny or audit processes, such as Select Committees. Incorporation in training or CPD material. Outcome measures, including measures of outcomes for beneficiaries. Examples of Evidence (from Panel C)

Research Impact Narrative Is the narrative coherent, linear and chronological? Is there a clear story of change - leading from research to impact? Answer the What, Who, Why, When, Where & How questions? Is there an interdisciplinary or collaborative aspect to the case study? Is it fully explained? Can a reader without specialist knowledge understand the narrative?

Research Impact Key Points Is research eligible? Is there a clear link between the eligible research & impact Is the impact generated within the REF period What is the expected/claimed reach & significance – how would you grade it? Does the impact topic fit the panel criteria Can you maximise your claimed reach? Can you provide the context for the claimed significance? Are the impact claims verifiable? Realistic? What metrics can be used to evidence the impact?

Research Impact Main issues with REF case studies Lack of clarity over what constitutes impact Eligibility – e.g. impact belongs to HEI Lack of documentation and evidence Evidence not read by Panel Narrative skill

Research Impact Solution 1: Embed impact! Impact needs to be taken seriously (integral to research) The long-game Make connections Focused & realistic Use & create resources Measurable & documented

Research Impact Solution 2: Capture impact! Across sector – acknowledged need for tools to archive evidence, plan, institutional oversight & memory of research impact. We have adopted VV-impact tracker! Provides: – Evidence Vault – Institutional overview – Link outputs, grants, activities, stakeholder

Research Impact Log-in