1. Law Sub-panel Generic Feedback - Impact
UWE and BAM: maximising research impact
27 April 2016

2. Feedback Session for Law
The sub-panel and assessors
The submissions
The process
The outcomes

3. The law sub-panel and assessors
17 academic members
2 user members
8 academic assessors, including one who also assessed for SP22, SP23 (Criminology); some expert advisors for foreign language outputs
6 impact assessors

4. What was assessed
Panels judged the overall quality of each submission, made by 67 units
Quality of research outputs
Impact of research on society
The research environment
65%
20%
15%
The review was based on data and information about the environment
5,525 research outputs by 1,553 Cat A staff were reviewed (3.36 per person)
225 impact case studies were reviewed

5. The Process – allocation
Impact:
Case studies – Assigned to one user member/assessor and two academic members as lead readers
Templates – Assigned to one user member and two academic members as lead readers
All judgments arrived at individually, then discussed, and then determined in plenary
Strict procedures were in place to avoid conflicts of interest
Consistency of judgement
Calibration exercises were conducted at both main and sub-panel level for each element of the submission, with guidance from MPC feeding into sub-panel considerations

6. The outcomes: Development of sub-profiles
Each impact case study, the impact template, was graded on a scale from 4* - U, using ‘half-marks’ for borderline judgments
Each sub-profile for each submission was collectively agreed by the sub- panel and recommended to the main panel for approval.
The impact sub-profile was aggregated with environment and research quality into the overall quality profile for each submission, according to the standard weightings
NOW ONTO OUTCOMES FOR LAW
Each impact case study, the impact template, and each scored section of the environment template was graded on a scale from 4* - U, using ‘half-marks’ for borderline judgments
Criteria for impact were reach and significance – panel formed an overall view of that, taken as a whole, rather than assessing them separately.
For the templates the panel considered the extent to which a unit’s approach described in the template was conducive to achieving impacts of ‘reach and significance;.
4 = outstanding; 3 = very considerable; 2= considerable; 1 = recognised but modest; U = little or no, impact ineligible or not underpinned by excellent research that produced the submitted impact
Each sub-profile (impact, environment and research quality) for each submission was collectively agreed by the sub-panel and recommended to the main panel for approval.
The overall quality profile for each submission aggregates the three sub-profiles, according to the standard weightings

7. Average overall quality profile for submissions in Law (FTE weighted)
HIGH LEVEL OUTCOMES FOR LAW
27% of submissions were 4*
45% were 3*
24% were 2*
3% were 1*
1% was Unclassified

8. The average sub-profiles: Law compared with MPC
Profile Type
%4*
%3*
%2*
%1*
% Unclassified
Law
MPC
Outputs
20.1
21.1
47.1
43.0
28.7
29.8
3.7
5.5
0.4
0.6
Impact
38.3
39.1
41.1
40.3
17.7
16.4
2.4
3.6
0.5
Environment
43.6
39.9
43.1
39.2
11.7
1.6
3.0
0.2
Overall 27 46 42 23 26 4 1
Overall, comparing averages, law did better than MPC (combining 4* and 3*)
Impact scores broadly similar to the MPC average = 79.4% for both combining 4* and 3*.

9. Impact 225 impact case studies
Subject-matter included (but was not limited to)
criminal justice/human rights
regulation of business, commerce, environment
medicine and bio-ethics
EU and international law
Audience primarily
policy and law-makers
judiciary and the Bar
Small number of examples related to public engagement and legal education

10. Impact case studies: strengths and weaknesses
Gave clear indication of what was the underpinning research and explanation for its 2* (at least) quality
Understood the distinction between dissemination and impact
Gave clear explanation of how the research had been utilised to bring about change or effect
Provided clear (and not overstated) account of the reach and significance of the impact
When corroborating sources were followed up, they justified the claims made
Failed to set out the thread of evidence linking the research to the impact and failed to establish the 2* quality of the research
Did not understand the distinction between dissemination and impact
Made unconvincing or overstated claims of the reach and significance of the impact
When corroborating sources were followed up, they did not back up the claims made

11. Impact templates: strengths and weaknesses
Provided evidence of thought in identifying pathways to impact and a strategic approach reflecting the unit’s own research context and priorities
Understood the distinction between dissemination and impact
Had (or planned) pro-active mechanisms for linking with end-users
Assisted and supported staff to develop impact skills whilst recognising that not all research leads to non-academic impact
Offered an unimaginative list of possible ways of engaging with end-users
Did not demonstrate a clear understanding of impact
Lacked a clear strategic approach to moving forward
Were unambitious in their plans for achieving impact in future

12. Law’s quantitative data for research, compared with Politics
Law (1,553 FTE)
Politics and International Studies (1,275 FTE)
Research income
£77.4m
£144,180m
Doctoral awards
1,586
2,198

13. Further information www.ref.ac.uk includes:
The results and submissions
Summary data and analysis
Panel overview reports
Confidential feedback on submissions was provided to institutions in January
Evaluations of the REF are currently being carried out by the funding bodies, with reports due in March