Long live the REF!

 The RAE looks at three main areas: ◦ Outputs ◦ Environment ◦ Esteem  We are used evaluations of Environment and Esteem being “informed” by metrics  The intention is that under the REF all three areas will be metric determined.  This will require the introduction of bibliometrics into the REF.

 You might guess that the three most important words in the REF were: Unfortunately, you would be wrong!

 Well then you might, quite reasonably suspect that any three or the following would be strong candidates: Sadly, you would still be wrong!

 Without any doubt the three most important words in the REF are: The worrying thing here is that, at the moment, “WE” includes HEFCE!

 What bibliographic metrics will be used?  Will it be the publications of ALL academic staff in a UoA?  Will it be ALL publications or a selection?  Over what period will the data be taken? We – i.e. HEFCE – don’t know.

“Clearly there are still some details of the new procedure which remain to be finalised”. We do know one thing (hurray!)– HEFCE are adamant that this new system WILL inform funding from 2011 onwards!

 HEFCE have conducted a pilot study into gathering bibliometric data.  The pilot was to run concurrently with the 2008RAE to provide a basis for comparison.  Unfortunately this proved to be impossible – and so no benchmark is available!

 Involved 16 universities  Covered 35 UoA’s – good coverage in; Science, Computing, Engineering. Very little social science or arts coverage.  The candidates were asked to provide bibliometrics – including citation data on all staff in the UoA which were submitted to RAE2008.  The candidates were asked to provide the information as quickly as possible, consistent with quality checking.

 The “best” university managed to comply in 3 days!  One university took 150 days! – using an undisclosed, but substantial, number of staff.  In two years time we could be asked to do this on an annual basis!

 Difficulties with older or existing information.  The process must be on-going and not just pre-submission.  Data cleansing is very time consuming – formats, mistakes, missing fields etc.  Difficulties linking publications to staff and staff to UoA’s (HESA id).  In some subjects the coverage simply is not there – i.e. the information does not exist!

 Not a bit of it! ◦ Autumn 2009 – consult on main features of REF ◦ May 2010 – outcome of consultation, phasing in timetable, main operational features in place. ◦ – metrics begin to inform funding. ◦ 2012 – submissions to 2013 ‘light touch’ peer review process. ◦ 2013 – undertake full assessment using ‘light touch’ peer review and metrics ◦ 2014 – HEFCE research funding for all subjects driven by REF from this time on.

 JMU Working Party set up. This includes; ◦ UoA co-ordinators from science/technical, social science and arts subject areas. ◦ Colleagues from Computer Services and the Library. ◦ Representatives from Research Office.  Decided that we must plan for the “worst case” scenario. That is, we will be required: ◦ to return bibliometric data on ALL publications, ◦ by ALL academic staff, ◦ referenced back to UoA’s, ◦ Over a 10 year period, ◦ annually!

 Sympletic is a publications management system developed in collaboration with Imperial College.  What Sympletic does: ◦ We supply a list of names, aliases, addresses, affiliations and UoA data. ◦ Sympletic trawls databases looking for publications by these individuals. ◦ If it finds a candidate publication it s the person concerned and asks them to confirm if the paper is theirs.

 On confirmation... ◦ Sympletic will download all information regarding that publication. ◦ Sympletic will maintain this information in an updated condition – particularly citation data. ◦ Staff will be able to examine their own record and notify Simpletic of any errors or omissions.

 The plus points: ◦ Once set up, the system is self maintaining. ◦ Producing up to date reports at, multiple levels,  University  Faculty  School  Research Group  Individual ◦ Can automatically update websites ◦ Resource for REF, CV’s, Grant Applications, Corporate Publications etc. ◦ Sympletic “talks” to Oracle and Oracle HR.

 The minus points: ◦ We need a definite staff list – referenced to UoA’s ◦ Staff engagement  Personal benefits  Managerial benefits ◦ Issues with coverage – WoS and PubMed. Other databases? ◦ Does not solve the problem of the data simply not existing.

 Software Demonstration  If used the timetable would be: ◦ Spring 2009 – Pilot ◦ Sept 2009 – roll out ◦ March 2010 – full deployment.  Management input needed to promote staff engagement by adopting Sympletic for other processes:  Profs/Readers  Website maintenance  Validation/Accreditation documents  PDPR