1. CERIF for research evaluation including REF Keith G Jeffery STFC / euroCRIS keith.jeffery@stfc.ac.uk 20/04/20111Ready 4 Ref End of Project Workshop www.eurocris.org

2. Structure CERIF CERIF in Use Evaluation REF Results 20/04/2011Ready 4 Ref End of Project Workshop2

3. Project Person / CV Institution Event Equipment Books Journal/article Patent Research Group Publisher Some entities (objects) of interest

4. CERIF Model EU Recommendation to Member States

5. PROJECTORGUNITPERSON Result_Publication RESULT_PUBLICATION Concepts: (1) temporally-bound role linking relations (2) >1 linking relation : Result_Publication and other entities (3) PERSON role may be author, co-author, editor, reviewer…. (4) ORGUNIT role may be publisher, IPR or copyright owner, funder.. (5) PROJECT role may be the source of the idea

6. RESULT_PUBLICATION PROJECT ORGUNIT PERSON Result_Publication Can Express (assertions in pseudo-English): Person A (DT1 - DT2) (is author of) Publication X Orgunit O (DT1 - DT2) (is owner of IPR in) Publication X Person A (DT1 – DT2) (is employee of ) Orgunit O Person A (DT1 - DT2) (is project leader of) Project P Person A (DT1 - DT2) (is member of) Orgunit M Person A (DT1 - DT2) (is member of)Orgunit N Orgunit M (DT1 - DT2) (is part of) Orgunit O Orgunit N (DT1 - DT2) (is part of) Orgunit O

7. Result_Publication Instance Diagram Person A Publication X OrgUnit O OrgUnit M OrgUnit N Project P member employee Part of owns IPR author Project leader

8. CERIF-CRIS at One Organisation Publication repository Dataset Software repository Finance system Human Resources system Project Management system CERIF-CRIS Web pages Directory Services This is fine for one organisation but research is international, so…

9. CERIF Interoperation CERIF-CRIS CERIF provides interoperation of CRIS and associated systems with formal syntax and declared semantics so that it is reliable and scalable. Interconnect Backplane

10. CRIS + Repositories at 1 institution CRIS Research Context [projects, persons, organisational units funding, products, patents, publications facilities, equipment, events] OA Repository (hypermedia) Documents e-Research repository Datasets and Software OAI- PMH Various protocols End-User CERIF

11. ….and multiple institutions CRIS OA repository e-Research repository CRIS OA repository e-Research repository CRIS OA repository e-Research repository End-User Institution AInstitution BInstitution C

12. Key Points Formal syntax and declared semantics Extensible yet backward-compatible Multilingual Interoperable Can generate semantic web EU Recommendation to member states 2000 Following independent technical and business case studies adopted for UK by RCs, HEFCE, UUK and others 2010 20/04/2011Ready 4 Ref End of Project Workshop12

13. Structure CERIF CERIF in Use Evaluation REF Results 20/04/2011Ready 4 Ref End of Project Workshop13

14. CERIF in Use National systems: IS, NO, DK, NL, SK, SL, BE, RU (SE) In use in funding organisations and universities in many more countries: DE, FR, IT, FI, IE, GR, CZ In UK ~40 universities using it or working on it HEFCE, UUK, RCUK agreed (2010) CERIF is UK standard for research information Used for REF and for RC evaluation of research output Native CERIF-CRIS or CERIF-wrapped legacy CRIS

15. Structure CERIF CERIF in Use Evaluation REF Results 20/04/2011Ready 4 Ref End of Project Workshop15

16. Evaluation Research Environment – ‘standing’ including professional activities – ‘reputation’ of institution Quality and Quantity of Research – Securing research funding Outputs of Research – Scholarly works – Patents – Products Impact of Research – Wealth creation (including cost savings) – Improvement in the quality of life – Further research Funding Funding organisations Research organisations Facility managers Innovation managers Innovators Media / Public 20/04/2011Ready 4 Ref End of Project Workshop16

17. RC Evaluation RCS Fund ~£3B per year: peer reviewed proposals No final reports Assess outputs and outcomes Scholarly works in institutional repository – linked to grant reference (in CRIS) – Linked to organisational unit (in CRIS) – Linked to person (in CRIS) In time patents, products etc – At present only evidence is case studies 20/04/2011Ready 4 Ref End of Project Workshop17

18. HEFCE Evaluation HEFCE Funds ~£1.6B per year of which £1.3B is quality related by subject by institution From RAE to REF Still basic parameters – Quality of research – Research environment And adding more strongly – Impact of research 20/04/2011Ready 4 Ref End of Project Workshop18

19. Structure CERIF CERIF in Use Evaluation REF Results 20/04/2011Ready 4 Ref End of Project Workshop19

20. REF: Research Excellence Framework Not defined yet but 3 aspects: – quality of research, impact of research, research environment Likely to have similar parameters to RAE Expert Review plus indicators – Not totally automated based on measures Impact: historical not future Environment: vitality – Equalities and diversity http://www.hefce.ac.uk/pubs/circlets/2010/cl04_10 /cl04_10a.pdf Use of HESA statistics Use of templates Expert panels 20/04/2011Ready 4 Ref End of Project Workshop20

21. Structure CERIF CERIF in Use Evaluation REF Results 20/04/2011Ready 4 Ref End of Project Workshop21

22. Results Initial assessment of CERIF for REF purposes at euroCRIS members’ meeting St Andrews November 2009 – Only 2 additional attributes needed – Principle of atomic attribute values and aggregation Ready 4 Ref project analysis and structure – Ensure required measures are in CERIF data model Only 2 additional attributes needed – Ensure use of CERIF XML for data interoperation demonstrated 20/04/2011Ready 4 Ref End of Project Workshop22

23. And looking forward MICE project new structure – Attempt to ensure measurable parameters for impact are in CERIF data model New group of entities with attributes Proposal from MICE to euroCRIS CERIF Task Group for discussion May 12-13 Copenhagen 20/04/2011Ready 4 Ref End of Project Workshop23