Exeter – Implementation of a Crosswalk Connector S. Trowell, University of Exeter Nov 2013

The problem from an Institutional perspective 2 The 3 UKRISS partner Institutions of Kings, Brunel and Exeter all have different organizational structures, methods of data processing and system architectures. Such differences are common across the sector; in fact no two institutions will be the same in this regard. However, it is also very apparent that we all share the same common issues: Many manual processes to support all forms of information flow between the Institution and funders, particularly in support of research reporting. Responsibility for completion of many reporting streams is split between PI’s and central research support administrative staff, with little visibility of data entered. Much of the required data resides across a range of internal systems, each storing the data in different database structures and with different naming conventions for data fields. Institutions recognize that improving information flow processes would yield benefits, yet competing institutional priorities mean only limited resources can be invested in this area.

Exeter – Implementation aims 3 Deliver a proof of concept to demonstrate principles for automation of data from institutions. Since all Institutions are different in terms of systems architecture, need a ‘Universal Connector’. Demonstrate how an open-source crosswalk connector can be used to: Extract information from existing source institutional systems Arrange the data into templates that reproduces the data required for ROS and Research Fish Convert this native data into a standardised data model [CERIF] Deposit the outputs in an accessible location to enable recipient [funding body] to easily and securely access the data and decode into a suitable format for ingest into funder’s internal systems. Demonstrate how the connector can be used in conjunction with data validation tools to improve the underlying data quality.

The approach 4 Focus on the ROS and Research Fish requirements by way of an example. Reproducing the current ROS and RF outputs, define a global data set that encompasses all ROS and RF requirements. Working with a local [Exeter] software supplier, CERTUS Technology Associates, to develop an open source connector tool. Demonstrate this tool working in conjunction with Cottage Labs’ data validation tools. Demonstrate how this approach may be used for data sets other than ROS and Research Fish and therefore be used to directly benefit all stakeholders [funders, public bodies and institutions].

Connector Detail – what is it? Open Source Pentaho Data Integration Community Edition (PDI CE), sometimes referred to as Kettle (Kettle Extraction Transformation Transport Load Environment). Graphical ‘drag and drop’ environment to combine extract, transform and loading steps. Able to schedule ‘jobs’ to run at a pre-determined time. UKRISS Connector methodology 5 External to Institution, hosted database Sponsors Funding Body Destination server Institution X Research System A HR System Finance System Student System Secure ftp Reporting capability Research System B Connector Tool

UKRISS Connector methodology 6 Institution X Research System A HR System Finance System Student System Research System B Connector Tool Map data fields Convert to CDM Create output e.g. combined ROS/RF output Map to CERIF Apply data validation tools

Connector Detail – how does it work? 7 Extract - mapping data fields: Integrating multiple sources of information relating to the same entity [e.g. a project or person] requires ability to identify matching records from these systems. For each data field, UKRISS has agreed name, type, format, meaning, vocabulary, and representation in CERIF. Transform - convert to CDM: Source data may be in a range of formats: Database input to CDM CSV file input to CDM XML to CDM Transform – convert the CDM into the latest version of CERIF XML. Load – output to a destination(s), including security layer Output data may be in a range of formats CDM to database CDM to CSV file CDM to CERIF XML Connector Tool Map data fields Convert to CDM Create output Map to CERIF

Simple example of a transform to generate CERIF XML 8

Specifying the fields to be ‘CERIFised’ 9

A Flexible approach… different data, different directions 10 This concept may be used for data sets other than ROS and Research Fish, e.g. HE-BCI survey. The Crosswalk Connector functionality is reversible so could use within Funders use to pick up Institutions’ outputs and ingest into existing internal systems External to Institution, hosted database Sponsors Funding Body Destination server Institution X Research System A HR System Finance System Student System Secure ftp Reporting capability Research System B Connector Tool

A Flexible approach… different data, different directions 11 Sponsors Funding Body Institution X ROS or RF report Connector Tool Institution z ROS or RF report Institution y ROS or RF report External to Institution Reporting capability

A Flexible approach… different data, different directions 12 This concept may be used for data sets other than ROS and Research Fish and therefore be used to directly benefit all stakeholders [funders, public bodies and institutions]. The Crosswalk Connector functionality is reversible so could use within Funders use to pick up Institutions’ outputs and ingest into existing internal systems. Or, could use to aggregate Funding Opportunity announcements for coordinated dissemination to Institutions.

Receiving CERIF XML messages 13

In summary 14 Implementations are only at a proof-of-concept stage. Production solutions would require further investment and development and consensus across the sector. The UKRISS ‘Universal Connector’ is a modular approach, fully compatible with additional data validation tools. Provides a flexible and sustainable systems architecture that can grow as CERIF standards evolve. Acknowledgement: thanks to Dr Brian Lings, Certus Technology Associates For Further information contact Dr Steve Trowell, Exeter: