EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th EuroRec Certification Guidelines and Procedures

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Generic Version Model Certification Guidelines & Procedures  Introduction  What is Certification?  Market Assessment  The Management of the Certification Process in Europe  Generic Certification Process  Generic Certification Documentation/Manual  Issues/Options in introducing Certification  Legal & Management  Organisation & Process  Next steps

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Introduction Generic Version, Model Certification Guidelines & Procedures  Describes “what” to certify and “how” to perform the certification  Documents preliminary model - generic guidelines and  procedures  Generic documentation - Benchmarking Manual for Certification and quality labelling of EHRs In the context of serving as a “template” for countries and/or organisations in Europe planning the certification and quality labelling :  Outlines the process, organisational, management & legal issues to be addressed in order to introduce certification and quality Labelling of EHRs

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th What is EHR Certification? Critical component in ensuring the quality of EHR systems is the introduction of a Certification and Quality Labelling process Certification of EHRs can be defined as the procedure and action by which a body duly authorised and recognised as a legitimate provider of this service evaluates and certifies an Electronic Health Record system as meeting predetermined quality standards and certification criteria  Certification addresses functional quality of EHR systems considered as end products.  Certification or quality labelling processes can only evaluate the functionality of a software application.  The technical design (the way the function was implemented) cannot be subject of a certification or labelling exercise. The latter one remains the full responsibility and the exclusive authority of the software vendor.

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Market Assessment - Certification of EHRs Questionnaire administered through national ProRec centres to 4 categories of Respondent Developers/Suppliers Policy Makers Purchasers System End-Users Respondent Developers/Suppliers 33 Policy Makers 31 Purchasers 28 System End-Users 33 Total 125 Country Belgium 9 Bulgaria 62 Denmark 2 Germany 6 Ireland 22 Romania 13 Slovenia 11 Total 125

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th % - 97% of all 4 respondent categories agree that the certification process is important to the development of EHRs Developers/ Policy Makers Purchasers System Suppliers End-users 88%-91% 94%-100% 54%-64% 73%-82% believe that EHR certification will result in one or more of these benefits:  improve prospects of competing favourably in EHR market  improve quality, safety & cost effectiveness of healthcare  improve business process, including security, interoperability and data reliability  allow the provision of a better service to patients  Above suggests need for special focus on purchasers Market Assessment - Certification of EHRs

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Market Assessment - Certification of EHRs Developers/Suppliers  88% do not currently have certified EHR systems  66% plan to submit EHR system for certification  66% say their organisation prepared to pay for certification from a nationally/internationally recognised certification body Purchasers  61% interested in buying a certified EHR system knowing the financial commitment this involves but  only 36% say their organisation is prepared to pay extra for a certified system

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Market Assessment - Certification of EHRs All 4 respondent categories give a High rating to certification and other QREC products that will:  Define what an EHR product is expected to provide and how it will meet specifications (Range 81% - 93%)  Significantly reduce investment and associated risks of implementation of  EHR systems by specifying interoperability requirements (Range 70%-89%) High ratings in the main also given to certification and other QREC products that will:  Provide an independent, unbiased, professional and trustworthy quality assurance mechanism for EHR products  Support industry co-operation by convergence towards known and new national standards and criteria, improving interoperability of EHR systems  Improve the quality and safety of healthcare:  by assessing the quality and suitability of EHR product  by providing an inventory of resource  by educating vendors/users on requirements of successful EHRs

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Market Assessment - Certification of EHRs A significant majority of all 4 respondent categories say they/their organisation would be interested in:  Using the certification process service - (82%-90%)  Utilising a register of certified EHR systems - (90%-93%)  Utilising a register of criteria that EHRs need to fulfil in order to gain certification – (82%-93%)  Receiving advice on benchmarking processes for formal testing of EHR  Products for certification and quality labelling – (86%-91%)  Having access to libraries of terminology underpinning certification and quality labelling – (81%-97%) 100% of Purchasers say their organisation would appreciate if:  suppliers/sellers of their EHR system had access to current up-to-date information regarding terminology underpinning certification  if seller had received advice on the testing of that system on the way to achieving certification  A number of key issues now to be finalised for certification process

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th How Certification will be Managed EuroRec will act as a central repository of validated quality criteria and other relevant materials that can be used to harmonise European testing, certification & quality labeling and procurement specification for EHR systems Rather than impose particular certification models or specific criteria on any member country EuroRec will foster, via ProRec centres and other channels, the progressive adoption of consistent and comparable approaches to EHR system certification and quality labelling The management of the certification process in Europe will operate through the EuroRec Institute and the National Certification Authorities and/or the National ProRec Centres The certification process may be carried out by the National Certification Authority itself, or a mandated subcontractor, - National ProRec centre or other agency

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th How Certification will be Managed The role of EuroRec - initially through the Q-Rec project  Build and develop the Certification process centrally  Distribute the Certification process to National Certification Authorities/ProRec Centres  Assist and support National Certification Authorities/ ProRec Centres in introduction and adoption  Providing guidance and education to all stakeholders in the market  Auditors of the process  Establish and Maintain Registers  Maintaining the standards and certification criteria The role of the National Certification Authorities/ ProRec Centres  Taking the Certification process and adapting for local needs. Be the centre of certification for each country  Establish and Maintain Registers (based on EuroRec registers)  Maintaining the standards and certification criteria

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th The Generic Certification Process Generic certification process stages are:  Initial contact by supplier/developer/purchaser  Formal application  Application process Contract between Certification Authority and Applicant Certification Manual describing: Certification process Statement of criteria for Certification Documentation to be supplied by vendor Testing Schedule  Test/Retest – 2 levels Pre-test self-assessment External evaluation by C.A.  Pass/Fail  Certification Register Details certification status of each product Schedule of dates for retesting/recertification  Issue of Certificate/Seal - issued on successful completion of test  Publication

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Generic Certification Documentation/Manual Generic Certification Documentation/Manual includes specific forms and statement of requirements in regard to:  Application form  Contract  Instruction /information /guidelines for the Certification process  Application Test Pack  Vendor Documentation

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Application Form Application form, details:  Supplier’s name, address, contact details  Product name, version, status (live. pre-live, development - if product is live, details of at least one site where it is in use – location and contact)  Requested test date/alternative date  Signature of director/company representative with appropriate authority

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Contract Contract is completed between Certification Authority/Agency and Applicant Contract to cover the certification process, specifying Certification Authority/Agency and Vendors rights, accountability, responsibilities, limitations, confidentiality, freedom of information and waivers.

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Certification Manual Instruction /information /guidelines for the Certification process Details:  Overview of certification process  How to apply  Documentation required  Criteria to attain certification  Overview of range of functional, operational, implementation, standards and user criteria used for tests  Overview of tests and testing methodologies  Approval process  Retest and appeal process  Use of Seals and Certification  Fees

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Application/Test Pack Application /Test Pack contains:  Detailed description of certification process  Detailed description of test process/methodology  Documentation to be completed  Supporting documentation required  Detailed set of criteria to be utilised for testing  Statement of Criteria Requirements for Certification regarding  Functionality  Interoperability  Security  Data Structures, Codes and Standards  Test Scripts/Scenarios to test each of the criteria

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Vendor Documentation Documentation to be supplied by vendor:  Software/Hardware Maintenance. Agreement(s)  Employment contracts and agreements  Procedures document  System documentation (including operating systems and database)  Hardware/communications specifications  User Manual  Conformance statement  Training manual and arrangements  Software/Product Licence  Escrow agreement  Licences from relevant Third party suppliers.

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Implementing the Certification Process A number of issues and have been identified. Options are proposed for consideration and decision by each national certification authority. These are categorised into:  Legal & Management Issues/Options  Organisation & Process Issues/Options

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Legal & Management Issues/Options in the Certification Process The major issues/options to be decided are: 1.Balance between Pan European vs. National based certification 2.Status of Certification/Accreditation Authority (C.A.) - Government vs. Non-Government 3.Mandatory vs. Voluntary Approach 4.Supplier Participation in Evaluation

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Pan European vs. National based certification Process issues/considerations comprise primarily the following: Considerations here include the following:  Pan European based approach likely to:  Be more efficient  Assure consistency  Prove more acceptable to suppliers  Raise National difficulties  National based approach likely to:  Be better supported at Country level  Diminish consistency/efficiency  Increase cost both to C.A. and suppliers  Be opposed by suppliers  Joint Pan European/ National  European certification - generic criteria/functionality  National Certification – specific national requirements

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Market Assessment Should certification be organised/provided on: A – a Pan-European basis B - a national basis C – a combination of A and B Suppliers/Developers Policy Makers Purchasers End-Users A 33% 13% 25% 42% B 12% 6% 11% 9% C 54% 74% 64% 46%

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Certification Authority (C.A.) - Government vs. Non-Government Considerations here include the following:  Government Body likely to:  Be less acceptable to suppliers  Be more independent  Non-Government Body likely to:  Be more acceptable to suppliers  Hold more monopolistic status  Be dependent on suppliers

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Market Assessment Should certification be organised/provided by: A – a Government organisation B – a Non-Government organisation Suppliers/Developers Policy Makers Purchasers End-Users A 9% 52% 39% 33% B 91% 45% 61% 58%

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Mandatory vs. Voluntary Approach Important to determine the approach Considerations include the following:  Mandatory approach likely to:  Give greater assurance to purchasers/end-users  Be welcomed by higher performing suppliers  Drive product improvements in lesser performing suppliers  Create potential difficulty for new entrants  Voluntary approach likely to:  Be generally more welcomed by suppliers  Effectively produce the same participation outcome as mandatory approach  Create more amenable relational arrangement with suppliers

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Market Assessment Should certification be A Mandatory or B Optional? Suppliers/Developers Policy Makers Purchasers End-Users A 15% 52% 85% 85% B 85% 45% 11% 12%

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Supplier Participation in Evaluation Considerations here include the following:  Supplier participation likely to:  Enhance supplier confidence  Assure peer status  Enhance supplier acceptance  Raise questions on objectivity  Potentially undermine business confidentiality  Advantage larger/more established suppliers  Supplier non-participation likely to:  Assure greater objectivity  Enhance procurer acceptance  Improve confidence of and acceptance by funding authorities  To raise arguments on feasibility of the requirements, correctness of the tests and professionalism of the process

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Should developer/supplier participate in certification process? Suppliers/Developers Policy Makers Purchasers End-User Agree 100% 93% 86% 73% Market Assessment

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Organisation & Process Issues/Options in the Certification Process The organisation and process issues/options to be decided are: 1.Audit vs. Self-Assessment / Surveyor Review Evaluation Approach 2 Audit/Surveyor Workforce 3.Quality Assurance vs. Quality Improvement Focus 4.Scoring / Rating Scheme 5.Certification Award 6.Evaluation Event For Certification 7.Scheme Review and Evaluation

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Audit vs. Self-Assessment / Surveyor Review Evaluation Approach Considerations here include the following:  Audit Approach likely to:  Be more rigid and extreme  Lack nuance  Potentially unfairly advantage larger/more established suppliers  Be more objective  Self-Assessment /Surveyor Review Approach likely to:  Be more rounded in outcome  Avoid extremes  Be better able to reflect potential of suppliers  Be generally more acceptable to suppliers  Potentially be more open to surveyor persuasion  Potentially be less objective

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Audit/Surveyor Workforce Important considerations here include:  Competence / personal profile considerations  Job specification  Part-time (temporary) vs. full time (permanent) appointees  Peer Status  Credibility  Calibre  Selection  Training  Knowledge / skills maintenance

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Quality Assurance vs. Quality Improvement Focus Considerations here include the following:  QA approach likely to:  Be more rigid  Distance C.A. from procurers and suppliers  Minimise value return of scheme in the context of its potential to develop procurer / supplier synergies  QI approach likely to:  Add significant value for procurers/suppliers  Encourage and benefit smaller / start up suppliers  Optimise value return of scheme in the context of its potential to develop procurer / supplier synergies  Enhance engagement between C.A. and procurers/ suppliers  Be more resource demanding  Further EuroRec’s mission objectives

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Scoring / Rating Scheme Considerations here include the following:  Pure numeric scoring / rating vs. combined numerate / judgement scoring / rating  Standards / criteria weightings  Mandatory standard / criteria  Minimum score / rate requirements Decision on scoring/rating scheme will impact on final criteria definition

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Certification Award: Pass / Fail vs. Graded Considerations here include the following:  Pass / fail approach likely to:  Benefit established / large scale suppliers  Militate against small scale suppliers  Potentially inhibit market entry  Improve certainty for procurers  Graded approach likely to:  Benefit small scale suppliers  Encourage / assist new entrant suppliers  Create less certainty for procurers

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Evaluation Event For Certification Considerations here include:  Necessity to develop a clear and unambiguous template setting out the process to be followed for the evaluation event  Imparting details to / training suppliers with respect to the process  Articulation of the process and mechanism by which the evaluation outcome is converted into a certification award  Need to provide for a formal and objective appeals mechanism with respect to a contested evaluation / certification outcome

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Scheme Review and Evaluation There is a need to incorporate provision in the scheme process for:  Scheme review (3/5 year cycle)  Continuous scheme evaluation (i.e. to establish extent to which scheme is meeting its stated objectives / expectations)

EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5th Next Steps Utilise the pilot implementations in Europe of a quality labelling and certification process for EHR systems in compliance with the “good practice requirements” elaborated by EuroRec Validate  Model - generic guidelines and procedures  Generic documentation - Benchmarking Manual for Certification and quality labelling of EHRs Determine and agree and – on a Country/European level  Options are that are proposed for consideration and decision by each national certification authority Implement on a Country and European level  Quality labelling and certification process for EHR systems