1. EuroRec Certification Guidelines and Procedures
EuroRec Certification Guidelines and Procedures

2. Generic Version Model Certification Guidelines & Procedures
Introduction
What is Certification?
Market Assessment
The Management of the Certification Process in
Europe
Generic Certification Process
Generic Certification Documentation/Manual
Issues/Options in introducing Certification
Legal & Management
Organisation & Process
Next steps

3. Introduction Generic Version, Model Certification Guidelines
& Procedures
Describes “what” to certify and “how” to perform the
certification
Documents preliminary model - generic guidelines and
procedures
Generic documentation - Benchmarking Manual for
Certification and quality labelling of EHRs
In the context of serving as a “template” for countries and/or organisations in Europe planning the certification and
quality labelling :
Outlines the process, organisational, management & legal issues
to be addressed in order to introduce certification and quality
Labelling of EHRs

4. What is EHR Certification?
Critical component in ensuring the quality of EHR systems is the introduction of a Certification and Quality Labelling process
Certification of EHRs can be defined as the procedure and action by which a body duly authorised and recognised as a legitimate provider of this service evaluates and certifies an Electronic Health Record system as meeting predetermined quality standards and certification criteria
Certification addresses functional quality of EHR systems considered as end products.
Certification or quality labelling processes can only evaluate the functionality of a software application.
The technical design (the way the function was implemented) cannot be subject of a certification or labelling exercise. The latter one remains the full responsibility and the exclusive authority of the software vendor.

5. Market Assessment - Certification of EHRs
Questionnaire administered through national ProRec
centres to 4 categories of Respondent
Developers/Suppliers
Policy Makers
Purchasers
System End-Users
Country
Belgium
Bulgaria
Denmark
Germany
Ireland
Romania
Slovenia
Total  
Respondent
Developers/Suppliers
Policy Makers
Purchasers
System End-Users
Total

6. Market Assessment - Certification of EHRs
91% - 97% of all 4 respondent categories agree that the certification process is important to the development of EHRs
Developers/ Policy Makers Purchasers System
Suppliers End-users
88%-91% %-100% %-64% %-82%
believe that EHR certification will result in one or more of these
benefits:
improve prospects of competing favourably in EHR market
improve quality, safety & cost effectiveness of healthcare
improve business process, including security, interoperability and data reliability
allow the provision of a better service to patients
Above suggests need for special focus on purchasers

7. Market Assessment - Certification of EHRs
Developers/Suppliers
88% do not currently have certified EHR systems
66% plan to submit EHR system for certification
66% say their organisation prepared to pay for certification from a nationally/internationally recognised certification body
Purchasers
61% interested in buying a certified EHR system knowing the financial
commitment this involves
but
only 36% say their organisation is prepared to pay extra for a certified system

8. Market Assessment - Certification of EHRs
All 4 respondent categories give a High rating to certification and other QREC products that will:
Define what an EHR product is expected to provide and how it will meet specifications (Range 81% - 93%)
Significantly reduce investment and associated risks of implementation of
EHR systems by specifying interoperability requirements (Range 70%-89%)
High ratings in the main also given to certification and other QREC products that will:
Provide an independent, unbiased, professional and trustworthy quality assurance mechanism for EHR products
Support industry co-operation by convergence towards known and new national standards and criteria, improving interoperability of EHR systems
Improve the quality and safety of healthcare:
by assessing the quality and suitability of EHR product
by providing an inventory of resource
by educating vendors/users on requirements of successful EHRs

9. Market Assessment - Certification of EHRs
A significant majority of all 4 respondent categories say they/their organisation would be interested in:
Using the certification process service - (82%-90%)
Utilising a register of certified EHR systems - (90%-93%)
Utilising a register of criteria that EHRs need to fulfil in order to gain certification – (82%-93%)
Receiving advice on benchmarking processes for formal testing of EHR
Products for certification and quality labelling – (86%-91%)
Having access to libraries of terminology underpinning certification and quality labelling – (81%-97%)
100% of Purchasers say their organisation would appreciate if:
suppliers/sellers of their EHR system had access to current up-to-date information regarding terminology underpinning certification
if seller had received advice on the testing of that system on the way to achieving
certification
A number of key issues now to be finalised for certification process

10. How Certification will be Managed
EuroRec will act as a central repository of validated quality criteria and other relevant materials that can be used to harmonise European testing, certification & quality labeling and procurement specification for EHR systems
Rather than impose particular certification models or specific criteria on any member country EuroRec will foster, via ProRec centres and other channels, the progressive adoption of consistent and comparable approaches to EHR system certification and quality labelling
The management of the certification process in Europe will operate through the EuroRec Institute and the National Certification Authorities and/or the National ProRec Centres
The certification process may be carried out by the National Certification Authority itself, or a mandated subcontractor, - National ProRec centre or other agency

11. How Certification will be Managed
The role of EuroRec - initially through the Q-Rec project
Build and develop the Certification process centrally
Distribute the Certification process to National Certification Authorities/ProRec Centres
Assist and support National Certification Authorities/ ProRec Centres in introduction and adoption
Providing guidance and education to all stakeholders in the market
Auditors of the process
Establish and Maintain Registers
Maintaining the standards and certification criteria
The role of the National Certification Authorities/ ProRec Centres
Taking the Certification process and adapting for local needs. Be the centre of certification for each country
Establish and Maintain Registers (based on EuroRec registers)

12. The Generic Certification Process
Generic certification process stages are:
Initial contact by supplier/developer/purchaser
Formal application
Application process
Contract between Certification Authority and Applicant
Certification Manual describing:
Certification process
Statement of criteria for Certification
Documentation to be supplied by vendor
Testing Schedule
Test/Retest – 2 levels
Pre-test self-assessment
External evaluation by C.A.
Pass/Fail
Certification Register
Details certification status of each product
Schedule of dates for retesting/recertification
Issue of Certificate/Seal - issued on successful completion of test
Publication

13. Generic Certification Documentation/Manual
Generic Certification Documentation/Manual includes specific forms and statement of requirements in regard to:
Application form
Contract
Instruction /information /guidelines for the Certification process
Application Test Pack
Vendor Documentation

14. Application Form Application form, details:
Supplier’s name, address, contact details
Product name, version, status (live. pre-live, development - if product is live, details of at least one site where it is in use – location and contact)
Requested test date/alternative date
Signature of director/company representative with appropriate authority

15. Contract
Contract is completed between Certification Authority/Agency and Applicant
Contract to cover the certification process, specifying Certification Authority/Agency and Vendors rights, accountability, responsibilities, limitations, confidentiality, freedom of information and waivers.

16. Certification Manual
Instruction /information /guidelines for the Certification process
Details:
Overview of certification process
How to apply
Documentation required
Criteria to attain certification
Overview of range of functional, operational, implementation, standards and user criteria used for tests
Overview of tests and testing methodologies
Approval process
Retest and appeal process
Use of Seals and Certification
Fees

17. Application/Test Pack
Application /Test Pack contains:
Detailed description of certification process
Detailed description of test process/methodology
Documentation to be completed
Supporting documentation required
Detailed set of criteria to be utilised for testing
Statement of Criteria Requirements for Certification regarding
Functionality
Interoperability
Security
Data Structures, Codes and Standards
Test Scripts/Scenarios to test each of the criteria

18. Vendor Documentation Documentation to be supplied by vendor:
Software/Hardware Maintenance. Agreement(s)
Employment contracts and agreements
Procedures document
System documentation (including operating systems and database)
Hardware/communications specifications
User Manual
Conformance statement
Training manual and arrangements
Software/Product Licence
Escrow agreement
Licences from relevant Third party suppliers.

19. Implementing the Certification Process
A number of issues and have been identified. Options are proposed for
consideration and decision by each national certification authority.
These are categorised into:
Legal & Management Issues/Options
 Organisation & Process Issues/Options

20. Legal & Management Issues/Options in the Certification Process
 The major issues/options to be decided are:
Balance between Pan European vs. National based certification
Status of Certification/Accreditation Authority (C.A.) - Government vs. Non-Government
Mandatory vs. Voluntary Approach
4. Supplier Participation in Evaluation

21. 1. Pan European vs. National based certification
Process issues/considerations comprise primarily the following:
Considerations here include the following:
Pan European based approach likely to:
Be more efficient
Assure consistency
Prove more acceptable to suppliers
Raise National difficulties
National based approach likely to:
Be better supported at Country level
Diminish consistency/efficiency
Increase cost both to C.A. and suppliers
Be opposed by suppliers
Joint Pan European/ National
European certification - generic criteria/functionality
National Certification – specific national requirements

22. Market Assessment Should certification be organised/provided on:
A – a Pan-European basis B - a national basis C – a combination of A and B
Suppliers/Developers Policy Makers Purchasers End-Users
A % % % %
B % % % %
C % % % %

23. 2. Certification Authority (C.A.) - Government vs. Non-Government
Considerations here include the following:
Government Body likely to:
Be less acceptable to suppliers
Be more independent
Non-Government Body likely to:
Be more acceptable to suppliers
Hold more monopolistic status
Be dependent on suppliers

24. Market Assessment Should certification be organised/provided by:
A – a Government organisation B – a Non-Government organisation
Suppliers/Developers Policy Makers Purchasers End-Users
A % % % %
B % % % %

25. 3. Mandatory vs. Voluntary Approach
Important to determine the approach
Considerations include the following:
Mandatory approach likely to:
Give greater assurance to purchasers/end-users
Be welcomed by higher performing suppliers
Drive product improvements in lesser performing suppliers
Create potential difficulty for new entrants
Voluntary approach likely to:
Be generally more welcomed by suppliers
Effectively produce the same participation outcome as mandatory approach
Create more amenable relational arrangement with suppliers

26. Market Assessment Should certification be A Mandatory or B Optional?
Suppliers/Developers Policy Makers Purchasers End-Users
A % % % %
B % % % %

27. 4. Supplier Participation in Evaluation
Considerations here include the following:
Supplier participation likely to:
Enhance supplier confidence
Assure peer status
Enhance supplier acceptance
Raise questions on objectivity
Potentially undermine business confidentiality
Advantage larger/more established suppliers
Supplier non-participation likely to:
Assure greater objectivity
Enhance procurer acceptance
Improve confidence of and acceptance by funding authorities
To raise arguments on feasibility of the requirements, correctness of the tests and professionalism of the process

28. Market Assessment
Should developer/supplier participate in certification process?
Suppliers/Developers Policy Makers Purchasers End-User
Agree % % % %

29. Organisation & Process Issues/Options in the Certification Process
The organisation and process issues/options to be decided are:
Audit vs. Self-Assessment / Surveyor Review Evaluation Approach
2   Audit/Surveyor Workforce
Quality Assurance vs. Quality Improvement Focus
Scoring / Rating Scheme
Certification Award
Evaluation Event For Certification
Scheme Review and Evaluation

30. 1. Audit vs. Self-Assessment / Surveyor Review Evaluation Approach
Considerations here include the following:
Audit Approach likely to:
Be more rigid and extreme
Lack nuance
Potentially unfairly advantage larger/more established suppliers
Be more objective
Self-Assessment /Surveyor Review Approach likely to:
Be more rounded in outcome
Avoid extremes
Be better able to reflect potential of suppliers
Be generally more acceptable to suppliers
Potentially be more open to surveyor persuasion
Potentially be less objective

31. 2. Audit/Surveyor Workforce
Important considerations here include:
Competence / personal profile considerations
Job specification
Part-time (temporary) vs. full time (permanent) appointees
Peer Status
Credibility
Calibre
Selection
Training
Knowledge / skills maintenance

32. 3. Quality Assurance vs. Quality Improvement Focus
Considerations here include the following:
QA approach likely to:
Be more rigid
Distance C.A. from procurers and suppliers
Minimise value return of scheme in the context of its potential to develop procurer / supplier synergies
QI approach likely to:
Add significant value for procurers/suppliers
Encourage and benefit smaller / start up suppliers
Optimise value return of scheme in the context of its potential to develop procurer / supplier synergies
Enhance engagement between C.A. and procurers/ suppliers
Be more resource demanding
Further EuroRec’s mission objectives

33. 4. Scoring / Rating Scheme
Considerations here include the following:
Pure numeric scoring / rating vs. combined numerate / judgement scoring / rating
Standards / criteria weightings
Mandatory standard / criteria
Minimum score / rate requirements
Decision on scoring/rating scheme will impact on final criteria definition

34. 5. Certification Award: Pass / Fail vs. Graded
Considerations here include the following:
Pass / fail approach likely to:
Benefit established / large scale suppliers
Militate against small scale suppliers
Potentially inhibit market entry
Improve certainty for procurers
Graded approach likely to:
Benefit small scale suppliers
Encourage / assist new entrant suppliers
Create less certainty for procurers

35. 6. Evaluation Event For Certification
Considerations here include:
Necessity to develop a clear and unambiguous template setting out the process to be followed for the evaluation event
Imparting details to / training suppliers with respect to the process
Articulation of the process and mechanism by which the evaluation outcome is converted into a certification award
Need to provide for a formal and objective appeals mechanism with respect to a contested evaluation / certification outcome

36. 7. Scheme Review and Evaluation
There is a need to incorporate provision in the scheme process for:
Scheme review (3/5 year cycle)
Continuous scheme evaluation (i.e. to establish extent to which scheme is meeting its stated objectives / expectations)

37. Next Steps
Utilise the pilot implementations in Europe of a quality labelling and certification process for EHR systems in compliance with the “good practice requirements” elaborated by EuroRec
Validate
Model - generic guidelines and procedures
Generic documentation - Benchmarking Manual for
Certification and quality labelling of EHRs
Determine and agree and – on a Country/European level
Options are that are proposed for consideration and decision by each national certification authority
Implement on a Country and European level
Quality labelling and certification process for EHR systems