1. 1 IHE Gazelle ObjectsChecker : Abderrazek Boufahja IHE-Europe, Gazelle team abderrazek.boufahja@ihe-europe.net HL7 WGM Paris 2015 Eric Poiseau IHE-Europe, Gazelle team Eric.poiseau@inria.fr

2. Context IHE / IHE-Europe IHE is an initiative by healthcare professionals and industry to improve interoperability between healthcare IS IHE Europe : A non-for-profit association attached to IHE International Develop Test tools and organize European Connectathon Gazelle an open source test-bed platform that provides a wide set of tools to validate information exchange between healthcare system => for more details visit the website : http://gazelle.ihe.nethttp://gazelle.ihe.net Connectathon A meeting between healthcare systems developers in order to test the interoperability between their systems/devices => last one : Luxembourg IHE CDA content profiles A list of profiles restricting the CDA standards with specific requirements 2

3. Our problem ! Need to test XML documents for conformity with specification CDA documents : IHE, epSOS, national projects… But also other documents (XDW, HL7v3, XD*,…) Need for a methodolody where inheritence can be an advantage. Model based methodology Need better specifications for better testing We’d like to test the implementation of the specs, not the specs !

4. Conformance validation steps

5. Gazelle ObjectsChecker Principles (1) A methodology to describe informal requirements in healthcare IT specifications based on CDA standard, into a formal description An architecture that allows : The validation of the conformance of any kind of XML requirements Provide metrics and documentary features Improve the coupling between rules and requirements Support the validation of inheritance between specifications 5 Source : eHealth Suisse, Format d’échange, Rapports de laboratoire soumis à déclaration en Suisse (Projet)

6. Gazelle ObjectsChecker Principles (2) 6 Automatic and thorough Testing Requirements Entry Scripts that extract information from UML models and generate structured texts related to these models Description of the structure of the XML doc. Injection of rules and requirements into the model → OCL constraints

7. Gazelle ObjectsChecker Implementation Multiple CDA validators (each testing one “content profile” specification) were developed and used by IHE and multiple national projects around the world : 14 IHE CDA documents validators 11 epSOS CDA documents validators 15 CDA document validators for vaious national projects 30 000 CDA documents validated against Gazelle ObjectsChecker validators Heavily used during European and North American Connectathon and epSOS Projectathon Easily integrated into third party repositories as a front end validation tool 7

8. Standalone Document/Message Validation EVSClient 8 Gazelle Test Platform EVS Front End HL7v2/v3 Validator Dicom Validator CDA Validator Schematon Validator XDW Validator 1 - HTTP : Upload of document 2 – WS Call 3 - Report SUT EVSClient : a front end to the conformance validation services used by the Gazelle Platform (http://gazelle.ihe.net/EVSClient)http://gazelle.ihe.net/EVSClient

9. EVSClient demonstration for CDA validation

10. EVSClient installations Kanta (Finland) epSOS (EU Cross-border) eSanté (Luxembourg) Venice Region (Italy)

11. Need of strong fondations

12. HL7 CDA R2 requirements Extraction of all CDA requirements not expressed in the XSD schema from: HL7 Clinical Document Architecture, Release 2.0 HL7 Reference Information Model, Release 2.07 Data Types - Abstract Specification, Release 1, 2004 XML Implementation Technology Specification - Data Types, Release 1, 2004 The output : a document named « HL7 CDA R2 Basic Requirements» (reference 12) Number of identified requirements : 160 12 A unique reference that uniquely identifies the CDA basic requirements

13. Application 1: National Projects Samples Studies 153 samples from different European and north American national projects 2200 errors found using Gazelle ObjectsChecker : an average of 14 errors per document !!!

14. Application 2: IHE Schematrons Validation Studies 1700 “valid” IHE CDA samples based on schematrons validation 18000 errors detected by gazelle ObjectsChecker 11 errors per document 60 different kind of errors found Conclusion ? 1- Many CDA basic requirements are not checked by the schematrons 2- The weakness of schematrons could spread to healthcare systems

15. Most Frequent Errors in CDA Documents found by Gazelle ObjectsChecker 15  DTITS-007: the use of reference element under an ED data type  CDADT008 /CDADT-006: the use of attributes related to CD data type (nullFlavor, code, displayName, etc)  CDADT-011: the use of UUID structure  CDADT-013/CDADT-014: the specification of URL references  … 2 kind of errors : Errors that may create troubleshooting for document consumer systems Errors that Corrupt of Healthcare Information

16. Tooling: CDA Basic Requirements Coverage Analysis Requirement Coverage Indicator Specification Describes the % of requirements tested by a validation tool, regarding the total number of requirements 16

17. Conformance validation steps art-decor trifolia MDHT Gazelle ObjectsChecker Eclipse Instance Editor Nist Validation tool schematrons from regional projects etc

18. Tooling: CDA Basic Requirements Coverage Analysis 18

19. CDA conformance quantification How to identify the conformance of a CDA document regarding the CDA standard and validated against a validation tool ? (Requirements Coverage Indicator, Number of errors found) ( I cov, N error ) Tools of validation SHALL always provide a report of their coverage indicator

20. Gazelle ObjectsChecker Advantages Requirement coverage capability support of complex requirements (complex algorithm, etc) conditional /iterations validation XML elements type verification Data types requirements checking Runtime access to coded value sets from a repository of terminologies Easier to maintain than hand written schematrons Validation is faster than schematrons Linking between the rules tested and the requirements from the specifications 20

21. Gazelle ObjectsChecker Principles (2) 21 Automatic and thorough Testing Requirements Entry Scripts that extract information from UML models and generate structured texts related to these models Description of the structure of the XML doc. Injection of rules and requirements into the model → OCL constraints

22. Art-Decor Art-decor and Gazelle ObjectsChecker coupling 22 ObjectsChecker Requirements Entry

23. Getting specifications right: The ART-DECOR Framework

24. ART-DECOR in Europe Shared repositories with collection of artifacts Template Repository (building bocks for clinical document definitions) and Value Sets (code lists) Functional Models (datasets) Sharing artifacts between countries has already started by Germany, Austria, Netherlands ART-DECOR specs  Input for Testing Tool Testing “Scripts”

25. ART-DECOR & IHE Europe Memorandum of Understanding signed: Synergistic tooling: ART-DECOR Framework and IHE Gazelle ObjectsChecker  facilitate the creation and consistent standardized documentation of CDA based specifications and  support rigorous compliance validation and testing. The overall goal is: to provide these projects with easy-to-use efficient combined tooling that enhance the quality of their implementations and information exchange.

26. Advantages of coupling ObjectsChecker with art-decor Art-decor moves rigor at point of Content Profiles/Impl. Guides documentation and avoid discovery of issues/gaps at the time ObjectsChecker input is created. Reduces gaps and misunderstanding of CDA specifications Automate the generation of formal OCL description avoiding test tool manual entry 26

27. Conclusion Insufficiency of XSD schema validation for CDA documents Numerous CDA validation tools are far from covering 100% of basic CDA requirements The national projects in Europe need to improve the validation tools they use to increase the quality of their CDA documents Validation tools should provide requirements coverage reporting Gazelle ObjectsChecker was efficient to detect errors in national projects samples Perspective Possibility to extends Gazelle ObjectsChecker methodology to other XML based standards like HL7V3, FHIR, etc Possibility of coupling Gazelle ObjectsChecker with some editors of CDA requirements 27

28. More details.. More details : paper “Model-based Analysis of HL7 CDA R2 Conformance and Requirements Coverage” (IHIC 2015) Web pages : http://gazelle.ihe.nethttp://gazelle.ihe.net Contacts IHE Europe: eric.poiseau@ihe-europe.net abderrazek.boufahja@ihe-europe.net Art-Décor hl7@kheitmann.de28

29. Any question ? 29

30. How does IHE Gazelle ObjectsChecker compares to Schematron Schematrons are useful but have inherent limitations in term of coverage. Requirements generally not covered : CDA R2 base standards requirements : Generic data types requirements Complex requirements between CDA elements XML elements type checking Typical Content Profile/implementation Guides requirements Complex attributes specifications (like the person identifiers structure/algorithm, telecom structures, etc) Complex Conditional requirements between sections or entries Specific data types requirements Relationships consistency with other document content and metadata Elements based on the paper to be presented at IHIC conference: “Model-based Analysis of HL7 CDA R2 Conformance and Requirements Coverage” Coverage of CDA R2 requirements improvements : typically from 50- 60% to 100%. For details see above paper. 30