1 Work Plan for Testing the LIS and EHR Systems Define Test Flow based from Work Flow Define a testing methodology Develop high-level requirements for LIS Test Harness and LIS Test Tool Identify conformance requirements external to machine-processable XML message profile (on-going) –To include conditionals and variations in profiles (e.g., OIDs/no OIDs) –Include challenging cases (e.g., parent/child) Determine an initial set of use cases for testing (on-going) –Select from in-scope tests –Vocabulary subgroup identified 137 tests that accounts for 99% of lab results –Select core set of 5-10 (Start with one to establish template) Develop test cases –Test Data (Elincs/vendor test plans?; vendor data; NIST data for ELR IG (2007) and PH IG (2010) –Create Test Messages –Variation to core messages Create coverage analysis mapping test case coverage to conformance requirements –Continue until all conformance requirements handled Organize into a test plans (start with the EHR test plan)

2 Proposed Approach for Testing the EHR System LIS Test Harness EHR RI (System under Test) ORU R01 Message 1.The EHR system is required to consume lab results messages that conform to the referenced standards and “incorporate” the data into their workflow. 2.The LAB system is a test tool and in this layout is a test harness. 3.The test harness will maintain a collection of pre-defined test cases that will include a range of conformant and non-conformant messages covering various lab results and scenarios. The tester will also have the option to edit the test messages. 4.The tool sends LRI (sender profile) messages to the EHR (SUT). 5.The EHR is expected to consume the message (as specified by the LRI receiver profile specification) and “incorporate” into their system 6.The EHR can be tested using a combination of inspection testing and automated testing. The tests are designed to verify to a reasonable degree of certainty that the EHR correctly “incorporated” the lab results. For the most part this will be black box testing. 7.The Lab test tool will generate a juror document for inspection testing and a validation context for automated testing (of the acknowledgement message). LRI Sender LRI Receiver ACK R01 Message Inspection testing automated testing

3 EHR Validation Test Workflow I Develop Specific Use Case Scenarios Create Test Data Select from in-scope tests (small core set) Typical and high priority For each create relevant lab results data Document (e.g., in Excel) Export to word document (resembles lab data sheet) Add messaging information and map to HL7 (e.g., Excel) Create/Generate Message Create Message Profiles (use MWB) Generate Messages (Message Profile + Test Data) Verify Message Verify that the message is correct according to the test case (e.g., contains correct LONIC code; missing element) This is more than validation of the message Create Test Validation Artifacts Generate a juror document for inspection testing Generate validation context file for automated testing Send Test Message to EHR SUT Configure message (e.g., Sending Application ID) Configure test system for routing information Send message to EHR SUT using MLLP * * The core set of messages can be modified to target specific test points

4 EHR Validation Test Workflow II Receive/Validation Acknowledge Message Perform Inspection Testing Receive acknowledgement message Validate using message profile and validation context file Generate Report Tester will instruct vendor to demonstrate incorporate of lab results into EHR system (compare w/ juror document) Tester will use tool to manually add test outcome Combine Validation Reports Combine automated and inspection testing results Display/Save/Print reports End

5 Questions/Concerns (for testing the EHR) What assumptions can we make about the EHR system? –What is the EHR required to display? Is it sufficient to adequately assess the system? What does CLIA buy us? How do we make the link between what we sent and how they have incorporated it into their system? E.g., the EHR will probably not display a LONIC code; it may map the code to its internal code (representation) and display only the internal description as a text string—what is reasonable to accept? What are other things to look out for? –Could leverage other MU requirements (e.g., create CDA or send Lab results message to public health) What are the elements that need to be inspected/verified? –i.e., what needs to be in the juror document –How can we ensure that every element specified in the IG is handled correctly in the EHR? –Is it practical to assess each required element without automated testing? If so how? What can be validated automatically? How do we test the incorporation of vocabulary? –Inspection testing –View configuration files?

6 Next Steps Develop Test Plans –LIS –EHR (initial focus) Test Plans to include –Work Flow –Test Flow and methodology –Test Cases Test Data (creation of a subgroup) –Conformance requirements and coverage analysis appendix Dimensions (OIDs, no OIDs, minimally populated, maximally populated, invalid, etc) Develop high-level design architecture/requirements –LIS Test Tool –LIS Test Harness Note: tool development discussion will not be part of this subgroup –We’ll have periodic tool demonstration and review