1. IHE Pre-Connectathon PIX and PDQ Test Tool Tutorial and Guide Robert Snelick National Institute of Standards and Technology November 10 th 2009 Contact: rsnelick@nist.gov

2. 2 NIST PIX/PDQ Pre-connectathon Test Tool: Overview I Supports Pre-Connectathon testing for: Profile/HL7 VersionActors IHE PIX HL7 v2Source, Manager, Consumer IHE PDQ HL7 v2 (new tests)Supplier, Consumer IHE PIX HL7 v3Source, Manager, Consumer IHE PDQ HL7 v3Supplier, Consumer Part of the next generation of IHE tools (project Gazelle) developed through the NIST Healthcare Testing Infrastructure initiative supported by ARRA For the 2010 pre-connectathon testing cycle the tool is –an alternative for the MESA PIX v2 tests developed at the MIR –a replacement for the MESA PIX/PDQ v3 tests developed at the University of Quebec Test cases are the same as the previous MESA tests except for the new PDQ v2 tests As we continued development we are rewriting/clarifying the test case descriptions and test assertions PIX/PDQ only; XDS and PCD testing is supported by other “NISTers”

3. 3 NIST PIX/PDQ Pre-connectathon Test Tool: Overview II Options for meeting the 2010 V2 PIX/PDQ Pre-connectathon testing requirements: –Use the existing PIX/PDQ MESA tests –Use the NIST PIX/PDQ Pre-connectathon test tool –The NIST Tool is the preferred method for pre-connectathon testing (Steve and/or Lynn will provide a “Cookie”) Options for meeting the 2010 V3 PIX/PDQ Pre-connectathon testing requirements: –Use the NIST PIX/PDQ Pre-connectathon test tool –December 1 st is relaxed –NIST took on this role recently Moving forward (2011  ) –It is planned that this tool will replace the existing PIX/PDQ V2 mesa tests –Serve as a platform to support other IHE domain pre-connectathon testing –Serve as a platform to support ARRA meaningful use testing Tool Benefits –Automated Testing –Centralized Testing –Low barrier to entry for tool use –Explicit test descriptions and test assertions –Raises the bar of conformance testing (will propagate to connectathon testing) –Has the capability to raise bar further

4. 4 Acknowledgements NIST design and development team: Harold Affo, Roch Bertucat, Len Gebase, Sydney Henrard, Mary Laamanen, Andrew McCaffrey, Caroline Rosin, and Robert Snelick. Eric Poiseau (INRIA/IRISA) provided the initial versions of IHE PIX/PDQ HL7 V2 conformance profiles. The original MESA test cases for PIX HL7 V2 were developed by Steve Moore at the Mallinckrodt Institute of Radiology. The original MESA test cases for PIX and PDQ HL7 V3 were developed by Rita Noumeir and Jean-Francois Pambrun at the University of Quebec. Abby George of Booz-Allen-Hamilton made significant contributions to this tutorial and the user’s guide. Beta Testers: George Cole, Laurie Flaherty (Allscripts), Umberto Cappellini, Susanne Hartenstein (TIANI SPIRIT), Fred Harmon, Anthony LaRocca (Sage), Vassil Peytchev, Justin Stauffer (Epic) Wenzhi Li (Open Health Tools-OHT, MISYS) for making available an on-line PIX/PDQ Server that we could use for testing

5. 5 NIST PIX/PDQ Test Tool: Access Tool Access—Open for business on November 12 th, 2009 –http://141.156.15.209:8080/NIST-PreConnectathonTestshttp://141.156.15.209:8080/NIST-PreConnectathonTests –Web-based application so downloads and installations are not necessary –User’s Guide is available at this site Tool may be used in Anonymous Mode or Registered Mode –Anonymous: Does not require user registration and may be used to conduct ad-hoc system testing –Registered Mode: Required to save pre-connectathon test results Test reports are made available to the IHE project manager Avenue for meeting pre-connectathon requirements to participate in the connectathon IHE PIX/PDQ Google Group –Group to discuss testing issues and report tool bugs –http://groups.google.com/group/ihe_pix_pdq_testinghttp://groups.google.com/group/ihe_pix_pdq_testing –Go to site and request membership to the group –Preferred method of communication for discussing common issues –Vendor specific issues or bugs can be sent directly to rsnelick@nist.govrsnelick@nist.gov

6. 6 Testing Environments Instance Testing –Conformance (e.g., against HL7 2.x or CDA) Test object conforms to specification on which it is based IHE Model: ~Virtual and Pre-Connectathon NIST IHE PCD V2 Message Validation Test Tool Isolated System Testing –Includes Instance Testing Activities –Protocol Conformance –Functional Behavior Conformance Features and operational behavior correspond to specifications IHE Model: ~Virtual and Pre-Connectathon NIST IHE PIX/PDQ V2/V3 Test Tool Peer-to-Peer System Testing –Includes Isolated System Testing Activities –Interoperability Testing Testing complete application environment May include interacting w/ Database, using Network Communications, or interacting w/ other hardware, apps, or systems if appropriate IHE Model: ~Connectathon

7. 7 NIST PIX/PDQ Test Tool: Operational Process END-USER (VENDOR) SYSTEM UNDER TESTNIST PIX/PDQ HL7 v2/v3 TEST TOOL SPECIFICATIONS (test material that defines test assertions) INTERACTION/REPORTS MESSAGES (TEST OBJECTS) STIMULUS OR RESPONSE (MESSAGES) MANUAL OR AUTOMATED SUT Web Application Interface (via the communication protocol)

8. 8 NIST PIX/PDQ Test Tool: PIX Test Coverage ActorTest Cases SourceTest Case 10512: PIX Patient Feed A04 Test Case 10515: PIX Patient Merge A04 ConsumerTest Case 10501: PIX Query Case 1 and 2 Test Case 10502: PIX Query Case 3 Test Case 10503: PIX Query Case 4 ManagerTest Case 10501: PIX Query Case 1 and 2 Test Case 10502: PIX Query Case 3 Test Case 10503: PIX Query Case 4 Test Case 10506: PIX Query, Patient Update Test Case 10511: PIX Patient Feed A01 Test Case 10512: PIX Patient Feed A04 Test Case 10515: PIX Patient Merge A04 ActorTest Cases SourceTest Case 10512v3: PIX Patient Feed Test Case 10515v3: PIX Patient Identity Merge ConsumerTest Case 10501v3: PIX Query Case 1 and 2 Test Case 10502v3: PIX Query Case 3 Test Case 10503v3: PIX Query Case 4 ManagerTest Case 10501v3: PIX Query Case 1 and 2 Test Case 10502v3: PIX Query Case 3 Test Case 10503v3: PIX Query Case 4 Test Case 10506v3: PIX Query, Patient Update Test Case 10512v3: PIX Patient Feed Test Case 10515v3: PIX Patient Identity Merge PIX HL7 v2 PIX HL7 v3

9. 9 NIST PIX/PDQ Test Tool: PDQ Test Coverage ActorTest Cases Consumer * Test Case 11311: Exact Name Search Test Case 11312: Exact Name Search – No match Test Case 11315: Partial Name Search Test Case 11320: Complete ID Search – Unspecified Domain Test Case 11325: Complete ID Search – Single Domain Test Case 11330: Complete ID Search – Multiple Domains Test Case 11335: Partial ID Search – Single Domain Test Case 11340: Date of Birth Search Test Case 11350: Multiple Key Search 1 Test Case 11365: Continuation Test 1 Supplier * Test Case 11311: Exact Name Search Test Case 11312 : Exact Name Search – No match Test Case 11315 : Partial Name Search Test Case 11320 : Complete ID Search – Unspecified Domain Test Case 11325 : Complete ID Search – Single Domain Test Case 11330 : Complete ID Search – Multiple Domains Test Case 11335 : Partial ID Search – Single Domain Test Case 11340 : Date of Birth Search Test Case 11350 : Multiple Key Search 1 Test Case 11365 : Continuation Test 1 PDQ HL7 v2 PDQ HL7 v3 ActorTest Cases ConsumerTest Case 11311v3: Exact Name Search Test Case 11312v3: Exact name search - No Match Test Case 11315v3: Partial Name Search Test Case 11320v3: Complete ID Search - Unspecified Domain Test Case 11325v3: Complete ID Search - Single Domain Test Case 11330v3: Complete ID Search - Multiple Domains Test Case 11335v3: Partial ID Search - Single Domain Test Case 11340v3: Date of Birth Search Test Case 11350v3: Multi Key Search 1 Test Case 11365v3: Continuation Test 1Test Case 11365v3: Continuation Test 1 (Optional) SupplierTest Case 11311v3: Exact Name Search Test Case 11312v3: Exact name search - No Match Test Case 11315V3: Partial Name Search Test Case 11320v3: Complete ID Search - Unspecified Domain Test Case 11325v3: Complete ID Search - Single Domain Test Case 11330v3: Complete ID Search - Multiple Domains Test Case 11335v3: Partial ID Search - Single Domain Test Case 11340v3: Date of Birth Search Test Case 11350v3: Multi Key Search 1 Test Case 11365v3: Continuation Test 1Test Case 11365v3: Continuation Test 1 (Optional) * Additional test cases created for the 2010 Pre-Connectathon Cycle

10. Test Case Walk-Through PIX Manager Test Case 10501: PIX Query Case 1 and 2 – HL7 Version 2

11. 11 Test Case 10501 – PIX Query Case 1 and 2 – HL7 Version 2 IHE ITI Technical Framework Volume 2a (ITI TF-2a) 3.9 PIX Query Case 1: The Patient Identifier Cross-reference Manager Actor recognizes the specified Patient Identification Domain and Patient ID sent by the Patient Identifier Cross-reference Consumer in QPD-3, and corresponding identifiers exist for the specified patient in at least one of the domains requested in QPD-4 (one identifier per domain). (See Case 6 below for the required behavior if there are multiple identifiers recognized within a given Identifier Domain by the Patient Identifier Cross- reference Manager Actor.) AA (application accept) is returned in MSA-1. OK (data found, no errors) is returned in QAK-2. A single PID segment is returned in which one repetition of PID- 3 Patient Identifier List is populated for each of the domains, if any, that the Patient Identifier Cross-reference Manager Actor did recognize in which a single identifier exists for the requested patient, not including the queried-for patient identifier that is returned in QPD-3. Case 1

12. 12 Test Case 10501 – PIX Query Case 1 and 2 Case 2: The Patient Identifier Cross-reference Manager Actor recognizes the Patient Identification Domain and Patient ID sent in QPD-3, but no identifier exists for that patient in any of the domains sent in QPD-4. AA (application accept) is returned in MSA-1. NF (no data found, no errors) is returned in QAK-2. No PID segment is returned. IHE ITI Technical Framework Volume 2a (ITI TF-2a) 3.9 PIX Query Case 2

13. 13 Test Case 10501 – PIX Query Case 1 and 2 Description  Test case 10501 covers the PIX Query Case 1 and 2. One patient (ALPHA) is registered in two different domains. A second patient (SIMPSON) is registered in a single domain. Three registration messages are sent to a Cross Reference Manager.  A PIX Query is sent to resolve a reference to ALPHA.  A second PIX Query is sent to resolve a reference to SIMPSON. Because SIMPSON is not registered in the second domain, the response to that PIX Query will indicate no data. References  IHE ITI TF-2a: Transactions Part A (Revision 6.0, August 10, 2009)  3.9.4.2.2.6 Patient Identifier Cross-reference Manager Actor Query Response Behavior Actors  Patient Identity Source (Test Agent)  Patient Identity Consumer (Test Agent)  Patient Identity Cross-Reference Manager (Vendor System Under Test)

14. 14 Test Case 10501 – PIX Query Case 1 and 2 1 The NIST PIX Source sends a registration message (ADT^A04) to register patient ALPHA ALAN in domain HIMSS2005&1.3.6.1.4.1.21367.2005.1.1&ISO. Patient ID is “PIX10501”. Your PIX Manager shall register the patient and send a correct ACK message back. PIX Source Reference Message PIX Manager Reference Example Message 2 The NIST PIX Source sends a registration message (ADT^A04) to register patient ALPHA ALAN in domain XREF2005&1.3.6.1.4.1.21367.2005.1.2&ISO. Patient ID is “XYZ10501”. Your PIX Manager shall register the patient and send a correct ACK message back. PIX Source Reference Message PIX Manager Reference Example Message 3 The NIST PIX Source sends a registration message (ADT^A04) to register patient SIMPSON CARL in domain HIMSS2005&1.3.6.1.4.1.21367.2005.1.1&ISO. Patient ID is “ABC10501”. Your PIX Manager shall register the patient and send a correct ACK message back. PIX Source Reference Message PIX Manager Reference Example Message 4 The NIST PIX Consumer sends a query message (QBP^Q23) to ask for ALPHA ALAN's ID in domain XREF2005&1.3.6.1.4.1.21367.2005.1.2&ISO using his id (“PIX10501”) in domain HIMSS2005&1.3.6.1.4.1.21367.2005.1.1&ISO. Your PIX Manager shall answer correctly to the query with ALPHA ALAN's (ID = “XYZ10501”) is in domain XREF2005&1.3.6.1.4.1.21367.2005.1.2&ISO. PIX Consumer Reference Message PIX Manager Reference Example Message 5 The NIST PIX Consumer sends a query message (QBP^Q23) to ask for SIMPSON CARL's ID in domain XREF2005&1.3.6.1.4.1.21367.2005.1.2&ISO using his ID in domain HIMSS2005&1.3.6.1.4.1.21367.2005.1.1&ISO. As SIMPSON CARL was not registered in the XREF2005&1.3.6.1.4.1.21367.2005.1.2&ISO domain, your PIX Manager shall send a query response back indicating that no data was found. PIX Consumer Reference Message PIX Manager Reference Example Message TEST STEPSDESCRIPTIONSOURCEDESTINATION

15. 15 Test Case 10501 – PIX Query Case 1 and 2 (v2) NIST PIX Source Test Agent NIST PIX Source Test Agent ADTA04 – Register patient “ALAN ALPHA” (ID = PIX10501 ) in domain “ HIMSS2005 ” ACKA04 – Send acknowledgement 1 1a PIX/PDQ Test Tool Vendor PIX Manager Application Vendor PIX Manager Application End User (Vendor) ADTA04 – Register patient “ CARL SIMPSON ” (ID = ABC10501 ) in domain “ HIMSS2005 ” ACKA04 – Send acknowledgement 2 2a ADTA04 – Register patient “ ALAN ALPHA ” (ID = XYZ10501 ) in domain “ XREF2005 ” ACKA04 – Send acknowledgement 3 3a System Under Test NIST PIX Consumer Test Agent NIST PIX Consumer Test Agent QBPQ23 – Send query message for patient “ ALAN ALPHA ” in domain “ HIMSS2005 ” seeking the patient ID in domain “ XREF2005 ” 4 4a RSPK23– Send response indicating that the identifier (“ XYZ10501 ”) for patient “ ALPHA ” was found in domain “ XREF2005 ” QBPQ23 – Send query message for patient “ CARL SIMPSON ” in domain “ HIMSS2005 ” seeking the patient ID in domain “ XREF2005 ” 5 RSPK23 – Send response message indicating no data was found “ NF ” 5a Web Application Interface

16. Validation HL7 v2 Messages and IHE Actor Functional Behavior

17. 17 HL7 v2 Transactions Validation Process HL7 v2 Syntax Validation: –Ensure the message structure is valid with respect to the HL7 V2 message definition and the HL7 V2 conformance profile –Ensure that the conformance requirements such as usage, cardinality, data type usage, etc. are valid with respect to the standard and the HL7 V2 conformance profile that constrains the message in accordance with the IHE technical framework for a given transaction Value Set Semantic Validation: –Ensure that elements that refer to a table are valid with respect to the value set it references. The value set is given by the HL7 V2 standard or as constrained by the IHE technical framework. An example includes confirming that the administrative sex element (PID.8) of the PID segment contains a value that is given in the HL7 V2 Administrative Sex table (HL70001). –Identify conformance violations of constraints implied by the IHE Integration Profile, these constraints are captured in a validation context file. An example of such a test includes an element that is fixed in the IHE integration profile (e.g., RCP.1 = I). 3. HL7 v2 MLLP Validation: –Ensure the message is valid with respect to the HL7 V2 MLLP specification for wrapping and unwrapping HL7 V2 messages. This is an implied test. 4. Test Case Specific Validation: –Verify element content against a validation context file that captures test values as defined in the pre-connectathon test cases.

18. 18 Test Case 10501 – Test Case Specific Validation Checks Validation Criteria 1 Assert an “AA” acknowledge message for registration of patient “ALPHA” in domain “HIMSS2005”; Patient ID is “PIX10501” 2 Assert an “AA” acknowledgement message for registration of patient “ALPHA” in domain “XREF2005”; Patient ID is “XYZ10501” 3 Assert an “AA” acknowledgement message for the registration of patient “SIMPSON” in domain “HIMSS2005”; Patient ID is “ABC10501” 4 PIX Query Case 1: Assert response message with MSA.1 = “AA” PIX Query Case 1: Assert response message with PID.3.1 = “XYZ10501” PIX Query Case 1: Assert response message with QAK.2 = “OK” more… 5 PIX Query Case 2: Assert response message with MSA.1 = “AA” PIX Query Case 2: Assert response message with QAK.2 = “NF” PIX Query Case 2: Assert that no PID segment is found in response message more… STEPSASSERTIONS

19. 19 Validation Operational Process: Origin of Test Assertions HL7 v2 Standard Message Definition IHE TF Message Transaction Constraints HL7 v2 Standard Value Sets IHE TF Message Transaction Value Set Constraints Test Case Specific Test Assertions** IHE TF Message Transaction Test Assertions Validation Context File (XML) Table Library (XML) Conformance Profile (XML) Validation Engine ** Dynamic Data (Future Work)

20. 20 HL7 V2 Conformance Profile Example (ADT A04 Message) - - 11.6.1 - 11.2.2 - 11.2.3 - 11.2.4 - 11.6.6

21. 21 HL7 V2 Table Library Example

22. 22 HL7 v2 Validation Context Example - - XYZ10501 - - XREF2005

23. 23 HL7 v2 Message Validation Report

24. 24 Testing Validation Types Validation against ‘failure types’: –VERSION*: The version in the message and in the profile should match. –MESSAGE_STRUCTURE_ID*: The message type (MSH.9 element) in the profile and in the message should match. –MESSAGE_STRUCTURE: The message should have a valid message structure (correct usage, correct cardinality, and correct element name). –USAGE: R elements should be present; X elements should not be present in the message. –CARDINALITY: Elements should be present at least the minimum times and at most the maximum times specified in the profile. It should also take into account the usage of the element (X element with a minimum of 4 should not be present in the message). –LENGTH: The value of the element should have a length equal or less than the value specified in the profile. –DATATYPE: For the datatype NM, DT, DTM, SI and TM, the value of the element should match the regular expression defined in the standard. –DATA: The value of the element should match a constant specified in the profile, a value set specified in a table, a value or a regular expression specified in the message validation context. –MESSAGE_VALIDATION_CONTEXT*: This is a user input error when the location specified in the message validation context can't be found in the message. –TABLE_NOT_FOUND*: This is a user input when a table can't be found in the table files (TableProfileDocument). –AMBIGUOUS_PROFILE*: The profile should not be ambiguous.

25. 25 Conceptual View of HL7 v2 PIX Manager Test End-User (Vendor) Vendor PIX Manager Application Vendor PIX Manager Application NIST PIX Source Test Agent NIST PIX Source Test Agent PIX/PDQ Test Tool NIST PIX Consumer Test Agent NIST PIX Consumer Test Agent Web Application Real- Time Log Screen Test Cases Reports NIST Server Communication HL7 v2 HL7 v3 Test Executor Validation MLLP (v2) Other NIST Test Agents Other NIST Test Agents Test Flow Engine

26. 26 Step 1 and 1a Validation Criteria  Check against ADTA04 Profile  Check against Value Set  Check against TC10501-1 Validation Context  Assert an “AA” acknowledge message for registration of patient “ALPHA” in domain “HIMSS2005”; Patient ID is “PIX10501” NIST PIX Source Test Agent NIST PIX Source Test Agent Vendor PIX Manager Application Vendor PIX Manager Application TC10501-1 Generation Context (XML) ADTA04 Profile (XML) Value Set (XML) - Profiles - Test Scenarios - Test Cases - Value Sets - Generation and Validation Context Files Test Repository Patient IDLast NameFirst Name PIX10501ALPHAALAN XYZ10501ALPHAALAN ABC10501SIMPSONCARL TC10501-1 Validation Context (XML) ACK Profile (XML) ADTA04 – Register patient “ALAN ALPHA” (ID = PIX10501 ) in domain “ HIMSS2005 ” ACKA04 – Send acknowledgement 1 1a

27. 27 Step 2 and 2a Validation Criteria  Check against ADTA04 Profile  Check against Value Set  Check against TC10501-1 Validation Context  Assert an “AA” acknowledgement message for the registration of patient “ALPHA” in domain “XREF2005”; Patient ID is “XYZ10501” NIST PIX Source Test Agent NIST PIX Source Test Agent Vendor PIX Manager Application Vendor PIX Manager Application - Profiles - Test Scenarios - Test Cases - Value Sets - Generation and Validation Context Files Patient IDLast NameFirst Name PIX10501ALPHAALAN XYZ10501ALPHAALAN ABC10501SIMPSONCARL TC10501-1 Generation Context (XML) ADTA04 Profile (XML) Value Set (XML) TC10501-1 Validation Context (XML) Test Repository ACK Profile (XML) ACKA04 – Send acknowledgement 2 2a ADTA04 – Register patient “ ALAN ALPHA ” (ID = XYZ10501 ) in domain “ XREF2005 ”

28. 28 Step 3 and 3a Validation Criteria  Check against ADTA04 Profile  Check against Value Set  Check against TC10501-1 Validation Context  Assert an “AA” acknowledgement message for registration of patient “SIMPSON” in domain “HIMSS2005”; Patient ID is “ABC10501” NIST PIX Source Test Agent NIST PIX Source Test Agent Vendor PIX Manager Application Vendor PIX Manager Application - Profiles - Test Scenarios - Test Cases - Value Sets - Generation and Validation Context Files Patient IDLast NameFirst Name PIX10501ALPHAALAN XYZ10501ALPHAALAN ABC10501SIMPSONCARL TC10501-1 Generation Context (XML) ADTA04 Profile (XML) Value Set (XML) TC10501-1 Validation Context (XML) Test Repository ACK Profile (XML) ADTA04 – Register patient “ CARL SIMPSON ” (ID = ABC10501 ) in domain “ HIMSS2005 ” ACKA04 – Send acknowledgement 3 3a

29. 29 Step 4 and 4a Validation Criteria  PIX Query Case 1: Assert response message with MSA.1 = “AA”  PIX Query Case 1: Assert response message with PID.3.1 = “XYZ10501”  PIX Query Case 1: Assert response message with QAK.2 = “OK”  And more… NIST PIX Consumer Test Agent NIST PIX Consumer Test Agent Vendor PIX Manager Application Vendor PIX Manager Application - Profiles - Test Scenarios - Test Cases - Value Sets - Generation and Validation Context Files Patient IDLast NameFirst Name PIX10501ALPHAALAN XYZ10501ALPHAALAN ABC10501SIMPSONCARL TC10501 Generation Context (XML) QBPQ23 Profile (XML) Value Set (XML) TC10501 Validation Context (XML) Test Repository RSPK23 Profile (XML) QBPQ23 – Send query message for patient “ ALAN ALPHA ” in domain “ HIMSS2005 ” seeking the patient ID in domain “ XREF2005 ” 4 4a RSPK23– Send response indicating that the identifier (“ XYZ10501 ”) for patient “ALPHA” was found in domain “XREF2005”

30. 30 Step 5 and 5a Validation Criteria  PIX Query Case 2: Assert response message with MSA.1 = “AA”  PIX Query Case 2: Assert response message with QAK.2 = “NF”  PIX Query Case 2: Assert that no PID segment is found in response message NIST PIX Consumer Test Agent NIST PIX Consumer Test Agent Vendor PIX Manager Application Vendor PIX Manager Application - Profiles - Test Scenarios - Test Cases - Value Sets - Generation and Validation Context Files Patient IDLast NameFirst Name PIX10501ALPHAALAN XYZ10501ALPHAALAN ABC10501SIMPSONCARL TC10501 Generation Context (XML) QBPQ23 Profile (XML) Value Set (XML) TC10501 Validation Context (XML) Test Repository RSPK23 Profile (XML) QBPQ23 – Send query message for patient “ CARL SIMPSON ” in domain “ HIMSS2005 ” seeking the patient ID in domain “ XREF2005 ” 5 RSPK23 – Send response message indicating no data was found “ NF ” 5a

31. Registration and Login

32. 32 NIST PIX/PDQ Test Tool: Steps for Test Execution 1.Register/Log-in 2.Select Test Case Client Type (PIX Source and Consumer, PDQ Consumer) Server Type (PIX Manager, PDQ Supplier) 3.Configure Test 4.Run Test 5.View/Save/Submit Reports

33. 33 Registration/Log-in To register, send the following information via email to rsnelick@nist.gov: rsnelick@nist.gov – Name – Company Name – Email Address – Application Name (MSH 3.1) for V2 Only – Facility Name (MSH 4.1) for V2 Only – The IHE Actors that you plan on testing (e.g. V2 PIX Manager) NIST Administrator will then send log-in credentials Log-in is required to save test results, submit results to the IHE project manager, and gain credit for test completion NIST PIX/PDQ Test Tool IHE Kudu *Push/Pull to Kudu IHE Project Manager can view database of results * not yet implemented

34. Tool Demonstration PIX Manager Test Case 10501: PIX Query Case 1 and 2 – HL7 Version 2

35. 35 Review Overview After registration, users can log-in using the ‘Log-in’ tab to submit test results to the IHE Project Manager To register, send the following information via email to rsnelick@nist.gov:rsnelick@nist.gov Name Company Name Email Address Application Name (MSH 3.1) For V2 Only Facility Name (MSH 4.1) For V2 Only IHE Integration Profiles Planned to Test Go to the PIX/PDQ Test Tool User Guide and Support Files for step- by-step instructions Send defects, comments, and suggestions to rsnelick@nist.gov

36. 36 Select Test Case - Client Select an ‘HL7 Version’ and ‘Actor’ to view a list of available test cases Select a test case by selecting the corresponding radio button to view a description, references, and list of transactions Select the ‘Display Configuration Information’ button to view the test configuration details – the ‘Start Test’ button will become enabled once the configuration information dialog been closed

37. 37 Select Test Case - Server Select an ‘HL7 Version’ and ‘Actor’ to view a list of available test cases Select a test case by selecting the corresponding radio button to view a description, references, and list of transactions Select the ‘Configure’ button to enter the test configuration details – the ‘Start Test’ button will become enabled once the configuration information has been entered

38. 38 Configure – User Client View the ‘Application Name’, ‘Facility Name’, ‘IP Address’, ‘Port Number(s)’ needed to configure your client application to send messages. Please note that each NIST PIX Manager instance is unique The name of the NIST PIX server is temporary and will changed for every test that is run. So you must obtain the configuration information each time and change the receiving application name accordingly. Using an old name will result in failure; you will receive an acknowledge error message over the network (nothing will appear in the console).

39. 39 Configure – User Server Single IP/Port Multi IP/Port Enter the ‘Application Name’ and ‘Facility Name’ about the System Under Test (SUT) – these fields are associated with MSH.3.1 and MSH.4.1 in the HL7 v2 message Select ‘Single IP/Port’ or ‘Multi IP/Port’ and enter the information accordingly – ‘Multi IP/Port’ may be used to handle ADT and PDQ messages Example PIX Manager to Test (OHT Implementation) V2 Application Name: PAT_IDENTITY_X_REF_MGR_MISYS V2 Facility Name: ALLSCRIPTS IP Address: 198.160.211.53 Port: 3600 Use this PIX Manager implementation from Open Health Tools (OHT)—when online—to try the tool if you don’t have an application to test

40. 40 Run Test Test cases are executed one transaction at a time – real-time results can be tracked in the ‘Console Log’ Individual validation reports can be viewed by selecting the ‘Magnifying Glass’ icon at the end of each transaction Select ‘Next Step’ at the conclusion of each transaction to initiate the next Results, Messages Sent/Received, and Detailed Results (XML/HTML) can be viewed in between each transaction

41. 41 View/Save/Submit Report Select the ‘Submit’ button to submit test results to the IHE Project Manager for review – the ‘Finish’ button should be selected to end the current testing session At the completion of the test, users can view individual validation reports by selecting the ‘Magnifying Glass’ icon A summary-level report will also be available – reports may be printed or saved to the user’s local file system

42. 42 View Results – Validation and Summary Reports A summary-level report that compiles individual validations reports will also be available – reports may be printed or saved to the user’s local file system Individual validation reports are available at the end of each transaction and again at the conclusion of a test case

43. 43 View Results – Test Summary Registered users can view their test history on the ‘Reports’ tab – this tab provides access to saved transaction and summary-level reports Users can download and save the ‘Result Summary’ in both HTML and PDF formats Users may select the ‘Actor Tested’ and ‘HL7 Version’ to view their list of previously run test cases. Users may then select the corresponding radio button to view the results for a particular test case

44. Validation HL7 v3 Messages and IHE Actor Functional Behavior

45. 45 HL7 V3 Validation Overview Check for correct Interaction ID (although no check on structure outside of correct root element name) Implied communication message exchange test (i.e., SOAP— no WS-addressing) Check against the validation context for the specific test case –Similar to v2 Not quite as robust and flexible as v2 testing yet This will be improved in the coming weeks/months Will incorporate our v3 External Validation Service (EVS)

46. Configuration and Trouble-Shooting

47. 47 Configuration Information V2 Applications sending ER7 –IP Address: 141.156.15.209 –Port Number: 9080 V2 Applications sending XML –IP Address: 141.156.15.209 –Port Number: 9070 V3 Application –IP Address: 141.156.15.209 –Port Number: 9090 V3 SOAP end-point –http://141.156.15.209:9090

48. 48 Example PIX Source Vendor Configuration Information Configuration Information for sending V2 messages Application Name: NIST_Swan_PIX_Manager_78 (varies) Facility Name: NIST IP Address:141.156.15.209 Port Number:9080 (ER7 Messages) Port Number:9070 (XML Messages) Configuration Information for sending V3 messages Application Name: 2.16.840.1.113883.3.72.6.5.92 (varies) Facility Name: 2.16.840.1.113883.3.72.6.1 (Optional) IP Address: 141.156.15.209 Port Number: 9090 (V3 Messages)

49. 49 Dynamic PIX/PDQ Server Application Name Allocation For each test case instance the name of the PIX Manager or PDQ Supplier is created dynamically In order to interact with our PIX Manager or our PDQ Supplier Test Agent you will need to use the information provided in “Configuration Information” dialog –For example (V2): Application Name: NIST_Swan_PIX_Manager_150 Facility Name: NIST –For example (V3): Application Name: 2.16.840.1.113883.3.72.6.5.17 Facility Name: 2.16.840.1.113883.3.72.6.1 (Optional) –The life of this test agent exists only for the duration of the test –This ensures that each vendor test is isolated If you send to an application name that no longer exists you will get the following response from our system: –Nothing from the user console—we’re still listening for messages –An acknowledge message with error is sent back to you over the network

50. 50 Trouble-Shooting Site is unavailable –Typically means we are redeploying the application –This is a web application so we need to redeploy a.war file when we make changes –The downtime will be in the neighborhood of 15-20 minutes –We will not announce short term downtimes –DSL may be down No response from console log –You may have sent to a destination that we could not process –i.e., An incorrect receiving name –In this case we will send back an acknowledge/error over the network –You need to check on your end –On our console log nothing will happen –If you determined that this was the problem you can resend a properly addressed message; we’ll still be listening up to the session timeout Site appears to be down –Restart in a new browser window Completed Report is missing –When we discover an error in the test case (i.e., the test case is wrong) we fixed it –Tests that passed using an incorrect test are no longer valid –This should not occur often

51. Summary

52. 52 Summary New PIX/PDQ Pre-connectathon testing tool Trial for the 2010 testing cycle The tool is evolving… –So stick in there –We need your feedback Serve as the platform for testing other IHE domains in future Serve as the platform for ARRA meaningful certification testing as part of the NIST Healthcare Testing Infrastructure project Components will be available in the NIST HIT Testing Infrastructure which will be enable integration into other tooling project (e.g., IHE Gazelle) Current to-do list –Improve V3 validation –Improve test case descriptions –Bug tracking system –Development of test plans, cases, and documentation –Make software available

53. 53 Support IHE PIX/PDQ Google Group: http://groups.google.com/group/ihe_pix_pdq_testing We anticipate bugs and the need for clarification of the tests. We will responsive. NEED FEEDBACK: Report bugs, what you like, what you hate, what annoys you, and suggestions for a better user interface and features. Harold Affo Len Gebase Sydney Henrard Mary Laamanen Andrew McCaffrey Caroline Rosin Robert Snelick Send e-mail to Rob Snelick: rsnelick@nist.gov OR

54. 54 Stick with us, things will only be muddy for a while…green grass is ahead