1. THE GITB TESTING FRAMEWORK Jacques Durand, Fujitsu America | JDurand@us.fujitsu.com December 1, 2011 GITB | http://www.ebusiness-testbed.euhttp://www.ebusiness-testbed.eu

2. Overview  Objectives of a “Global” e-Business Test Bed  Testing Scenarios  Aspects of a Testing Framework 2

3. Some Objectives  Promote and Facilitate Adoption of eBusiness Standards  Testing is key to standards quality & adoption  Global standards need a Global Test Platform / Practices  A Common & Global Testing infrastructure for eBusiness Standards  Addressing the needs of various stakeholders  Designed and used/reused globally and across industries 3

4. Testing the e-Business Stack Messaging protocol Meta-Data : Interfaces, Agreements Business Process/ Exchange Choreography. Business Documents Messages Combination of standards / profiles  Integrated testing 4

5. Test Bed Objectives  A Test Bed platform agnostic to eBusiness standards  Extensible: Plug-in Components Test suite Engine(s), Message Adapters (AS4, ebMS2, etc.), Document Validators  Portable Test Artifacts across Test Beds  Standardized Test Suites, Configurations, Test Logs …  Test Registry/Repository  archive / publish / download  Versatile Test Bed  Test Bed can simulate a business party (interaction)  Or just analyze captured exchanges (validation)  Both for Conformance and Interoperability Testing 5

6. Overview  Objectives of a “Global” e-Business Test Bed  Testing Scenarios  Aspects of a Testing Framework 6

7. GITB-compliant Test bed “Validating” Test Suite Document Validator Document Validator Test Case Document Assertion Set (DAS) Test Registry / Repository Query existing Test Cases Create / Deploy New Test Suite Test Scenario 1: Document Conformance Creating & Using a “Validating” Conformance Test Suite for Structural & Semantic Validation of Business Documents Human-driven Test Designer Get test Report Business Documents Use Test Suite Upload Documents Test Participant Service Interface Service Interface 7

8. GITB-compliant Test bed Document Validator Document Validator Validating Test Case Create / Deploy Interacting Test Suite Execute Get Conformance Test Report Test Scenario 2: Document + Message Conformance Creating & Using an “Interacting / Validating” Conformance Test Suite [Messaging protocol + QoS + Document] Reusing the “Validating” Suite of Scenario #1 SUT Test Case [SUT] Interacting & Validating Test Suite Validating Test Suite For Document Validation GITB Test Repository Service Interface 8

9. GITB-compliant Test bed Document Validator Document Validator Validating Test Case Create / Deploy Interacting Test Suite Get Conformance Test Report Test Scenario 2 Variant (a): delegating to another Test Bed SUT Test Case Interacting / Validating Test Suite Validating Test Suite For Document Validation GITB Test Repository GITB-compliant Test bed Delegates Document Validation to Service Interface Service Interface 9

10. Test bed Test Execution Log Test Report Test bed drive capture 2-Phase Testing (or pipe-line) Test Scenario 2 Variant (b): separating the SUT interaction from the analysis (validation) Interacting Test Suite Validating Test Suite SUT 10

11. GITB-compliant Test bed Create / Deploy Interacting Test Suite Register Get Status Get Notified File Issues Test Scenario 3: Document + Message + Process Interoperability Creating & Using an Interoperability Test Suite (driving several SUTs) SUT 1 Interacting / validating Test Suite Validating Test Suite SUT 2 Drive Capture Drive Capture Service Interface Register Get Status Get Notified File Issues 11

12. Overview  Objectives of a “Global” e-Business Test Bed  Testing Scenarios  Aspects of a Testing Framework 12

13. Test Registry & Repository Remote Test Bed / Test Agent System Under Test System Under Test GITB Test Bed Enable Implement / process Manage Test Designer Test Manager Test Participant Test Services - Test Design - Test Deployment - Test Execution - Test Repository Test Artifacts - Test Logic (T.Suite…) - Test Configurations - Test Outputs (report..) Test Bed Components - User-facing components - Core Platform components - Test Capability components Control Testing Framework in Action 13

14. Three Major Categories of Test Artifacts  Test Logic Artifacts (define testing logic)  Test Suites/Cases, Document Assertions…  Test Configuration Artifacts (parameters and bindings for Test Logic)  Mesg Adapters, Timing/Tuning, Message samples…  Test Output Artifacts (products of a Test run)  Test Report, Test Logs Depend on eB. Standards Depend on User Environment Depend on SUT behavior 14

15. Test Case Test Suite A logical test unit to verify : - A business document - A business transaction - A message Document Validator Document Assertion Set A set of processable requirements on a document: - schema - rules - dependencies Test Logic Artifacts Test Configuration 15

16. Test Artifact Standardization  Standardizing Test Artifacts: how far?  Not enough  lack of portability and reusability  Too much  too rigid to evolve, may not fit all tests  Level 1: Only standardize the header (metadata)  Allows for storage, tagging and search.  Level 2: L1 plus some integration features:  References or interfaces to other artifacts / capabilities  Container elements (e.g. message wrapper in a log)  Level 3: Whole content standardization  e.g. XML schema. 16

17. Test Services  Standardized Access  Hide Test Bed specifics (e.g. legacy test beds) Service Interfaces TestBed A Test Designer Test Participant Service Interfaces TestBed B Test Repository Service Interfaces 17

18. Service Interfaces Service Interfaces TestBed B TestBed A Test Participant Test Services  Reuse of Testing Capabilities (e.g. Test Beds) Test Suite HL7 Transactions + messaging Test Suite HL7 document validation System Under Test Delegation of document validation 18

19. Test Services  Test Design service: to manage Test Artifacts lifecycle in a Test Bed  Create/Update/Delete local to a testBed.  Test Deployment service: to prepare for execution:  Add Configuration to Test logic (bindings)  Test Execution service: to control all operations of a Test Bed related to Test execution.  Test Repository service: to archive, publish, discover Test Artifacts. 19

20. Full Test Bed Operational Platform Test Bed Core Platform Document Assertions Test Suite AS4 Test report Test report Test report Test Deployment Mgr Test Operation Mgr Test Artifacts Persistence Mgr Test Agent Client Repository Client Test Control Mgr Test Suite Instantiator User-facing Components Test Suite/Case Editor Management Console Test Capability Components (plug-ins) Messaging Adapter Document Validator Document Validator Test Suite Engine Test Suite Engine Test Deployment Service Test Design Service Test Execution Service Test RegRep Service Registry/RepositoryTest bed Test Bed Architecture 20

21. GITB Compliance Levels  How does a Test Bed comply with GITB recommendations?  GITB-Framework Compliance  Test Bed designed in compliance with GITB recommendations  Test Services interfaces, Test Artifacts, allow for plug-in Components  GITB-Service Compliance  Adding [GITB] Test Service interfaces to “Legacy” Test Beds  Produce [GITB] Test Reports 21