1 Opentest Architecture Table of Content –The Design Basic Components High-Level Test Architecture Test Flow –Services provided by each Layer Test Mgt System Test Master Controller Service-Providers layer –Interfaces between layers Test Mgt System Test Master Controller Test Master Controller Service-Providers layer –User Interaction w/ the System

2 Test Architecture’s Basic Components 3-Layer Architecture: 1) Test Management System. 2) Test Master Controller. 3) Service Providers Test Projects Test Management System Custom Fields* Test Master Controller TMS Writer TestLink (GPL) + Staf Staf (EPL v1.0) Test Reports Test Requirements Test Execution Engine … Equipment Drivers * Custom fields= Test Execution Engine, Test Execution Logic, Required HW assets, Required SW assets and Test Parameters. Staf + Vatf|Stax|iCOTS|LinuxTEE|Other Test Cases Front-End Execution Logic Monitor Resource Manager n 1 1 STAF Request + XML Build Execution Engine … Execution Logic STAF Request n Dispatcher

3 High-Level Test Architecture Test Master Controller Build Execution Engine Test Management System Test Artifacts (Mysql) Service Provider Test Execution Engine Test Execution Engine Service Provider Test Execution Engine Site A Build Execution Engine Test Management System Test Artifacts (Mysql) Service Provider Test Execution Engine Test Execution Engine Service Provider Test Execution Engine Site B IP Network Test Artifacts Databases must be synchronized on a periodic basis The 3 rd layer is the service-providers layer. TEE and BEE are just special Service Providers. It should be possible to reuse Service Providers across sites if/when desired Typically 1 TMS : 1 TMC 1 TMC : Many Service Providers Test Master Controller

4 Sample TMS-Driven Test Flow TMS TMC 2: Triggers Test Execution. Specifies: Test plan, Platform, SW assets versions 15: Generate Test Reports 13: Send (optionally) Front-End 1: Creates Test Plan (optionally) Test Plan (XML) 3: Export Test Plan to XML Dispatcher 4: Send Request to TMC (Pass Test Plan XML file) TMS Writer 6:Send build job request to BEE BEE BEE Iface Service Provider Service Iface Res Mgt TEE TEE Iface 9: Send test job request to TEE 10: Optionally request job to a service provider via Resource Manager Service 11: Notify completion Provide Results 14: Save test results in TMS 7:Notify Build completion 5,8: Request BEE or TEE 12:Send Request to save Results

5 Test Management System’s Services TestLink Basic Definitions: –Test Project holds together the multiple test artifacts (test cases, test requirements, test plans, etc.) –Test Case describes a testing task and expected results. Test Cases are the fundamental piece of TestLink. –Test Suite is a collection of Test Cases. It organizes Test Cases to units. –Test Build is the software to be tested. It is part of the Test Plan. –Test Plan is created when you'd like to execute Test Cases. Test Plans can be made up of the Test Cases from the current Test Project. Main Services provided by the TMS: –Store all Test Artifacts –Provides ways to Add/Edit/Delete Test Artifacts:  Test Cases  Test Suites  Test Requirements  Test Builds  Test Plans (Test Cases + Build Tag) Test Results –Export/Import Test Plans to/from XML –Trigger test execution (optionally)

6 Test Master Controller’s Services Dispatch Jobs to appropriate BEE & TEE Translate Test parameters between TMS and BEE/TEE Provide TMS-independent test case representation to TEE Provide a mechanism by which TEE can save test results in TMS Manage Service Providers

7 Service Providers’ Services At least there will be two types of service providers: –BEE: Build Execution Engines to build the required software assets. –TEE: Test Execution Engines to run the test cases. The TEE maybe as simple as a shell script or as complex as VATF. BEE & TEE are the only service providers that the TMC Dispatcher directly request services from. Complex TEE, such as VATF, will typically perform these tasks: –Initialize HW assets (both Device Under Test and Test Equipment) –Create Test Parameters representation –Run Test Execution Logic Load Device Under Test (DUT) w/ appropriate SW assets Boot DUT Connect DUT to Test Equipment Configure DUT and Test Equipment Check conditions or measure metrics Determine Pass/Fail –Save Results & Logs

8 TMS Test Controller IFACE Test Plan (Mysql) TestLink Test Master Controller Test Plan (XML) Export Import Staf Request Trigger Exec Write Back Test Results Dispatcher Read Test Cases Staf Request Trigger Exec TMS GUI Trigger Exec Front-End XML format provides a TMS-independent Test Data Representation Staf provides a generic communication mechanism, both the TestLink server and the Test Master Controller must have Staf installed

9 Test Controller Service Providers IFACE Test Master Controller BEE/TEE Service Providers Dispatcher Staf Request Process, FS or other services Similar to TMS-TMC iface, Staf provides a generic communication mechanism. The Test Controller Dispatcher is a Staf-Enabled Java application Service Providers, including BEE & TEE, must have Staf on it (StafProc process) The Dispatcher reads BEE/TEE name from Test Plan and dispatch the appropriate job request to the appropriate engine. For example, it reads SW assets info in the Test Plan and interact w/ a BEE to build them. Similarly, it reads the TEE type from the test plan and submit a test request to the appropriate TEE. The TEE will adapt or transform the test plan data according to its needs. The Dispatcher may dispatch a Test job that contains multiple test cases to streamline the test execution. The Dispatcher adapts TEE responses before writing back to Test Plan database Staf Request Notify, FS or other services Resource Mgt Monitor StafProc Service-Provider -specific Actions

10 User Interaction w/ the system Test Management System Test Master Controller Test Execution Engine Creates/Edits/Deletes Test Artifacts Triggers Test Execution Generate Test Reports Monitor Test Execution Send Monitor Test Execution Send