System Integration Verification and Validation

Remember V-Cycle for all Increments? SW Requirements SW Architecture SW Design SW Coding SW Module Test SW Integration SW Verification SW Activities System Requirements System Architecture & Design System Integration System Verification SE/PVV Activities EE Activities ME Activities Code Review V1 V3 V4 Planning V2 Verification & Validation Realization Disc.

Test Overview Test Area Who Tests What Against What Where SW Module Test The SW-Developer Tests a SW Package (or single Modules) Against the SW Design Simulation on PC, Target SW Integration Test The SW-Test Engineer Tests a SW Component (possibly within a SW System) Against the SW Architecture Partial or whole System SW Verification Test Tests SW Components (within a SW System) Against the SW Requirements Usually whole System

Why SI? Why do we need System Integration ? The purpose of System Integration is to assemble a system from its defined components to ensure that the interfaces between the components of the integrated system function properly Focus is on the System Level, i.e. releases of sub projects are handled as one part (system modules / discipline components) interfaces between these parts are the object of tests Remark: Even if a project has no official System Integration Team, the activities are performed anyway (by highest / last integration in the project)

Integration testing(interface testing) Examines the interaction of software elements (components) after system integration Integration is the activity of combining individual software components into a larger subsystems Further integration of subsystems is also part of the system integration process Each component has already been tested for its internal functionality (component test). Integration tests examine the external functions Scope: Integration tests examine the interaction of software components (subsystems) with each other: interfaces with other components interfaces among GUIs/ MMIs Integration tests examine the interfaces with the system environment Tests cases may be derived from interface specifications, architectural design or data models

Verification vs. Validation Verification: Proof of compliance with the stated requirements (def. after ISO 9000) " Did we proceed correctly when building the system?" Validation: Proof of fitness for expected use " Did we build the right system software system ?"

Verification within the general V-Model Each development level is verified against the contents of the level above it -to verify: to give proof of evidence -to verify: means to check whether the requirements and definitions of the previous level were implemented correctly requirements definition acceptance test functional system design system test technical system design integration test Development and Integration component specification component test programming Verification

System test(Verification) Testing the integrated software system to prove compliance with the specified requirements -software quality is looked at form the user's point of view System tests refer to functional requirements: suitability, accuracy, interoperability, compliance, security "what the system does" nonfunctional requirements: reliability, usability, efficiency, portability, maintability "how the system works"

Terminologie RELIABILITY: the ability of the software product to perform required functions under stated conditions for a specified period of time for a specified number of operations USABILITY: testing to determine the extent of which the software product is understood, easy to learn, easy to operate EFFICIENCY: the process of testing to determine the efficiency of software product MAINTABILITY: the process of testing to determine the maintability of software product PORTABILITY: the process of testing to determine the easy to transfer the software product from one hardware to another (or software environment to another) SUITABILITY: the capability of a software product to provide an appropriate set of functions for specified tasks and user objectives ACCURACY: the capability of a software product to provide the right or agreed results or effects with the needed degree of precision COMPLIANCE: ability of software product to adhere to standards, conventions INTEROPERABILITY: ability of software product to interact with one or more specified components or system SECURITY: attributes of software product that bear on its ability to prevent unauthorized access to programs and data

System test Test cases may be derived from: functional specifications use cases business processes risk assessments Scope: Test of the integrated system from the user's point of view The test environment should match the true environment: All external interfaces are tested under true conditions No tests in the real life environment!

Validation within the general V-Model Validation refers to the correctness of the each development level to validate: to give proof of having value to validate: means to check the appropriateness of the results of one development level requirements definition acceptance test functional system design system test technical system design integration test Development and Integration component specification component test programming Verification Validation

Testing process @ Continental Discipline (Entertainment, Navi etc.) System Integration PVV (Product Verification and Validation) Requirements / Architecture Failed Test Result Passed PRODUCTION

Verification vs.Validation

Test design techniques Deriving test cases from requirements Designing test cases must be a controlled process test object and requirements on the test object defining test requirements and test criteria test case 1 test case 1 Test cases can be created in a formal way or in a informal way depending on the project delimitations and on the maturity of the process in use.

Test design techniques Traceability Tests should be traceable: which test case was included in the test portofolio, based on which requirement? test scenarios test object and requirements on the test object test case test case test case4 defining test requirements and test criteria test case 1 test case 1 test case test case test case 1 Traceability helps to determine requirement coverage

Test design techiques Definitions Test object Terminology the subject to be examined (a document or a piece of software in the software development process) Test condition an item or an event: a function ,a transaction ,a quality criterion or an element in the system Test criteria the test object has to confirm the test criteria in order to pass the test Terminology

Test design techniques Test case description according to IEEE 829: Input values: description of the input data on the test object Preconditions: situation previous to test execution or characteristics of the test object before conducting the test case Expected results: output data that the test object is expected to produce Postconditions: characteristics of the test object after test execution, description of its situation after test Dependencies: order of execution of test cases, reason for dependencies Distinct identification: Id or key in order to link ,for example ,an error report to the test cases where it appeared Requirements: characteristics of the test object that the test case will examine

Test level PVV (Product Verification and Validation) - Test Level Definition To optimize test execution, the following test levels were defined: XXS (extreme small): Screening-Test -> handover test between PVV and SI/SW XS (extra small): Quick-Test -> to get a fast overview about all main functions/features S (small): Pre-release-Test -> check all functions and find major errors (feedback for SW dev.) M (medium): Release-Test (part 1) -> to find all serious errors L (large): Release-Test (part 2) -> to find all errors Remarks: M- and L-level are forming 100% of all test cases, therefore all other test levels are sub groups of the complete set (e.g. S will be a sub group of M). The complete release test can be divided in two parts (M- and L-level). This gives the opportunity, to deliver the release to the customer with a finalized M-test first. Note: It is not allowed to change the SW version through hole release test run!

Types of tests Module tests (MP3 Player, GPS, HMI) - smoke tests Integration tests (e.g. MP3 player vs HMI) Verification tests Validation tests

System requirements document????

Improvements for applications-user point of view?

Test suites A Test suite is a collection of test cases which are grouped based on one criteria or more A system can have an unlimited number of test suites based on different criteria: - A test suite for HMI tests - A test suite for Performance tests - A test suite for Navigation tests - A test suite for Entertainment tests - etc

Test plan-Test suite-Test package

Verification test -Example

Verification test-Examples

Examples

Examples

Examples

Examples

Validation test-Example

Evaluation Create System requirement document (1p) Create Test plan(1p) Improvements for HMI,GPS,MP3 player and speech applications (1p) Create 2 verification tests (1p) Create 2 validation tests for each application (1p) Free tests (EBTs –Experience Based Testing) on HMI, GPS, MP3 player ,Speech applications (1p) Find Bugs