3 Test Overview Test Area Who Tests What Against What Where SW Module TestThe SW-DeveloperTests a SW Package (or single Modules)Against the SW DesignSimulation on PC, TargetSW Integration TestThe SW-Test EngineerTests a SW Component (possibly within a SW System)Against the SW ArchitecturePartial or whole SystemSW Verification TestTests SW Components (within a SW System)Against the SW RequirementsUsually whole System
4 Why SI? Why do we need System Integration ? The purpose of System Integration isto assemble a system from its defined componentsto ensure that the interfaces between the components of the integrated system function properlyFocus is on the System Level, i.e.releases of sub projects are handled as one part (system modules / discipline components)interfaces between these parts are the object of testsRemark:Even if a project has no official System Integration Team, the activities are performed anyway(by highest / last integration in the project)
5 Integration testing(interface testing) Examines the interaction of software elements (components) after system integrationIntegration is the activity of combining individual software components into a larger subsystemsFurther integration of subsystems is also part of the system integration processEach component has already been tested for its internal functionality (component test). Integration tests examine the external functionsScope:Integration tests examine the interaction of software components (subsystems) with each other:interfaces with other componentsinterfaces among GUIs/ MMIsIntegration tests examine the interfaces with the system environmentTests cases may be derived from interface specifications, architectural design or data models
6 Verification vs. Validation Verification: Proof of compliance with the stated requirements (def. after ISO 9000)" Did we proceed correctly when building the system?"Validation: Proof of fitness for expected use" Did we build the right system software system ?"
7 Verification within the general V-Model Each development level is verified against the contents of the level above it-to verify: to give proof of evidence-to verify: means to check whether the requirements and definitions of the previous levelwere implemented correctlyrequirementsdefinitionacceptance testfunctional systemdesignsystem testtechnical systemdesignintegration testDevelopmentand IntegrationcomponentspecificationcomponenttestprogrammingVerification
8 System test(Verification) Testing the integrated software system to prove compliance with the specified requirements-software quality is looked at form the user's point of viewSystem tests refer tofunctional requirements: suitability, accuracy, interoperability, compliance, security"what the system does"nonfunctional requirements: reliability, usability,efficiency, portability, maintability"how the system works"
9 TerminologieRELIABILITY: the ability of the software product to perform required functions under stated conditions for aspecified period of time for a specified number of operationsUSABILITY: testing to determine the extent of which the software product is understood, easy to learn, easy to operateEFFICIENCY: the process of testing to determine the efficiency of software productMAINTABILITY: the process of testing to determine the maintability of software productPORTABILITY: the process of testing to determine the easy to transfer the software product from one hardware to another (or software environment to another)SUITABILITY: the capability of a software product to provide an appropriate set of functions for specified tasks and user objectivesACCURACY: the capability of a software product to provide the right or agreed results or effects with the needed degree of precisionCOMPLIANCE: ability of software product to adhere to standards, conventionsINTEROPERABILITY: ability of software product to interact with one or more specified components or systemSECURITY: attributes of software product that bear on its ability to prevent unauthorized access to programs and data
10 System test Test cases may be derived from: functional specifications use casesbusiness processesrisk assessmentsScope:Test of the integrated system from the user's point of viewThe test environment should match the true environment:All external interfaces are tested under true conditionsNo tests in the real life environment!
11 Validation within the general V-Model Validation refers to the correctness of the each development levelto validate: to give proof of having valueto validate: means to check the appropriateness of the results of one development levelrequirementsdefinitionacceptance testfunctional systemdesignsystem testtechnical systemdesignintegration testDevelopmentand IntegrationcomponentspecificationcomponenttestprogrammingVerificationValidation
12 Testing process @ Continental Discipline(Entertainment, Navi etc.)System IntegrationPVV(Product Verification and Validation)Requirements /ArchitectureFailedTestResultPassedPRODUCTION
14 Test design techniques Deriving test cases from requirementsDesigning test cases must be a controlled processtest object and requirementson the test objectdefining testrequirementsand test criteriatest case 1test case 1Test cases can be created in a formal way or in a informal way depending on the project delimitations and on the maturity of the process in use.
15 Test design techniques TraceabilityTests should be traceable: which test case was included in the test portofolio, based on which requirement?test scenariostest object and requirementson the test objecttest casetest casetest case4defining testrequirementsand test criteriatest case 1test case 1test casetest casetest case 1Traceability helps to determine requirement coverage
16 Test design techiques Definitions Test object Terminology the subject to be examined (a document or a piece of software in the software development process)Test conditionan item or an event: a function ,a transaction ,a quality criterion or an element in the systemTest criteriathe test object has to confirm the test criteria in order to pass the testTerminology
17 Test design techniques Test case description according to IEEE 829:Input values: description of the input data on the test objectPreconditions: situation previous to test execution or characteristics of the test object before conducting the test caseExpected results: output data that the test object is expected to producePostconditions: characteristics of the test object after test execution, description of its situation after testDependencies: order of execution of test cases, reason for dependenciesDistinct identification: Id or key in order to link ,for example ,an error report to the test cases where it appearedRequirements: characteristics of the test object that the test case will examine
18 Test levelPVV (Product Verification and Validation) - Test Level DefinitionTo optimize test execution, the following test levels were defined:XXS (extreme small): Screening-Test -> handover test between PVV and SI/SWXS (extra small): Quick-Test -> to get a fast overview about all main functions/featuresS (small): Pre-release-Test -> check all functions and find major errors (feedback for SW dev.)M (medium): Release-Test (part 1) -> to find all serious errorsL (large): Release-Test (part 2) -> to find all errorsRemarks: M- and L-level are forming 100% of all test cases, therefore all other test levels are sub groups ofthe complete set (e.g. S will be a sub group of M).The complete release test can be divided in two parts (M- and L-level). This gives the opportunity, to deliverthe release to the customer with a finalized M-test first.Note: It is not allowed to change the SW version through hole release test run!
19 Types of tests Module tests (MP3 Player, GPS, HMI) - smoke tests Integration tests (e.g. MP3 player vs HMI)Verification testsValidation tests
21 Improvements for applications-user point of view?
22 Test suitesA Test suite is a collection of test cases which are grouped based on one criteria or moreA system can have an unlimited number of test suites based on different criteria:- A test suite for HMI tests- A test suite for Performance tests- A test suite for Navigation tests- A test suite for Entertainment tests- etc
31 Evaluation Create System requirement document (1p) Create Test plan(1p)Improvements for HMI,GPS,MP3 player and speech applications (1p)Create 2 verification tests (1p)Create 2 validation tests for each application (1p)Free tests (EBTs –Experience Based Testing) on HMI, GPS, MP3 player ,Speech applications (1p)Find Bugs