1. Ossi Taipale, Lappeenranta University of Technology
ISO/IEC Parts 1 and 4 ISO/IEC Part 1, Concepts and definitions
Ossi Taipale,
Lappeenranta University of Technology

2. Part 1: Concepts and definitions
This standard defines a risk-based approach to testing. Risk-based testing is a recommended approach to strategizing and managing testing that allows testing to be prioritized and focused.
The ISO/IEC/IEEE software testing series of standards contain three standards where conformance may be claimed:
test processes; ISO/IEC/IEEE
test documentation; ISO/IEC/IEEE
test techniques; ISO/IEC/IEEE

3. Software testing is necessary because:
Information on the quality characteristics of the test item(s) is required by decision makers;
The test item(s) being tested does not always do what it is expected to do;
The test item(s) being tested needs to be verified (Verification ensures that software correctly implements a specific function and it answers the question: are we building the product right?). Basic verification methods include inspections, walkthroughs, and technical reviews;
The test item(s) being tested needs to be validated (Validation ensures that software that has been built is traceable to customer requirements. It answers the question: are we building the right product?). Validation activities can be divided into unit testing, integration testing, usability testing, function testing, system testing, and acceptance testing.

4. Introduction to software testing
Product quality has many aspects, including conformance to specifications, absence of defects, and fitness in meeting the requirements of the product's users. ISO/IEC 25010, System and software quality models defines eight quality characteristics that can be measured or assessed in testing.
Testing is a process. A process is a set of interrelated or interacting activities that transforms inputs into outputs. The objective of this standard is to present and describe generic testing processes (ISO/IEC/IEEE Test processes).
Testing processes and sub-processes can be applied to any phase or level of testing (e.g. system testing) or type of testing (e.g. performance testing).

5. Introduction to software testing
Testing can be carried out on a product without executing the product on a computer. This is called static testing in this standard and in many areas of the industry, although other standards (e.g. IEEE 1028) may more specifically call this reviews, walkthroughs or inspections. For static testing this standard acknowledges and identifies the role of the tester in these activities even though they may be performed by other groups within a project or defined by other non-testing standards. This is because the static testing activities are considered highly important for complete life cycle testing and test involvement has been shown to be critical for early defect detection, reduced overall project costs and an improved ability to meet schedule demands.
Static testing may also include the use of static analysis tools which find defects in the code or documents without the code executing (e.g. a compiler, a cyclomatic complexity analyzer, or a security analyzer for code).

6. Introduction to software testing
Dynamic testing consists of more than "just" running executable test items; it also includes both preparation activities and follow-up activities. The Dynamic Test Processes described in ISO/IEC/IEEE cover each of the activities to be performed in dynamic testing.
This standard addresses testing; it does not address the other activities of validation and verification (e.g. V&V analysis, formal methods).
Other standards, e.g. ISO/IEC and IEEE 1012, address also other verification or validation activities.

7. Annex A: The role of testing in verification and validation

8. Generic test process
The relationship between the generic test process, generic test sub-processes, test levels/phases and test types is described in more detail in Figure 2. The generic test sub-process can be applied in the following ways:
As a test level or phase (e.g. component test phase, acceptance test level);
As a test type (e.g. performance testing, usability testing). The diagram 2 also makes explicit the relationship between test types and quality characteristics (as defined in ISO/IEC 25010) ;
A test sub-process associated with a test level may contain more than one test type sub-process (e.g. functional and performance testing as part of system testing); and
The project test process may be comprised of a sequence of test sub-processes (e.g. component testing, integration testing, system testing and acceptance testing test sub-processes).

9. Figure 2 — The relationship between the generic test sub-process, test levels and test types

10. Annex C: Testing in different life cycle models
The development activities performed are more or less the same in all life cycle models; the main differences lie in the definition of their scopes, the amount and nature of documentation produced, and the frequency with which they are repeated during the course of the development life cycle.
Agile
Evolutionary
Sequential (i.e. the waterfall model)
Scrum is not a development methodology (i.e. it does not tell you the best method for describing requirements or how to write code) and is better described as a project management framework within which an agile methodology is applied by the developers (XP is often used).
A Scrum project comprises a number of iterations called sprints, with each sprint normally resulting in new functionality that can be delivered to the users (Figure).

11. Ossi Taipale, Lappeenranta University of Technology
29119 Part 4, Test techniques
Ossi Taipale,
Lappeenranta University of Technology

12. Part 4, Test techniques: Introduction
The purpose of ISO/IEC Test Techniques is to provide an International Standard that defines software test design techniques (also known as test case design techniques or test methods) that can be used during the test design and implementation process of ISO/IEC Test Processes.
The test design techniques presented in ISO/IEC can be used to derive test cases that, when executed, can be used to collect evidence that test item requirements have been met and/or that defects are present in a test item (i.e. that requirements have not been met). Activities:
 Derive Test Conditions (TD2),
 Derive Test Coverage Items (TD3), and
 Derive Test Cases (TD4).

13. Introduction: Figure 1, ISO/IEC 29119-2 Test Design and Implementation Process

14. Introduction
Experience-based testing practices like exploratory testing and other test practices such as model-based testing are not defined in ISO/IEC because this standard only describes techniques for designing test cases. Test practices such as exploratory testing are described in ISO/IEC

15. Test Design Techniques
ISO/IEC :
In specification-based testing, the test basis (e.g. requirements, specifications, models or user needs) is used as the main source of information to design test cases - "black-box testing“.
In structure-based testing, the structure of the test item (e.g. source code or the structure of a model) is used as the primary source of information to design test cases - "white-box testing" (or "clear-box testing").
Both - "grey-box testing"
In experience-based testing, the knowledge and experience of the tester is used as the primary source of information during test case design.
In practice they can be used interchangeably.

16. Test design techniques presented in ISO/IEC 29119-4
Figure 2, page 7

17. Equivalence Partitioning
Derive Test Conditions (TD2)
Equivalence partitioning (BS :1998; Myers 1979) uses a model of the test item that partitions the inputs and outputs of the test item into equivalence partitions (also called “partitions” or “equivalence classes”), where each equivalence partition shall be defined as a test condition.
These equivalence partitions shall be derived from the test basis, where each partition is chosen such that all values within the equivalence partition can reasonably be expected to be treated similarly (i.e. they may be considered "equivalent") by the test item. Equivalence partitions may be derived for both valid and invalid inputs and outputs.
Derive Test Coverage Items (TD3)
Each equivalence partition shall also be identified as a test coverage item

18. Equivalence Partitioning
Derive Test Cases (TD4)
Test cases shall be derived to exercise the test coverage items (i.e. the equivalence partitions).
Approach for selecting test coverage items:
One-to-one, in which each test case is derived to cover a specific equivalence partition. The number of test cases derived using one-to-one equivalence partitioning equals the number of test coverage items.
Minimized, in which equivalence partitions are covered by test cases such that the minimum number of test cases derived covers all equivalence partitions at least once.

19. Example: Equivalence Partitioning
Test basis:
The component receives as input an exam mark (out of 75) and a coursework (c/w) mark (out of 25), from which it outputs a grade for the course in the range 'A' to 'D'. The grade is calculated from the overall mark, which is the sum of the exam and c/w marks, as follows:
greater than or equal to 70 - 'A'
greater than or equal to 50, but less than 70 - 'B'
greater than or equal to 30, but less than 50 - 'C'
less than 30 - 'D'
Where a mark is outside its expected range then a fault message ('FM') is generated. All inputs are passed as integers.
Page 42

20. Annex A1: Testing Quality Characteristics
This annex contains examples of how quality characteristics defined in ISO/IEC (ISO/IEC 25010:2011) can be mapped to the test design techniques defined in ISO/IEC