1. Test Design Techniques
Dynamic Testing
Test Design Techniques

2. Dynamic Testing
Testing that involves the execution of the software of the component or system
Execution of the test object on a computer
Need a test bed

3. Test Bed Test Driver Test output Test Object Test stubs
Case 1
Test
Case 2
Test
Case n
Stub1
PoC
Test
output
Test Object
PoO
Stub 2
comparison
Stub k
Run time environment,
Analysis tools, monitors
Test stubs
PoC – Point of Control
PoO – Point of Observation

4. Incremental approach to execute tests
Steps are:
Determine conditions and preconditions for the test and the goals that are to be achieved
Specify individual test cases
Determine how to execute the tests (usually chaining together several test cases)
Eg: group test cases in such a way that a whole sequence of test cases is executed (test sequence or test scenario)
Document it in test procedure specification
Need test script, e.g., JUnit

5. Techniques for Testing
They are test case design techniques
Different techniques are:
Black Box Testing
White Box or Glass Box or Open Box Testing

6. Black Box Testing Techniques(1)
Test object is treated as a black box
The inner structure and design of the test object is unknown
Test cases are derived/design using the specification or the requirements of the test object
The behavior of the test object is watched from the outside (PoO is outside the test object)
It is not possible to control the operating sequence of the object other than choosing the adequate input test data (PoC is situated outside of test object)
Used for higher levels of testing
Any test design before the code is written (test-first programming, test-driven development) is black box driven

7. Black Box Testing Techniques(2)
Poc and PoO “outside” the test object
Test input data
PoC
Test
Output
data
Test Object
PoO

8. Black Box Testing Techniques: Equivalence Class Partitioning Method(1)
The domain of possible input data for each input data element is divided in equivalence classes
An equivalence class is a group of data values where the tester assumes that the test object processes them the same way
the behavior of the component or system is assumed to be the same
Test of one representative of the equivalence class is seen as sufficient
test equivalence class for correct input as well as incorrect input

9. Black Box Testing Techniques: Equivalence Class Partitioning Method(2)
Example of correct (valid) equivalence class vEC
Parameter
Equivalence Class
Representative values
Bonus calculation program, duration of employment in years
vEC1 : 0 <= x <= 3
vEC2 : 3 < x <= 5
vEC3 : 5 < x <= 8
vEC4 : x > 8 2 4 7 12

10. Black Box Testing Techniques: Equivalence Class Partitioning Method(3)
Example of incorrect (invalid) equivalence class iEC
Parameter
Equivalence Class
Representative values
Bonus calculation program, duration of employment in years
iEC1 : x < 0
(“negative” – thus incorrect – staff membership in a company)
iEC2 : x > 40
(unrealistically long and incorrect staff membership in a company -3 45

11. Black Box Testing Techniques: Equivalence Class Partitioning Method(4)
Systematic Derivation of the Test Cases
Determine the domain of all possible inputs for every input data element
E.g., function/method parameter at component tests or input screen field at system test
Partition the domain into equivalence classes
First the subdomain of correct inputs
The test object should process these according to specification
Then the values outside this domain as equivalence classes for invalid values
Observe how the test object behaves
Refine the equivalence classes
The equivalence classes should be divided until each different requirement corresponds to an equivalence class
Choose a representative value for every single equivalence class for testing
Define the preconditions and the expected result for every test case

12. Black Box Testing Techniques: Equivalence Class Partitioning Method(5)
the same principle of dividing into equivalence classes can also be used for the output data
Identification of the individual test case is more expensive, because for every output-representative the corresponding input value combination causing this output must be determined
Regard also the output values for the equivalence classes with incorrect values

13. Black Box Testing Techniques: Equivalence Class Partitioning Method(6)
Boundaries of the equivalence classes
Partitioning into equivalence classes and selecting the representative values should be carefully done
Probability of failure detection is highly dependent upon the quality of the partitioning, as well as which test cases are executed
The best values are those verifying the boundaries of the equivalence classes
Often misunderstanding or inaccuracies in the requirements at these spots
E.g., “… more than 32 credits…” might mean
The value 32 being inside equivalence class (EC : x >= 32) or
Outside equivalence class (EC : x > 32)
Additional test case with x=32 might detect a misunderstanding

14. Black Box Testing Techniques: Equivalence Class Partitioning Method(7)
Equivalence class construction for integer values(1)
All possible classes are to be identified
Example : based on integer parameter extras of the function calculate_price()
the domain of inputs partitioned into equivalence classes
Incorrect values:
Numbers that are greater or
Smaller than the range of the
Applicable interval or every
Nonnumeric value including
Floating point numbers
Parameter
Equivalence classes
extras
vEC1 : [MIN_INT,…,MAX_INT]
iEC1 : Not a Number

15. Black Box Testing Techniques: Equivalence Class Partitioning Method(8)
Equivalence class construction for integer values(2)
Further divide the equivalence classes with correct values because negative and positive values often must be treated differently
Zero is a further input value that often lead to failures
Refinement of Equivalence classes and representatives
Result: the equivalence class method, taking into account boundary values, generate the following test values:
(“f”, MIN_INT, -123, -1, 0, 654 MAX_INT)
Parameter
Equivalence classes
Representatives
extras
vEC1 : [MIN_INT,…,-1]
vEC2 : [0,…,MAX_INT]
iEC1 : Not a Number
-123
654
“f”

16. Black Box Testing Techniques: Equivalence Class Partitioning Method(9)
Guidelines for Determining Equivalence class(1)
For the input as well as for the outputs, identify the restrictions and conditions in the specification
For every restriction or condition, partition into equivalence classes:
If a continuous numerical domain is specified, then create one valid and two invalid ECs
E.g., “the item count can be from 1 to 999”, identify one vEC (1 < item < 999) and two iEC (item count < 1 and item count > 999)
If a number of values should be entered, then create one valid (with all possible correct values) and two invalid ECs (less and more than)
E.g., “one through six owners can be listed for the car”, identify one vEC and two iEC (no owner and more than six owners)

17. Black Box Testing Techniques: Equivalence Class Partitioning Method(10)
Guidelines for Determining Equivalence class(2)
For every restriction or condition, partition into equivalence classes:
If a set of values is specified where each value may possibly be treated differently, then create one vEC for each value of the set (containing exactly this one value) and one additional iEC (containing all possible other values)
E.g., “type of vehicle must be BUS, TRUCK, TAXI, MOTORCYCLE”, identify one vEC from the set and one iEC (“BICYCLE”)
If there is a condition that must be fulfilled, then create one vEC and one iEC
E.g., “first character of the identifier must be a letter”, identify one vEC (it is a letter) and one iEC (it is not a letter)
If there is any doubt that the values of one EC are treated equally, then the EC should be further divided into subclasses

18. Black Box Testing Techniques: Equivalence Class Partitioning Method(11)
Test Cases (1)
Combination of the representatives
Rules for test case determination
The representative values of all vECs should be combined to test cases, meaning that all possible combinations of vECs will be covered. Those combinations build “valid test case” or “positive test case”
Separate test for invalid value
The representative value of an iEC can only be combined with representatives of other vECs. Thus every iEC leads to an additional “invalid test case” or “negative test case”.
If a test case combines more than one incorrect value, defect masking may result
Thus only one of the possible exceptions is actually triggered and tested

19. Black Box Testing Techniques: Equivalence Class Partitioning Method(11)
Test Cases (2)
Restriction of the number of test cases
Reducing the number of valid test cases as the result of multiplicative combination of representatives
Rules for test case restriction
Combine the test cases and sort them by frequency of occurrence. Prioritize the test cases in this order. That way only relevant test cases are tested
Test cases including boundary values or boundary value combinations are preferred
Combine every representative of one EC with every representative of other EC (dual combination)
Ensure as minimum criteria that every representative of an EC appears in at least one test case
Representatives of iECs should not be combined with representatives of other iECs

20. Black Box Testing Techniques: Equivalence Class Partitioning Method(11)
Test Completion Criteria
Define as the percentage of executed ECs in comparison to the total number of specified EC
EC-coverage = (number of tested EC / total number of EC) * 100%
Test coverage is a measurable criterion for finishing testing

21. Black Box Testing Techniques: Equivalence Class Partitioning Method(11)
The value of ECP
Contribute to a complete test where specified conditions and restrictions are not overlooked
Minimize the generation of unnecessary test cases
EC can be determined by for input and output
Can also be prepared for internal values and states, time dependent values, and interface parameters
Can be used in system testing, integration testing, and component testing
Only single input or output conditions are considered, while possible dependencies or interactions between conditions are ignored
May not consider boundary problems

22. Black Box Testing Techniques: Boundary Value Analysis (BVA) (1)
Faults often appear at the boundaries of ECs or boundary conditions
This is because of misunderstanding or unclear definition
Boundary conditions are those situations directly on, above, and beneath the edges of input ECs and output ECs
BVA differs with ECP in two aspects:
Rather than selecting any element in an EC as representative, BVA requires the one or more elements be selected such that each edge of the EC is the subject of a test
Rather than just focusing attention on the input conditions, test cases are also derived by considering the output ECs

23. Black Box Testing Techniques: Boundary Value Analysis (BVA) (2)
Guidelines for BVA
If an input condition specifies a range of values, write test cases for the ends of the range, and invalid-input test cases for situations just beyond the ends.
E.g., if the valid domain is [-1.0,…,+1.0], write the test cases for the situations -1.0, +1.0, ,
If an input condition specifies a number of values, write test cases for the minimum and maximum number of values and one beneath and beyond these values
E.g., if an input file can contain records, write test cases for 0, 1, 255, 256

24. Black Box Testing Techniques: Boundary Value Analysis (BVA) (3)
Guidelines for BVA
If an output domain serve as the basis, then the analysis can be done as follows:
The output of the test object is an integer value between 500 and Test output that should be achieved: 500, 1000, 499, 1001
If the permitted number of output values is to be tested, proceed just as with the number of input values: if outputs of 1 to 4 data values are allowed, the test outputs to produce are: 1, 4 as well as 0 and 5 data values
If the input or output of a program is an ordered set (a sequential file, a linear list or a table), focus attention on the first and last elements of the set
If complex data structures are given as input or output, an empty list or zero matrixes can be considered a boundary value

25. Black Box Testing Techniques: Boundary Value Analysis (BVA) (4)
Guidelines for BVA
For numeric calculations, values that are close together, as well as values that are far apart, should be taken into consideration as boundary values
For iECs, BVA is only useful when different exception handling for the test object is expected depending on an EC boundary
Extremely large data structures, lists, tables, etc. should be chosen. For example, those that exceed buffer, file, or data storage boundaries; to check behavior in extreme cases

26. Black Box Testing Techniques: Boundary Value Analysis (BVA) (5)
Test Cases
The valid boundaries inside an EC may be combined as test cases
The invalid boundaries must be verified separately and cannot be combined with other invalid boundaries

27. Black Box Testing Techniques: Boundary Value Analysis (BVA) (6)
Test Completion Criteria
BV-Coverage = (number of tested BV/total number of BV) *100%
The Value of the Technique
Should be done together with ECP
Require a lot of creativity to define test data at boundaries

28. Black Box Testing Techniques: State Transition Testing(1)
Base on history of execution or event or input, influences of the output and how the test object behaves
Test object starts from an initial state and then come into different states
Events trigger state transitions
Event is normally a function invocation
State transitions can involve actions
The test object can be a complex system with different system states, as well as a class in OOS with different states
Finite state machine, state diagrams, or state transition tables model this behavior

29. Black Box Testing Techniques: State Transition Testing(2)
Example: Stack
Top
Pop [height > 1]
Push [height < max -1]
Push*
top
initialize
Push
[height = max – 1]
push
empty
filled
full
Pop [height = 1]
pop
delete

30. Black Box Testing Techniques: State Transition Testing(3)
A possible test case for accepting strings
Precondition: stack is initialized; state is “empty”
Input: push (“hello”)
Expected result: stack contains “hello”
Post-condition: state of the stack is “filled”
Further test cases for the stack example
Test case 1: initialize [empty], push [filled], push, push, push [full]
Test case 2: initialize [empty], push [filled], top, pop [empty], delete
*note: not all states have been reached

31. Black Box Testing Techniques: State Transition Testing(4)
Test criteria
Should execute all specified functions of a certain state at least once
The compliance between the specified and the actual behavior can thus be checked.
To identify test cases
The finite state machine is transformed into a transition tree which includes certain sequences of transitions

32. Black Box Testing Techniques: State Transition Testing(5)
Building a transition tree
The initial or start state is the root of the tree
For every possible transition from the initial state to a following state in a transition diagram, the transition tree receives a branch from its root to a node, representing this next state
The process for step 2 is repeated for every leaf in the tree (every newly added node) until one of the following two end-conditions is fulfilled:
The corresponding state is already included in the tree on the way from the root to the node. This end-condition corresponds to one pass of a cycle in the transition diagram
The corresponding state is a final state.

33. Black Box Testing Techniques: State Transition Testing(6)
Building a transition tree (example)
initial
initialize
empty
delete
push
deleted
filled
pop
pop
push
push
pop
empty
filled
full
filled
filled
top
pop
push*
full
full
filled

34. Black Box Testing Techniques: State Transition Testing(7)
For the example: there are 8 different paths. Each path represents a test case
In addition, the reaction of the state machine for wrong usage must be checked
The functions are called in state in which they are not supposed to be called (test of robustness)
E.g., to delete a stack while in “full” state
So, extend the transition tree to include a branch for every function from every node

35. Black Box Testing Techniques: State Transition Testing(8)
Building a transition tree (example)
initial
failure
pop
initialize
empty
delete
failure
push
top
deleted
delete
filled
pop
failure
pop
push
push
pop
empty
filled
full
filled
filled
delete
top
pop
push*
failure
full
full
filled

36. Black Box Testing Techniques: State Transition Testing(9)
Also good for system testing
E.g., to test GUI by modeling it as a finite state machine
Screen and user controls as states
Input reactions as state transition
Appropriate test cases and test coverage can be identified from the model

37. Black Box Testing Techniques: State Transition Testing(10)
Test Cases
The following information is necessary for a complete definition of a state-based test case
The initial state
The input
The expected behavior
The expected final state
Further, for each expected transition of the test case the following aspects must be defined
The state before the transition
The initiating event that triggers the transition
The expected reaction triggered by the transition
The net expected state

38. Black Box Testing Techniques: State Transition Testing(10)
Test Completion Criteria
Every state has been reached at least once
Every transition has been executed at least once
Every transition violating the specification has been checked
Percentages can be defined for the above

39. Black Box Testing Techniques: State Transition Testing(10)
The value of this technique
Apply where states are important and where the functionality is influenced by the state
State transition testing is important for object-oriented testing
Takes into account the special aspects of object orientation

40. Black Box Testing Techniques: Cause-effect Graphing Technique(1)
Previous techniques do not regard combinations of input circumstances
Cause-effect graphing is a technique that uses the dependencies for identification of the test cases
The logical relationships between causes and their effects in a component or a system are displayed in a cause-effect graph
Every cause is described as a condition that consists of input conditions (or combinations of those) can be TRUE or FALSE
The conditions are connected with logical operators (e.g, AND, OR and NOT)

41. Black Box Testing Techniques: Cause-effect Graphing Technique(2)
Basic cause-effect graph symbols
NOT
identify a b a b
If a is TRUE, b is FALSE
else b is TRUE
If a is TRUE, b is TRUE
else b is FALSE a a OR
AND   b d c
If a and b are TRUE,
c is TRUE
else c is FALSE
If a or b or c is TRUE
d is TRUE
else d is FALSE b c

42. Black Box Testing Techniques: Cause-effect Graphing Technique(3) - example
withdrawing money at an ATM. In order to get the money from the machine, the following conditions must be fulfilled:
1. The bankcard is valid
2. The PIN must be correctly entered
3. The maximum number of PIN input is three
4. There is money in the machine, and in account
The following actions/effects are possible at the machine:
10. Reject card
11. Ask for another PIN input
12. “eat” the card
13. Ask for alternate dollar amount
14. Pay the requested amount of money

43. Black Box Testing Techniques: Cause-effect Graphing Technique(4)- example10 1  11 2  12 3  13 4  14

44. Black Box Testing Techniques: Cause-effect Graphing Technique(5)- example
The graph must be transformed to a decision table
A decision table has four quadrants
The upper left is list of the inputs (causes)
The upper right is combinations of inputs
The lower left consists of the effects
The lower right is the expected effects
The steps are:
Choose an effect
Looking in the graph, find combinations of causes that have the effect and combinations that do not have this effect
Add one column into the table for every one of these cause-combinations and the cause states of the remaining effects
Check if decision table entries occur several times and, if yes, delete them

45. Black Box Testing Techniques: Cause-effect Graphing Technique(6)- example
Condition/cause 1 2 3 4 5
1. Bankcard is valid N Y
2. PIN is correct -
3. 3 incorrect PIN
N(exit)
4. Money available
Effect/action
10. Reject card
11. Ask new PIN
12. Eat card
13. Ask new amount
14. Pay money

46. Black Box Testing Techniques: Cause-effect Graphing Technique(7)
Test cases
Every column is a test case
The table defines the logical test cases
Must feed with concrete data values
Annotate necessary preconditions and post-conditions
The value of the technique
May reveal combinations which are not included when using other test case design techniques
However, errors can result from optimization of the table
Graph and table may grow very quickly and lose readability when the number of conditions and dependent actions increases
Without support by tools, the technique is not easily applicable