1. WARNING
These slides are not optimized for printing or exam preparation. These are for lecture delivery only. These slides are made for PowerPoint They may not show up well on other PowerPoint versions. You can download PowerPoint 2010 viewer from here. These slides contain a lot of animations. For optimal results, watch in slideshow mode.

2. SUT
Test case

3. SUT
Test case
Test case

4. SUT
Test case
Test case
To do this, we have to design these test cases

5. When do we define test cases? Are they pre-determined or on-the-fly?
SUT
Test case
Test case

6. SUT Pre-determined or on-the-fly?
Test case
Test case
How much internal info about the SUT should be revealed to the test case designer?

7. SUT Pre-determined or on-the-fly?
Test case
How to choose which test cases to use and which to discard?
Test case
How much info revealed?

8. Heuristics for Better Test Case Design
Pre-determined or on-the-fly?
How to choose?
Heuristics for Better Test Case Design
How much info revealed?

9. Pre-determined or on-the-fly?
SUT
Test case

10. Pre-determined or on-the-fly?
SUT
Test case

11. SUT scripted testing  which one? exploratory testing  which one?
Pre-determined or on-the-fly?
SUT
Test case
scripted testing  which one?
exploratory testing  which one?

12. SUT scripted testing  pre-determined exploratory testing  on-the-fly
Pre-determined or on-the-fly?
SUT
Test case
scripted testing  pre-determined
exploratory testing  on-the-fly
Most of the jokes done by comedians are scripted. They sometimes throw in a few on-the-spot jokes, but even they are not totally random.
how to choose?

13. Scripted+Exploratory
SUT
Test case

14. Scripted+Exploratory
SUT
Test case
Test case
How much info revealed?

15. Scripted+Exploratory
SUT
Test case
Test case
How much info revealed?

16. SUT
Developer
Tester
Test case
How much info revealed?

17. SUT
Developer
Tester
Test case
How much info revealed?

18. Glass-box / a.k.a white box
Gray-box
Test case
Black-box
How much info revealed?

19. SUT How to choose? Scripted+Exploratory Black/Glass/Gray-box Test case

20. How to choose?
Why choose?
Some heuristics
Test case

21. Do not panic! This is a simulation 
[extra] BSoD
Do not panic! This is a simulation 
[CS2103]

22. Windows without bugs? That’s crazy talk!
[extra] BSoD
Windows without bugs? That’s crazy talk!
[CS2103]

23. How to choose?
Why choose?
Test case

24. How to choose?
Why choose?
[Effectiveness & Efficiency]

25. Both tested the same system
How to choose?
Why choose?
Dilbert
Wally
Both tested the same system
[Effectiveness & Efficiency]
Found 20 bugs
Found 10 bugs

26. Both tested the same system
How to choose?
Why choose?
Dilbert
Wally
Both tested the same system
[Effectiveness & Efficiency]
Found 20 bugs
Found 20 bugs
Used 100 test cases
Used 40 test cases

27. How to choose?
Why choose?
[Effectiveness & Efficiency]
A test case should have a reasonable chance of finding a new bug not found by the other test cases.

28. How to choose?
E&E of testing
Some heuristics
Test case

29. Equivalence partitioning
How to choose?
E&E of testing
Some heuristics
Test case
Equivalence partitioning

30. Equivalence partitioning
How to choose?
E&E of testing
Some heuristics
Test case
Equivalence partitioning

31. Equivalence partitioning

32. 77577 OR tinyurl.com/answerpost
Equivalence partitioning
isValidMonth (int m): boolean
Description:
checks if m is in the range [1..12]. returns true if m is in the range, false otherwise.
Which values would you use to test this isValidMonth method?
cases {value1 value2 …}
e.g. cases 1 4 9
OR tinyurl.com/answerpost

33. Pass/fail? Hmmm… interesting. Let me try 275 Should he? Actual: true
Equivalence partitioning
isValidMonth (int m): boolean
Description:
checks if m is in the range [1..12]. returns true if m is in the range, false otherwise.
Pass/fail?
Hmmm… interesting.
Let me try 275
Test case 23:
Input: 273
Expected: false
Test Case 12
Test Case 13
Test case 23:
Input: 274
Expected: false
Should he?
Actual: true
Actual: true

34. 275 Must be broken too! Broken! Broken! isValidMonth (int m): boolean
Equivalence partitioning
isValidMonth (int m): boolean
275
273
274
Must be broken too!
Broken!
Broken!

35. Equivalence partitioning
isValidMonth (int m): boolean
275
273
274

36. 275 isValidMonth (int m): boolean 273 274 Equivalence partitioning
MIN_INT 1 12 13
MAX_INT
275
273
274

37. isValidMonth (int m): boolean
Equivalence partitioning
isValidMonth (int m): boolean
MIN_INT 1 12 13
MAX_INT

38. Equivalence partitioning
How to partition?

39. Equivalence partitioning
Input space
Output space
SUT

40. SUT Equivalence partitioning Output space Input space
Input partition 1
Input partition 2
Input partition 3
Output 1
Output 2
Output 3
Input space
Output space
Input partition 1a
SUT
false
true

41. SUT Equivalence partitioning Input partition 1 Output 1
Input partition 1a
Input partition 1b
SUT
false
true

42. Equivalence partitioning

43. Why does i need more than one eq. partition?
Equivalence partitioning
Why does i need more than one eq. partition?
int square (int i)
Description: returns the the square value of i

44. Description: returns the the square value of i
Equivalence partitioning
int square (int i)
Description: returns the the square value of i
+√MAX
-√MAX -1
MIN
MAX

45. Description: returns the the square value of i
Equivalence partitioning
int square (int i)
Description: returns the the square value of i
+√MAX
-√MAX -1
MIN
MAX

46. Equivalence partitioning is … It helps to improve E&E by …
How to choose?
E&E of testing
Some heuristics
Test case
Equivalence partitioning
Equivalence partitioning is …
It helps to improve E&E by …

47. Equivalence partitioning Boundary value analysis
How to choose?
E&E of testing
Some heuristics
Test case
Equivalence partitioning
Boundary value analysis

48. Equivalence partitioning Boundary value analysis
How to choose?
E&E of testing
Some heuristics
Test case
Equivalence partitioning
Boundary value analysis

49. isValidMonth (int m): boolean
Boundary value analysis
isValidMonth (int m): boolean
MIN_INT 1 12 13
MAX_INT

50. isValidMonth (int m): boolean
Boundary value analysis
isValidMonth (int m): boolean
MIN_INT 1 12 13
MAX_INT
if(m>=1)||(m<12) …

51. isValidMonth (int m): boolean
Boundary value analysis
isValidMonth (int m): boolean
MIN_INT 1 12 13
MAX_INT

52. What are the boundary values for i?
Boundary value analysis
What are the boundary values for i?
isPrimeNumber (int i):boolean
Description: Checks if i is a prime number. Returns true if i is a prime number, false otherwise.
Not prime numbers
Prime numbers

53. What are the boundary values for i?
Boundary value analysis
What are the boundary values for i?
isPrimeNumber (int i):boolean
Description: Checks if i is a prime number. Returns true if i is a prime number, false otherwise.

54. BVA helps to improve E&E by …
How to choose?
E&E of testing
Some heuristics
Test case
Equivalence partitioning
Boundary value analysis
BVA helps to improve E&E by …

55. Equivalence partitioning Boundary value analysis
How to choose?
E&E of testing
Some heuristics
Test case
Equivalence partitioning
Boundary value analysis
Combining multiple inputs

56. Equivalence partitioning Boundary value analysis
How to choose?
E&E of testing
Some heuristics
Test case
Equivalence partitioning
Boundary value analysis
Combining multiple inputs

57. Combining multiple inputs
:MSLogic
markCellAt(x,y)

58. Combining multiple inputs
markCellAt(x,y)
markCellAt(x,y)

59. Combining multiple inputs
markCellAt(x,y)
MSLogic x y
Cell
GAME_STATE

60. [MIN_INT..-1] [0..(W-1)] [W..MAX_INT] [MIN_INT..-1] [0..(H-1)]
Combining multiple inputs
markCellAt(x,y) x y
Cell
GAME_STATE
HIDDEN
MARKED
CLEARED
INC_MARKED
INC_CLEARED
PRE_GAME
READY
IN_PLAY
WON
LOST
[MIN_INT..-1] [0..(W-1)] [W..MAX_INT]
[MIN_INT..-1]
[0..(H-1)]
[H..MAX_INT]
W -> Width
H -> Height

61. [MIN_INT..-1] [0..(W-1)] [W..MAX_INT] [MIN_INT..-1] [0..(H-1)]
Combining multiple inputs
markCellAt(x,y) x y
Cell
GAME_STATE
HIDDEN
MARKED
CLEARED
INC_MARKED
INC_CLEARED
PRE_GAME
READY
IN_PLAY
WON
LOST
[MIN_INT..-1] [0..(W-1)] [W..MAX_INT]
[MIN_INT..-1]
[0..(H-1)]
[H..MAX_INT]
MIN_INT,
-2, -1,0, 2,
(W-1), W,
W+1,
MAX_INT

62. [MIN_INT..-1] [0..(W-1)] [W..MAX_INT] [MIN_INT..-1] [0..(H-1)]
Combining multiple inputs
markCellAt(x,y) x y
Cell
GAME_STATE
HIDDEN
MARKED
CLEARED
INC_MARKED
INC_CLEARED
PRE_GAME
READY
IN_PLAY
WON
LOST
[MIN_INT..-1] [0..(W-1)] [W..MAX_INT]
[MIN_INT..-1]
[0..(H-1)]
[H..MAX_INT]
MIN_INT,
-2, -1,0, 2,
(W-1), W,
W+1,
MAX_INT
MIN_INT,
-2, -1,0, 2,
(H-1), H,
H+1,
MAX_INT

63. HIDDEN MARKED CLEARED INC_MARKED INC_CLEARED PRE_GAME READY IN_PLAY
Combining multiple inputs
markCellAt(x,y) x y
Cell
GAME_STATE
HIDDEN
MARKED
CLEARED
INC_MARKED
INC_CLEARED
PRE_GAME
READY
IN_PLAY
WON
LOST
MIN_INT,
-2, -1,0, 2,
(W-1), W,
W+1,
MAX_INT
MIN_INT,
-2, -1,0, 2,
(H-1), H,
H+1,
MAX_INT

64. HIDDEN MARKED CLEARED INC_MARKED INC_CLEARED PRE_GAME READY IN_PLAY
Combining multiple inputs
markCellAt(x,y)
Cell
GAME_STATE
HIDDEN
MARKED
CLEARED
INC_MARKED
INC_CLEARED
PRE_GAME
READY
IN_PLAY
WON
LOST y
MIN_INT,
-2, -1,0, 2,
(H-1), H,
H+1,
MAX_INT x
MIN_INT,
-2, -1,0, 2,
(W-1), W,
W+1,
MAX_INT

65. HIDDEN MARKED CLEARED INC_MARKED INC_CLEARED PRE_GAME READY IN_PLAY
Combining multiple inputs
markCellAt(x,y)
Cell
GAME_STATE
HIDDEN
MARKED
CLEARED
INC_MARKED
INC_CLEARED
PRE_GAME
READY
IN_PLAY
WON
LOST y
MIN_INT,
-2, -1,0, 2,
(H-1), H,
H+1,
MAX_INT x
MIN_INT,
-2, -1,0, 2,
(W-1), W,
W+1,
MAX_INT

66. 2025 HIDDEN MARKED CLEARED INC_MARKED INC_CLEARED PRE_GAME READY
Combining multiple inputs
markCellAt(x,y)
Cell
GAME_STATE
HIDDEN
MARKED
CLEARED
INC_MARKED
INC_CLEARED
PRE_GAME
READY
IN_PLAY
WON
LOST y
MIN_INT,
-2, -1,0, 2,
(H-1), H,
H+1,
MAX_INT x
MIN_INT,
-2, -1,0, 2,
(W-1), W,
W+1,
MAX_INT
9x9x5x5=
2025

67. Combining multiple inputs

68. 77577 OR tinyurl.com/answerpost
Combining multiple inputs a b c
valid1
valid3
valid4
valid2
invalid2
valid5
invalid1
invalid3
3x3x2= 18
Payment for testing: $100
Cost per test case: $10
Penalty for uncaught bug: $200
Work out a set of test cases that is both E&E. How many do you have?
count {answer}
e.g. count 9
OR tinyurl.com/answerpost

69. 18 a b c valid1 valid3 valid4 valid2 invalid2 valid5 invalid1 invalid3
Combining multiple inputs a b c
valid1
valid3
valid4
valid2
invalid2
valid5
invalid1
invalid3
3x3x2= 18
Case a b c
Output 1
valid1
valid3
valid4
behavior 1 2
valid2
invalid2
valid5
error 1 3
invalid1
invalid3
-any -
error 2

70. 18 a b c valid1 valid3 valid4 valid2 invalid2 valid5 invalid1 invalid3
Combining multiple inputs a b c
valid1
valid3
valid4
valid2
invalid2
valid5
invalid1
invalid3
3x3x2= 18
Case a b c
Output 1
valid1
valid3
valid4
behavior 1 2
valid2
invalid2
valid5
error 1 3
invalid1
invalid3
-any -
error 2

71. 18 a b c valid1 valid3 valid4 valid2 invalid2 valid5 invalid1 invalid3
Combining multiple inputs a b c
valid1
valid3
valid4
valid2
invalid2
valid5
invalid1
invalid3
3x3x2= 18
Case a b c
Output 1
valid1
valid3
valid4
behavior 1 2
valid2
invalid2
valid5
error 1 3
invalid1
invalid3
-any -
error 2

72. Case a b c
Output 1
valid1
valid3
valid4
behavior 1 2
valid2
invalid2
valid5
error 1 3
invalid1
invalid3
-any -
error 2
Heuristic : Each valid input should appear at least once without invalid inputs

73. Case a b c
Output 1
valid1
valid3
valid4
behavior 1 2
valid2
invalid2
valid5
error 1 3
invalid1
invalid3
-any -
error 2
Heuristic : Each valid input should appear at least once without invalid inputs

74. Case a b c
Output 1
valid1
valid3
valid4
behavior 1 2
valid2
invalid2
valid5
error 1 3
invalid1
invalid3
-any -
error 2
Case a b c
Output 1
valid1
valid3
valid4
behavior 1 2
valid2
-any valid -
valid5
behavior 2 2a
invalid2
error 1 3
invalid1
invalid3
-any -
error 2
Heuristic : Each valid input should appear at least once without invalid inputs

75. Case a b c
Output 1
valid1
valid3
valid4
behavior 1 2
valid2
invalid2
valid5
error 1 3
invalid1
invalid3
-any -
error 2
Case a b c
Output 1
valid1
valid3
valid4
behavior 1 2
valid2
-any valid -
valid5
behavior 2 2a
invalid2
error 1 3
invalid1
invalid3
-any -
error 2
Heuristic : Each valid input should appear at least once without invalid inputs

76. Heuristic: Only one invalid input per test caseb c
Output 1
valid1
valid3
valid4
behavior 1 2
valid2
invalid2
valid5
error 1 3
invalid1
invalid3
-any -
error 2
Case a b c
Output 1
valid1
valid3
valid4
behavior 1 2
valid2
-any valid -
valid5
behavior 2 2a
invalid2
error 1 3
invalid1
invalid3
-any -
error 2
Heuristic: Only one invalid input per test case

77. Heuristic: Only one invalid input per test caseb c
Output 1
valid1
valid3
valid4
behavior 1 2
valid2
invalid2
valid5
error 1 3
invalid1
invalid3
-any -
error 2
Case a b c
Output 1
valid1
valid3
valid4
behavior 1 2
valid2
-any valid -
valid5
behavior 2 2a
invalid2
error 1 3
invalid1
invalid3
-any -
error 2
Heuristic: Only one invalid input per test case

78. Heuristic: Only one invalid input per test caseb c
Output 1
valid1
valid3
valid4
behavior 1 2
valid2
-any valid -
valid5
behavior 2 2a
invalid2
error 1 3
invalid1
invalid3
-any -
error 2 5
3x3x2= 18
Case a b c
Output 1
valid1
valid3
valid4
behavior 1 2
valid2
-any valid -
valid5
behavior 2 2a
invalid2
error 1 3
invalid1
error 2 4
invalid3
error 3
Heuristic: Only one invalid input per test case

79. Combining multiple inputs Equivalence partitioning
How to choose?
E&E of testing
Some heuristics
Combining multiple inputs
Test case
Equivalence partitioning
Boundary value analysis

81. Apply in *some* places. e.g.
//5 is a boundary value for size
assertEquals(5, getSize());

85. Test case design heuristics
Other QA techniques
Test case design heuristics