Download presentation
Presentation is loading. Please wait.
1
WARNING 40 min These slides are not optimized for printing or exam preparation. These are for lecture delivery only. These slides are made for PowerPoint They may not show up well on other PowerPoint versions. You can download PowerPoint 2010 viewer from here. These slides contain a lot of animations. For optimal results, watch in slideshow mode.
2
CS2103/T, Lecture 8, Part 3, [March, 2017]
Pre-determined or on-the-fly? How to choose? Heuristics for Better Test Case Design CS2103/T, Lecture 8, Part 3, [March, 2017] How much info revealed?
![CS2103/T, Lecture 8, Part 3, [March, 2017]](http://slideplayer.com/slide/13012520/79/images/2/CS2103%2FT%2C+Lecture+8%2C+Part+3%2C+%5BMarch%2C+2017%5D.jpg "Pre-determined or on-the-fly How to choose Heuristics for Better Test Case Design. CS2103/T, Lecture 8, Part 3, [March, 2017] How much info revealed")
3
SUT How to choose? Scripted+Exploratory Black/Glass/Gray-box Test case
4
How to choose? Why choose? Some heuristics Test case
5
[extra] BSoD This is a simulation [CS2103]
![[extra] BSoD This is a simulation [CS2103]](http://slideplayer.com/slide/13012520/79/images/5/%5Bextra%5D+BSoD+This+is+a+simulation+%EF%81%8A+%5BCS2103%5D.jpg "[extra] BSoD This is a simulation [CS2103]")
6
How to choose? Why choose? Test case Cost in term of time + money
7
How to choose? Why choose? [Effectiveness & Efficiency]
![How to choose Why choose [Effectiveness & Efficiency]](http://slideplayer.com/slide/13012520/79/images/7/How+to+choose+Why+choose+%5BEffectiveness+%26+Efficiency%5D.jpg "How to choose Why choose [Effectiveness & Efficiency]")
8
Both tested the same system
How to choose? Why choose? Dilbert Wally Both tested the same system [Effectiveness & Efficiency] Found 20 bugs Found 10 bugs
9
Both tested the same system
How to choose? Why choose? Dilbert Wally Both tested the same system [Effectiveness & Efficiency] Found 20 bugs Found 20 bugs Used 100 test cases Used 40 test cases
10
How to choose? Why choose? [Effectiveness & Efficiency] A test case should have a reasonable chance of finding a new bug not found by the other test cases.
![How to choose Why choose [Effectiveness & Efficiency]](http://slideplayer.com/slide/13012520/79/images/10/How+to+choose+Why+choose+%5BEffectiveness+%26+Efficiency%5D.jpg "A test case should have a reasonable chance of finding a new bug not found by the other test cases.")
11
How to choose? E&E of testing Some heuristics Test case
12
How to choose? E&E of testing Some heuristics Test case
13
Equivalence partitioning
How to choose? E&E of testing Some heuristics Test case Equivalence partitioning
14
Equivalence partitioning
How to choose? E&E of testing Some heuristics Test case Equivalence partitioning
15
Equivalence partitioning
16
77577 OR tinyurl.com/answerpost
Equivalence partitioning isValidMonth (int m): boolean Description: checks if m is in the range [1..12]. returns true if m is in the range, false otherwise. Which values would you use to test this isValidMonth method? cases {value1 value2 …} e.g. cases 1 4 9 OR tinyurl.com/answerpost
17
Pass/fail? Hmmm… interesting. Let me try 275 Should he? Why not?
Equivalence partitioning isValidMonth (int m): boolean Description: checks if m is in the range [1..12]. returns true if m is in the range, false otherwise. Pass/fail? Hmmm… interesting. Let me try 275 Test case 23: Input: 273 Expected: false Test Case 12 Test Case 13 Test case 23: Input: 274 Expected: false Should he? Why not? Actual: true Actual: true
18
275 Must be broken too! Broken! Broken! isValidMonth (int m): boolean
Equivalence partitioning isValidMonth (int m): boolean 275 273 274 Must be broken too! Broken! Broken!
19
Equivalence partitioning
isValidMonth (int m): boolean 275 273 274
20
275 isValidMonth (int m): boolean 273 274 Equivalence partitioning
MIN_INT 1 12 13 MAX_INT 275 273 274
21
isValidMonth (int m): boolean
Equivalence partitioning isValidMonth (int m): boolean MIN_INT 1 12 13 MAX_INT
22
Equivalence partitioning
Input space Output space SUT
23
SUT Equivalence partitioning Output space Input space
Input partition 1 Input partition 2 Input partition 3 Output 1 Output 2 Output 3 Input space Output space Input partition 1a SUT false true
24
SUT Equivalence partitioning Input partition 1 Output 1
Input partition 1a Input partition 1b SUT false true
25
Equivalence partitioning
26
How many equivalence partition?
Equivalence partitioning How many equivalence partition? int square (int i) Description: returns the the square value of i
27
Description: returns the the square value of i
Equivalence partitioning int square (int i) Description: returns the the square value of i +√MAX -√MAX -1 MIN MAX
28
Description: returns the the square value of i
Equivalence partitioning int square (int i) Description: returns the the square value of i +√MAX -√MAX -1 MIN MAX
29
It helps to improve E&E by …
How to choose? E&E of testing Some heuristics Test case Equivalence partitioning An Eq. partition is a … It helps to improve E&E by …
30
Equivalence partitioning
How to choose? E&E of testing Some heuristics Test case Equivalence partitioning
31
Equivalence partitioning Boundary value analysis
How to choose? E&E of testing Some heuristics Test case Equivalence partitioning Boundary value analysis
32
Equivalence partitioning Boundary value analysis
How to choose? E&E of testing Some heuristics Test case Equivalence partitioning Boundary value analysis
33
Boundary value analysis
34
isValidMonth (int m): boolean
Boundary value analysis isValidMonth (int m): boolean MIN_INT 1 12 13 MAX_INT
35
isValidMonth (int m): boolean
Boundary value analysis isValidMonth (int m): boolean MIN_INT 1 12 13 MAX_INT
36
isValidMonth (int m): boolean
Boundary value analysis isValidMonth (int m): boolean MIN_INT 1 12 13 MAX_INT if (m>=1) && (m<12) … if (m>=1) && (m<=12) …
37
isValidMonth (int m): boolean
Boundary value analysis isValidMonth (int m): boolean MIN_INT 1 12 13 MAX_INT
38
What are the boundary values for i?
Boundary value analysis What are the boundary values for i? isPrimeNumber (int i):boolean Description: Checks if i is a prime number. Returns true if i is a prime number, false otherwise. Not prime numbers Prime numbers
39
What are the boundary values for i?
Boundary value analysis What are the boundary values for i? isPrimeNumber (int i):boolean Description: Checks if i is a prime number. Returns true if i is a prime number, false otherwise.
40
BVA helps to improve E&E by …
How to choose? E&E of testing Some heuristics Test case Equivalence partitioning Boundary value analysis BVA helps to improve E&E by …
41
Equivalence partitioning Boundary value analysis
How to choose? E&E of testing Some heuristics Test case Equivalence partitioning Boundary value analysis Combining multiple inputs
42
Equivalence partitioning Boundary value analysis
How to choose? E&E of testing Some heuristics Test case Equivalence partitioning Boundary value analysis Combining multiple inputs
43
Combining multiple inputs
:MSLogic markCellAt(x,y)
44
Combining multiple inputs
markCellAt(x,y) markCellAt(x,y)
45
Combining multiple inputs
markCellAt(x,y) MSLogic x y Cell GAME_STATE
46
[MIN_INT..-1] [0..(W-1)] [W..MAX_INT] [MIN_INT..-1] [0..(H-1)]
Combining multiple inputs markCellAt(x,y) x y Cell GAME_STATE HIDDEN MARKED CLEARED INC_MARKED INC_CLEARED PRE_GAME READY IN_PLAY WON LOST [MIN_INT..-1] [0..(W-1)] [W..MAX_INT] [MIN_INT..-1] [0..(H-1)] [H..MAX_INT] W -> Width H -> Height
![[MIN_INT..-1] [0..(W-1)] [W..MAX_INT] [MIN_INT..-1] [0..(H-1)]](http://slideplayer.com/slide/13012520/79/images/46/%5BMIN_INT..-1%5D+%5B0..%28W-1%29%5D+%5BW..MAX_INT%5D+%5BMIN_INT..-1%5D+%5B0..%28H-1%29%5D.jpg "Combining multiple inputs. markCellAt(x,y) x. y. Cell. GAME_STATE. HIDDEN. MARKED. CLEARED. INC_MARKED. INC_CLEARED. PRE_GAME. READY. IN_PLAY. WON. LOST. [MIN_INT..-1] [0..(W-1)] [W..MAX_INT] [MIN_INT..-1] [0..(H-1)] [H..MAX_INT] W -> Width. H -> Height.")
47
[MIN_INT..-1] [0..(W-1)] [W..MAX_INT] [MIN_INT..-1] [0..(H-1)]
Combining multiple inputs markCellAt(x,y) x y Cell GAME_STATE HIDDEN MARKED CLEARED INC_MARKED INC_CLEARED PRE_GAME READY IN_PLAY WON LOST [MIN_INT..-1] [0..(W-1)] [W..MAX_INT] [MIN_INT..-1] [0..(H-1)] [H..MAX_INT] MIN_INT, -2, -1, 0, 2, (W-1), W, W+1, MAX_INT
![[MIN_INT..-1] [0..(W-1)] [W..MAX_INT] [MIN_INT..-1] [0..(H-1)]](http://slideplayer.com/slide/13012520/79/images/47/%5BMIN_INT..-1%5D+%5B0..%28W-1%29%5D+%5BW..MAX_INT%5D+%5BMIN_INT..-1%5D+%5B0..%28H-1%29%5D.jpg "Combining multiple inputs. markCellAt(x,y) x. y. Cell. GAME_STATE. HIDDEN. MARKED. CLEARED. INC_MARKED. INC_CLEARED. PRE_GAME. READY. IN_PLAY. WON. LOST. [MIN_INT..-1] [0..(W-1)] [W..MAX_INT] [MIN_INT..-1] [0..(H-1)] [H..MAX_INT] MIN_INT, -2, -1, 0, 2, (W-1), W, W+1, MAX_INT.")
48
[MIN_INT..-1] [0..(W-1)] [W..MAX_INT] [MIN_INT..-1] [0..(H-1)]
Combining multiple inputs markCellAt(x,y) x y Cell GAME_STATE HIDDEN MARKED CLEARED INC_MARKED INC_CLEARED PRE_GAME READY IN_PLAY WON LOST [MIN_INT..-1] [0..(W-1)] [W..MAX_INT] [MIN_INT..-1] [0..(H-1)] [H..MAX_INT] MIN_INT, -2, -1, 0, 2, (W-1), W, W+1, MAX_INT MIN_INT, -2, -1, 0, 2, (H-1), H, H+1, MAX_INT
![[MIN_INT..-1] [0..(W-1)] [W..MAX_INT] [MIN_INT..-1] [0..(H-1)]](http://slideplayer.com/slide/13012520/79/images/48/%5BMIN_INT..-1%5D+%5B0..%28W-1%29%5D+%5BW..MAX_INT%5D+%5BMIN_INT..-1%5D+%5B0..%28H-1%29%5D.jpg "Combining multiple inputs. markCellAt(x,y) x. y. Cell. GAME_STATE. HIDDEN. MARKED. CLEARED. INC_MARKED. INC_CLEARED. PRE_GAME. READY. IN_PLAY. WON. LOST. [MIN_INT..-1] [0..(W-1)] [W..MAX_INT] [MIN_INT..-1] [0..(H-1)] [H..MAX_INT] MIN_INT, -2, -1, 0, 2, (W-1), W, W+1, MAX_INT. MIN_INT, -2, -1, 0, 2, (H-1), H, H+1, MAX_INT.")
49
HIDDEN MARKED CLEARED INC_MARKED INC_CLEARED PRE_GAME READY IN_PLAY
Combining multiple inputs markCellAt(x,y) x y Cell GAME_STATE HIDDEN MARKED CLEARED INC_MARKED INC_CLEARED PRE_GAME READY IN_PLAY WON LOST MIN_INT, -2, -1, 0, 2, (W-1), W, W+1, MAX_INT MIN_INT, -2, -1, 0, 2, (H-1), H, H+1, MAX_INT
50
HIDDEN MARKED CLEARED INC_MARKED INC_CLEARED PRE_GAME READY IN_PLAY
Combining multiple inputs markCellAt(x,y) Cell GAME_STATE HIDDEN MARKED CLEARED INC_MARKED INC_CLEARED PRE_GAME READY IN_PLAY WON LOST y MIN_INT, -2, -1, 0, 2, (H-1), H, H+1, MAX_INT x MIN_INT, -2, -1, 0, 2, (W-1), W, W+1, MAX_INT
51
HIDDEN MARKED CLEARED INC_MARKED INC_CLEARED PRE_GAME READY IN_PLAY
Combining multiple inputs markCellAt(x,y) Cell GAME_STATE HIDDEN MARKED CLEARED INC_MARKED INC_CLEARED PRE_GAME READY IN_PLAY WON LOST y MIN_INT, -2, -1, 0, 2, (H-1), H, H+1, MAX_INT x MIN_INT, -2, -1, 0, 2, (W-1), W, W+1, MAX_INT
52
2025 HIDDEN MARKED CLEARED INC_MARKED INC_CLEARED PRE_GAME READY
Combining multiple inputs markCellAt(x,y) Cell GAME_STATE HIDDEN MARKED CLEARED INC_MARKED INC_CLEARED PRE_GAME READY IN_PLAY WON LOST y MIN_INT, -2, -1, 0, 2, (H-1), H, H+1, MAX_INT x MIN_INT, -2, -1, 0, 2, (W-1), W, W+1, MAX_INT 9x9x5x5= 2025
53
Combining multiple inputs
54
77577 OR tinyurl.com/answerpost
Combining multiple inputs gender size speed female large fast male medium slow blank small 3x3x2= 18 Payment for testing: $100 Cost per test case: $10 Penalty for uncaught bug: $200 Work out a set of test cases that is both E&E. How many do you have? count {answer} e.g. count 9 OR tinyurl.com/answerpost
55
18 gender size speed female large fast male medium slow blank small
Combining multiple inputs gender size speed female large fast male medium slow blank small 3x3x2= 18 Case gender size speed Output 1 female large fast behavior 1 2 male medium slow error 1 3 blank small -any - error 2
56
18 gender size speed female large fast male medium slow blank small
Combining multiple inputs gender size speed female large fast male medium slow blank small 3x3x2= 18 Case gender size speed Output 1 female large fast behavior 1 2 male medium slow error 1 3 blank small -any - error 2
57
18 gender size speed female large fast male medium slow blank small
Combining multiple inputs gender size speed female large fast male medium slow blank small 3x3x2= 18 Case gender size speed Output 1 female large fast behavior 1 2 male medium slow error 1 3 blank small -any - error 2
58
Case gender size speed Output 1 female large fast behavior 1 2 male medium slow error 1 3 blank small -any - error 2 Heuristic : Each valid input should appear at least once without invalid inputs
59
Case gender size speed Output 1 female large fast behavior 1 2 male medium slow error 1 3 blank small -any - error 2 Heuristic : Each valid input should appear at least once without invalid inputs
60
Case gender size speed Output 1 female large fast behavior 1 2 male medium slow error 1 3 blank small -any - error 2 Case gender size speed Output 1 female large fast behavior 1 2 male -any valid - slow behavior 2 2a medium error 1 3 blank small -any - error 2 Heuristic : Each valid input should appear at least once without invalid inputs
61
Case gender size speed Output 1 female large fast behavior 1 2 male medium slow error 1 3 blank small -any - error 2 Case gender size speed Output 1 female large fast behavior 1 2 male -any valid - slow behavior 2 2a medium error 1 3 blank small -any - error 2 Heuristic : Each valid input should appear at least once without invalid inputs
62
Heuristic: Only one invalid input per test case
gender size speed Output 1 female large fast behavior 1 2 male medium slow error 1 3 blank small -any - error 2 Case gender size speed Output 1 female large fast behavior 1 2 male -any valid - slow behavior 2 2a medium error 1 3 blank small -any - error 2 Heuristic: Only one invalid input per test case
63
Heuristic: Only one invalid input per test case
gender size speed Output 1 female large fast behavior 1 2 male medium slow error 1 3 blank small -any - error 2 Case gender size speed Output 1 female large fast behavior 1 2 male -any valid - slow behavior 2 2a medium error 1 3 blank small -any - error 2 Heuristic: Only one invalid input per test case
64
Heuristic: Only one invalid input per test case
gender size speed Output 1 female large fast behavior 1 2 male -any valid - slow behavior 2 2a medium error 1 3 blank small error 2 5 3x3x2= 18 Case gender size speed Output 1 female large fast behavior 1 2 male -any valid - slow behavior 2 2a medium error 1 3 blank error 2 4 small error 3 Heuristic: Only one invalid input per test case
65
Combining multiple inputs Equivalence partitioning
How to choose? E&E of testing Some heuristics Combining multiple inputs Test case Equivalence partitioning Boundary value analysis
67
Apply in *some* places. e.g.
//5 is a boundary value for size assertEquals(5, getSize());
71
Test case design heuristics
Other QA techniques Test case design heuristics
Similar presentations
