Presentation is loading. Please wait.

Presentation is loading. Please wait.

Dynamic Testing Techniques 1 Principles2 Lifecycle 4 Dynamic test techniques 3 Static testing 5 Management6 Tools Software Testing ISTQB / ISEB Foundation.

Similar presentations


Presentation on theme: "Dynamic Testing Techniques 1 Principles2 Lifecycle 4 Dynamic test techniques 3 Static testing 5 Management6 Tools Software Testing ISTQB / ISEB Foundation."— Presentation transcript:

1 Dynamic Testing Techniques 1 Principles2 Lifecycle 4 Dynamic test techniques 3 Static testing 5 Management6 Tools Software Testing ISTQB / ISEB Foundation Exam Practice Chapter 4

2 Contents What is a testing technique? Black and White box testing Black box test techniques White box test techniques Error Guessing Dynamic Testing Techniques ISTQB / ISEB Foundation Exam Practice

3 Why dynamic test techniques? n Exhaustive testing (use of all possible inputs and conditions) is impractical -must use a subset of all possible test cases -must have high probability of detecting faults n Need thought processes that help us select test cases more intelligently -test case design techniques are such thought processes

4 What is a testing technique? n a procedure for selecting or designing tests n based on a structural or functional model of the software n successful at finding faults n 'best' practice n a way of deriving good test cases n a way of objectively measuring a test effort Testing should be rigorous, thorough and systematic

5 Using techniques makes testing much more effective Advantages of techniques n Different people: similar probability find faults -gain some independence of thought n Effective testing: find more faults -focus attention on specific types of fault -know you're testing the right thing n Efficient testing: find faults with less effort -avoid duplication -systematic techniques are measurable

6 Measurement n Objective assessment of thoroughness of testing (with respect to use of each technique) -useful for comparison of one test effort to another n E.g. Project A 60% Equivalence partitions 50% Boundaries 75% Branches Project B 40% Equivalence partitions 45% Boundaries 60% Branches

7 Contents What is a testing technique? Black and White box testing Black box test techniques White box test techniques Error Guessing Dynamic Testing Techniques ISTQB / ISEB Foundation Exam Practice

8 Three types of systematic technique Static (non-execution) examination of documentation, source code listings, etc.examination of documentation, source code listings, etc. Functional (Black Box) based on behaviour / functionality of softwarebased on behaviour / functionality of software Structural (White Box) based on structure of softwarebased on structure of software

9 Some test techniques Static Dynamic Structural Behavioural Functional Non-functional Reviews Walkthroughs Desk-checking Data Flow Data Flow Symbolic Execution Symbolic Execution Definition -Use Definition -Use Statement Branch/Decision Branch Condition Branch Condition Combination LCSAJ Arcs Equivalence Partitioning Equivalence Partitioning Boundary Value Analysis Boundary Value Analysis Cause-Effect Graphing Random Usability Performance Static Analysis Inspection Control Flow Control Flow etc. State Transition

10 Black box versus white box? Integration Component Acceptance System Black box appropriate at all levels but dominates higher levels of testing White box used predominately at lower levels to compliment black box

11 Contents What is a testing technique? Black and White box testing Black box test techniques White box test techniques Error Guessing Dynamic Testing Techniques ISTQB / ISEB Foundation Exam Practice

12 Black Box test design and measurement techniques n Techniques defined in BS Equivalence partitioning -Boundary value analysis -State transition testing -Cause-effect graphing -Syntax testing -Random testing n Also defines how to specify other techniques = Yes = No Also a measurement technique?

13 Equivalence partitioning (EP) -divide (partition) the inputs, outputs, etc. into areas which are the same (equivalent) -assumption: if one value works, all will work -one from each partition better than all from one validinvalid

14 Boundary value analysis (BVA) -faults tend to lurk near boundaries -good place to look for faults -test values on both sides of boundaries validinvalid

15 Example: Loan application Customer Name Account number Loan amount requested Term of loan Monthly repayment Term: Repayment: Interest rate: Total paid back: 6 digits, 1st non-zero £500 to £ to 30 years Minimum £ chars.

16 Customer name Number of characters: invalidvalidinvalid 1 Valid characters: Any other A-Z a-z - space

17 Account number 567 invalid valid invalid number of digits: first character: invalid: zero valid: non-zero

18 Loan amount invalidvalidinvalid 499

19 Condition template

20 Design test cases Test Case DescriptionExpected OutcomeNew Tags Covered 1 2 Name:John Smith Acc no: Loan:2500 Term:3 years Name:AB Acc no: Loan:500 Term:1 year Term:3 years Repayment:79.86 Interest rate:10% Total paid: Term:1 year Repayment:44.80 Interest rate:7.5% Total paid: V1, V2, V3, V4, V B1, B3, B5,.....

21 Why do both EP and BVA? n If you do boundaries only, you have covered all the partitions as well -technically correct and may be OK if everything works correctly! -if the test fails, is the whole partition wrong, or is a boundary in the wrong place - have to test mid- partition anyway -testing only extremes may not give confidence for typical use scenarios (especially for users) -boundaries may be harder (more costly) to set up

22 Test objectives? n For a thorough approach: VP, IP, VB, IB n Under time pressure, depends on your test objective -minimal user-confidence: VP only? -maximum fault finding: VB first (plus IB?)

23 Decision tables n explore combinations of inputs, situations or events, n it is very easy to overlook specific combinations of input n start by expressing the input conditions of interest so that they are either TRUE or FALSE –record found –file exists –code valid –policy expired –account in credit –due date > current date

24 Example: student access A university computer system allows students an allocation of disc space depending on their projects. If they have used all their allotted space, they are only allowed restricted access, i.e. to delete files, not to create them. This is assuming they have logged on with a valid username and password. What are the input and output conditions?

25 List the input and output conditions list the output conditions under the input conditions list the input conditions in the first column of the table

26 Determine input combinations n add columns to the table for each unique combination of input conditions. n each entry in the table may be either T for true, F for false.

27 Rationalise input combinations n some combinations may be impossible or not of interest n some combinations may be equivalent n use a hyphen to denote dont care

28 Complete the table n determine the expected output conditions for each combination of input conditions

29 Determine test case groups n each column is at least one test case

30 Design test cases n usually one test case for each column but can be none or several

31 Rationalising outputs X T F F Y F T F Z F F T Output X Y Z n if outputs or effects are mutually exclusive, I.e. T occurs in only one place in each column, we can combine them n for example: is equivalent to:

32 Rationalising dangers n rationalising is based on assumptions n assumptions may be wrong! n assumptions should be stated n assumptions may change over time n be aware of the dangers -filling in the full table may find errors which will be missed if you rationalise -it is possible to rationalise too far

33 Extending decision tables n Entries can be more than just true or false -completing table needs to be done carefully -rationalising becomes more important n E.g.

34 Decision Tables in relation to EP and BVA Input valueOutput value FALSE TRUE

35 State Transition Testing Not covered in this course. Info available as supplement

36 Contents What is a testing technique? Black and White box testing Black box test techniques White box test techniques Error Guessing Dynamic Testing Techniques ISTQB / ISEB Foundation Exam Practice

37 White Box test design and measurement techniques n Techniques defined in BS Statement testing -Branch / Decision testing -Data flow testing -Branch condition testing -Branch condition combination testing -Modified condition decision testing -LCSAJ testing n Also defines how to specify other techniques = Yes = No Also a measurement technique?

38 Stronger structural techniques (different structural elements) More tests Using structural coverage Increasing coverage Coverage OK? What's covered? Results OK? Enough tests? Spec Software Tests More tests

39 The test coverage trap Structure exercised, insufficient function Structure exercised, insufficient function Function exercised, insufficient structure Function exercised, insufficient structure better testing % Statement % Decision % Condition Combination Structural testedness Functional testedness 100% coverage does not mean 100% tested! Coverage is not Thoroughness

40 Statement coverage n percentage of executable statements exercised by a test suite number of statements exercised total number of statements total number of statements n example: -program has 100 statements -tests exercise 87 statements -statement coverage = 87% = Typical ad hoc testing achieves % Statement coverage is normally measured by a software tool. Statement coverage is normally measured by a software tool. ?

41 Example of statement coverage Test case InputExpected output 177 As all 5 statements are covered by this test case, we have achieved 100% statement coverage read(a) IF a > 6 THEN b = a ENDIF print b Statement numbers

42 Decision coverage (Branch coverage) n percentage of decision outcomes exercised by a test suite number of decisions outcomes exercised total number of decision outcomes total number of decision outcomes n example: -program has 120 decision outcomes -tests exercise 60 decision outcomes -decision coverage = 50% Typical ad hoc testing achieves % = Decision coverage is normally measured by a software tool. Decision coverage is normally measured by a software tool. True False ?

43 Paths through code ?? 12 ?? ? ? 1234

44 Paths through code with loops ? …. for as many times as it is possible to go round the loop (this can be unlimited, i.e. infinite)

45 End Select trans... Yes Example 1 Wait for card to be inserted IF card is a valid card THEN display Enter PIN number IF PIN is valid THEN select transaction ELSE (otherwise) display PIN invalid ELSE (otherwise) reject card End Display Enter.. Yes Valid PIN? No Reject card Display PIN in.. No Wait for card to be inserted IF card is a valid card THEN Wait for card to be inserted IF card is a valid card THEN display Enter PIN number Wait for card to be inserted IF card is a valid card THEN display Enter PIN number IF PIN is valid THEN Wait for card to be inserted IF card is a valid card THEN display Enter PIN number IF PIN is valid THEN select transaction Wait for card to be inserted IF card is a valid card THEN display Enter PIN number IF PIN is valid THEN select transaction ELSE (otherwise) display PIN invalid Wait for card to be inserted IF card is a valid card THEN display Enter PIN number IF PIN is valid THEN select transaction ELSE (otherwise) display PIN invalid ELSE (otherwise) reject card Valid card? Wait

46 Read A IF A > 0 THEN IF A = 21 THEN Print Key ENDIF IF A > 0 THEN ENDIF Print Key IF A = 21 THEN ENDIF Example 2 -Cyclomatic complexity: _____ -Minimum tests to achieve: Statement coverage: ______ Statement coverage: ______ Branch coverage: _____ Branch coverage: _____ Read A Print Yes A=21 Yes No End A>0 No Read

47 Example 3 -Cyclomatic complexity: _____ -Minimum tests to achieve: Statement coverage: ______ Statement coverage: ______ Branch coverage: _____ Branch coverage: _____ Read A Read B IF A > 0 THEN IF B = 0 THEN Print No values ELSE Print B IF A > 21 THEN Print A ENDIF Print Yes Print A>21 No B=0 Print Yes Read A>0 End No

48 -Cyclomatic complexity: _____ -Minimum tests to achieve: Statement coverage: ______ Statement coverage: ______ Branch coverage: _____ Branch coverage: _____ Read A Read B IF A < 0 THEN Print A negative ELSE Print A positive ENDIF IF B < 0 THEN Print B negative ELSE Print B positive ENDIF Example 4 Read End Print No Print Yes B<0 Print No Print Yes A<0 Note: there are 4 paths

49 End No -Cyclomatic complexity: _____ -Minimum tests to achieve: Statement coverage: ______ Statement coverage: ______ Branch coverage: _____ Branch coverage: _____ Read A Read B IF A < 0 THEN Print A negative ENDIF IF B < 0 THEN Print B negative ENDIF Example 5 Read A<0 Print Yes No B<0 Print Yes

50 -Cyclomatic complexity: _____ -Minimum tests to achieve: Statement coverage: ______ Statement coverage: ______ Branch coverage: _____ Branch coverage: _____ Read A IF A < 0 THEN Print A negative ENDIF IF A > 0 THEN Print A positive ENDIF Example 6 Read Print Yes No End No A>0 Print Yes A<0

51 Contents What is a testing technique? Black and White box testing Black box test techniques White box test techniques Error Guessing Dynamic Testing Techniques ISTQB / ISEB Foundation Exam Practice

52 Non-systematic test techniques n Trial and error / Ad hoc n Error guessing / Experience-driven n User Testing n Unscripted Testing A testing approach that is only rigorous, thorough and systematic is incomplete A testing approach that is only rigorous, thorough and systematic is incomplete

53 Error-Guessing n always worth including n after systematic techniques have been used n can find some faults that systematic techniques can miss n a mopping up approach n supplements systematic techniques Not a good approach to start testing with

54 Error Guessing: deriving test cases n Consider: -past failures -intuition -experience -brain storming -What is the craziest thing we can do?

55 Summary: Key Points Test techniques are best practice: help to find faults Black Box techniques are based on behaviour White Box techniques are based on structure Error Guessing supplements systematic techniques Dynamic Testing Techniques ISTQB / ISEB Foundation Exam Practice


Download ppt "Dynamic Testing Techniques 1 Principles2 Lifecycle 4 Dynamic test techniques 3 Static testing 5 Management6 Tools Software Testing ISTQB / ISEB Foundation."

Similar presentations


Ads by Google