Presentation is loading. Please wait.

Presentation is loading. Please wait.

Testing (1)  Principles of testing  The testing process  Methods used in testing & debugging Let’s look at :

Similar presentations


Presentation on theme: "Testing (1)  Principles of testing  The testing process  Methods used in testing & debugging Let’s look at :"— Presentation transcript:

1 Testing (1)  Principles of testing  The testing process  Methods used in testing & debugging Let’s look at :

2 Testing (2) DesignAnalysis Code Test

3 Testing (3) Narrow View (unit level) Run a program component with the purpose of finding errors prior to delivery of product Broad View The process of ensuring that the software conforms to its specification and meets user requirements Validation A good test is one that has a high probability of finding a new error A successful test is one that discovers a new error Verification

4 Testing (4) Testing Principles Pareto Principle – 80% of errors occur in 20% of classes Tests should be planned long before testing begins – test planning should really be done with program design All tests should be traceable to user requirements Testing should begin “in the small” and proceed towards testing “in the large” Exhaustive (complete) testing is usually not possible To be effective, testing should be conducted by an independent 3 rd party 20% of the citizens own 80% of the wealth 20% of the land grows 80% of the crops

5 Testing (5) Who tests the software? Developer? Independent tester? Understands the system but will test It “gently” and be motivated by need to deliver the product Needs to learn about the system but will attempt to break it/crash it and is driven by quality

6 Testing (6) Features of Testable Software Operability - The better it works the more easily it can be tested (Bugs are easier to find in software which at least executes) Observability - The results of each test should be easy to observe Controlability - If we can control the execution of a separate parts of the software then it will be easier to set up specific test cases and perhaps to automate testing Simplicity - Simple system architectures are easier to test than complex ones Stability - Changes disrupt test planning and test cases Test case – see next slide

7 Testing (7) A test case is a controlled experiment that tests all or part of the system with defined test data Test process - However, this is very often not the case and test design often done badly and in an ad hoc manner Constraints – with a minimum of time & effort Criteria – in a complete manner Objective – to uncover errors “Make it up as we go along” Remember that “bugs lurk in corners and congregate at boundaries!” Why?

8 Testing (8) Two nested loops containing four if.. then.. else statements. Each loop can execute up to 20 times. There are 10^14 possible paths if we count each single iteration. If we execute one test per millisecond, it would take 3170 years to test this program! Exhaustive Testing (not feasible)

9 Testing (9) Selective Testing (feasible) Test a carefully selected execution path. Note that it cannot be comprehensive

10 Testing (10) Testing Methods Black Box testing – examines fundamental interface without looking at internal processing. In short, is the program’s output correct for a given set of inputs? White (Glass) Box testing – examines in detail the internal processing done by software components Debugging – fixes the errors identified during testing ? ? ? ? ? ?

11 Testing (11) The objective in white box testing is to ensure that all statements and conditions have been executed at least once. Derive test cases that: 1. Exercise all independent execution paths 2. Exercise all logical decisions on both the true and the false sides 3. Execute all loops at their boundaries and within operational bounds 4. Exercise all internal data structures to ensure they are valid and that read/write accesses are as they should be

12 Testing (12) Why cover all paths ?  Logic errors and incorrect assumptions are inversely proportional to the probability that a program path will be executed  We often may be inclined to believe that a logical path is not likely to be executed when, in fact, it may be frequently executed  Typographical errors occur at random and so it is likely that untested paths will contain some

13 Testing (13) Basis Path Testing - Provides a measure of the logical complexity of a method or code component and provides a guide for defining a basis set of execution paths It uses flow graph notation to represent the flow of control where nodes represent processing and arrows represent control flow. While If Sequence

14 Testing (14) Flow Graphs – Compound Conditions Separate nodes are created for each arm of a compound condition (e.g. a and b are separate nodes in the condition if(a && b)) Example: If(a || b) { x(); } else { y() } Z() b a z y x

15 Testing (15) Cyclomatic Complexity is a software metric that gives a quantitative measure of the logical complexity of a program Find the cyclomatic complexity, V(G), of a flow graph G: - Number of simple predicates (decisions) + 1 or - V(G) = E – N+2 (where E are edges and N are nodes or - Number of enclosed areas + 1 In this case, V(G) = 4

16 Testing (16) Analysis has shown that the number of errors and the maintenance Increases significantly for modules with a V(G) > 10. Another use of cyclomatic complexity is that V(G) identifies the number of independent paths though a program that need to be tested

17 Testing (17) Basis Path testing V(G) is the number of linearly independent paths through the program (each has at least one edge not covered by any other path)  Path 1:  Path 2:  Path 3:  Path 4: Test design must prepare test cases that will force the execution of each path in the basis set

18 Testing (18) Basis Path testing Example: Draw the flow graph, calculate the cyclomatic complexity, and list the basis paths using the following C++ piece of code: While(value[i]!= && totinputs < 100 { totinputs++ if(value[i] >= min && value[i] <= max) { totvalid++ sum = sum +value[i] } i++ } While(value[i]!= && totinputs < 100 { totinputs++ if(value[i] >= min && value[i] <= max) { totvalid++ sum = sum +value[i] } i++ }

19 Testing (19) Basis Path testing V(G) = number of enclosed areas + 1 = 5 V(G) = number of simple predicates + 1 = 5 V(G) = E – N + 2 = 10 – = 5 Basis Paths to be Tested are : (value[i] = ) (value[i] = 0, totinputs=100)

20 Testing (20) Other White Box Methods Condition testing: exercises the logical (boolean) conditions in a program Data Flow testing: selects test paths according to the location of the definition and use of variables in a program Loop testing: focuses on the correctness of loop structures

21 Testing (21) Loop Testing Nested Loops Simple Loop Concatenated Loops Unstructured Loops

22 Testing (22) Loop Testing Test Cases for simple Loops 2. Only one pass through the loop 1. Skip the loop entirely 3. Two passes through the loop 4. m passes through the loop (m

23 Testing (23) Loop Testing Testing Nested Loops 2. Test the min, min+1, typical, max-1 and max for the inner loops 1. Start at the innermost loop. Set all the outer loops to their minimum iteration parameter variables (i.e. loop control variable) 3. Move out one loop and set it up as in step 2, holding all other loops at typical values. Continue this until the outermost loop has been tested Testing Concatenated Loops If the loops are independent of one another then treat each as a simple loop, otherwise treat as nested loops Testing Unstructured Loops Don’t bother!Re-design!

24 Testing (24) Black Box Testing Requirements Events Inputs Outputs Black Box Testing is complementary to white box testing. Decide on what external conditions (i.e. inputs, requirements, events) that fully exercise all functional requirements (i.e. test all functions encoded in software)

25 Testing (25) Black Box Testing Black Box Strengths Attempts to find errors in the following categories:  Incorrect or missing functions  Interface errors  Errors in data structures or external database access  Behaviour or performance errors  Initialisation or termination errors Black box testing is performed during later stages of testing

26 Testing (26) Black Box Methods Equivalence Partitioning - Divide input domain into classes of data - Each test case then uncovers whole classes of errors - Examples: valid data (user supplied commands, files names, graphical data (e.g. mouse selections), invalid data (data outside bounds, physically impossible data e.g. negative value when only positive possible), valid data supplied in an invalid situation (e.g. an order_quantityof 5,000 might be valid for an item named chalk but would most likely be invalid for an item called projector.

27 Testing (27) Black Box Methods Boundary Value Analysis - More errors tend to occur at the boundaries of the input domain - Select test cases that exercise bounding values - Examples: an input condition specifies a valid range of input values bounded by values a and b. Test cases should be designed with values a and band just above and below a and b.

28 Testing (28) Debugging >.. > Debugging Suspected causes New test cases Regression tests Corrections Identified causes Test casesExecution of cases Testing is a structured process that identifies an error’s “symptoms” Debugging is a diagnostic process that identifies an error’s “cause” Testing for unintended knock-on effects

29 Testing (29) Debugging Effort Time required to correct the error and conduct regression tests Time required to diagnose the symptom and determine the cause Regression testing means re-execution of a subset of test cases to ensure that changes made to correct errors do not have unintended side effects

30 Testing (30) Bugs- Symptoms & Causes Symptom Cause  Symptom and cause may be geographically separated  Symptom may disappear when another problem is fixed  Cause may be due to a combination of non-errors  Cause may be due to a system or compiler error  Cause may be due to an assumption that everyone believes  Symptom may be intermittent

31 Testing (31) Not All Bugs are Equal ! damage Bug type annoying mild infectious catastrophic extreme serious disturbing Bug Categories: function-related bugs, data bugs, coding bugs, design bugs, documentation bugs, standards violations, etc

32 Testing (32) Debugging Techniques Brute Force: Backtracking: Cause Elimination: - Use when all else fails - Try memory dumps and run-time traces - Search through mass of information which may lead to source of error - Can work in small programs where there are few backward paths - Trace the source code backwards from the error to the source - Create a set of “cause – hypothesis” for each error - Use error data (program output) or further tests to prove or disprove these hypotheses Some people seem to have intuitive skill at debugging and can find the source of errors quickly

33 Testing (33) Debugging Tips Don’t immediately dive into the code, think about the symptom you are seeing Use tools (e.g. dynamic debuggers) to gain further insight about the error problem If you can’t solve the problem and locate the source of the error, get help from someone Ask these questions before attempting to “fix” the bug: 1.Is the cause of the bug reproduced in another part of the program, i.e. are there duplicates of the error in the code? 2. Could another bug be introduced by the fix? 3. What could have been done to fix the bug at a design or coding-plan level in the first place? Be absolutely sure to conduct regression tests when you do “fix” the bug


Download ppt "Testing (1)  Principles of testing  The testing process  Methods used in testing & debugging Let’s look at :"

Similar presentations


Ads by Google