Presentation on theme: "Analyzing Regression Test Selection Techniques"— Presentation transcript:
1 Analyzing Regression Test Selection Techniques -presented by Xuan Lin
2 Outline Introduction Concepts and Assumptions Analysis Framework Examples TechniquesConculsion and Discussion
3 Outline Introduction Concepts and Assumptions Analysis Framework Examples TechniquesConculsion and Discussion
4 Introduction What is Regression Testing? -Everybody knows … Retest-all strategy VS. Test SelectionNotions: P,P’,S,S’,T,T’,
5 Typical Selective Retest Process 1. Select T'≤T, a set of tests to execute on P'Regression Test Selection Problem2.Test P’ with T’, to establish the correctness of P’ with respect to T’Test Suite Execution Problem3.If necessary, create T’’, a set of new functional or structural tests for P’Coverage Identification Problem4.Test P’ with T’’, to establish the correctness of P’ with respect to T’’Test Suite Execution Problem5.Create T’’’, a new test suite and test history for P’, from T, T’ and T’’Test Suite Maintenance
6 Typical Selective Retest Process 1. Select T'≤T, a set of tests to execute on P'Regression Test Selection Problem2.Test P’ with T’, to establish the correctness of P’ with respect to T’Test Suite Execution Problem3.If necessary, create T’’, a set of new functional or structural tests for P’Coverage Identification Problem4.Test P’ with T’’, to establish the correctness of P’ with respect to T’’Test Suite Execution Problem5.Create T’’’, a new test suite and test history for P’, from T, T’ and T’’Test Suite Maintenance
7 Test Selection Techniques Specification-based VS. Code-basedThree distinct goals of code-based test selection techniques-Coverage TechniquesCompare and Evaluation !!!-Minimization Techniques-Safe Techniques
8 Outline Introduction Concepts and Assumptions Analysis Framework Examples TechniquesConculsion and Discussion
9 Concepts and Assumptions Fault-realing for P’: cause P’ to fail-No Effective procedure by which to find tests in Tthat are fault-realing for P’ -Under certain conditions, a technique can select aSuperset of the set of fault-revealing for PModification-revealing: casue the outputs of P and P’ to differ.
10 Concepts and Assumptions Modification-revealing = Fault-revealing ???P-Correct-for-T Assumption: For each test t in T, when P was tested with t, P halted and produced the correct outputObsolet-Test-Identification Assumption: There is an effective procedure for determining, for each test in t, whether t is obsolete for P’.Test t is obsolete for P’ if and only if t either specifies an input to P’ that, according to S’, is invalid for P’, or t specifies an invalid input-output relation for P’
11 Concepts and Assumptions Up to now, we can find the fault-revealing test cases by:1. Run our procedure for identifying obsolete test in T.2. Remove them.3. Find the modification-revealing test cases.- In the set of non-obsolete test cases, modification-revealing=fault-revealing
12 Concepts and Assumptions ObsoleteFault-RevealingNonbsolete???Fault-RevealingModification-Revealing
13 Concepts and Assumptions Modification-traversing: a test t is modification-traversing for P and P’ if and only if it (a) executes new or modified code in P’, or (b) formerly executed code that has since been deleted
14 Concepts and Assumptions ObsoleteFault-RevealingNonbsoleteFault-RevealingModification-Revealing???Modification-Traversing
15 Concepts and Assumptions Controlled Regression Testing Assumption: when P’ is tested with t, we hold all factors that might infuence the output of P’, except for the code in P’, constant with respect to their states when we tested P with t.
16 Why We Need Define These Concepts and Assumptions? Evaluate test selection techniques in terms of their ablities to select and avoid discarding fault-revealing tests.Three classes can be used to distinguish techniques even CRTA is not satisfied.Coverage techniques may omit tests from T’ that may reveal faults in P’
17 Outline Introduction Concepts and Assumptions Analysis Framework Examples TechniquesConculsion and Discussion
20 Analysis Framework-Inclusiveness There is no algorithm to determine the inclusiveness!However…We can prove M is safe.We can prove M is not safe.We can compare techinques in terms of inclusivenessWe can experiment to approximate
22 Analysis Framework-Precision There is no algorithm to determine the precision!However…We can compare techinques in terms of precision.We can prove M is not preciseWe can show M is precise.We can experiment to compare.
23 Analysis Framework-Efficiency Time & SpaceCost of selecting T’ < the cost of running T-T’Three Factors1.preliminary phase vs. critical phase2.automatability3. calculation informatin on program modifications4.ability to handle multiple modifications
24 Analysis Framework-Generality Should function for some identifiable and practical class of programShould handle realistic program modificationsShould be independent of assumptions about testing or maintenance enviroments.Should be independent of particular program analysis toolsShould support intraprocedural or interprocedural test selection
25 Analysis Framework-Tradeoffs Precision vs. Efficiency- both safe and unsafeInclusiveness vs.Efficiency-not safeGenerality vs. Inclusiveness, Efficiency or PrecisionMultiple modication vs. Efficiency
26 Outline Introduction Concepts and Assumptions Analysis Framework Examples TechniquesConculsion and Discussion
36 Examples: Dataflow-Generity Applied to procedural programs generally.Function for all program changes except those that do not alter d-u associationSome techiques applied to intraprocedural programs while others applied to interprocedural programsIncremental approach requires incremental dataflow analysis tools.
37 Examples: Graph Walk Techniques Build CFG for P and P’Collects traces for tests with CFG edges.Performs synchronous depth-first traversals of the two graphs, selects those are not lexically identical.
38 Examples: Graph Walk Techniques-Inclusiveness  shows that for controlled regression testing, the techniques will select all modification-traversing test.So, it is safe.
39 Examples: Graph Walk Techniques-Precision Not preciseMultiply-visisted-node
43 Examples: Graph Walk Techniques-Efficiency Generally:Property not hold:
44 Examples: Graph Walk Techniques-Generality Apply to procedural languages generallyAll type of modificationsBoth interprocedure and intraprocedureNo assumption on test suite or coverageRequire tools for constructing dataflow and tools for dataflow analysis
45 Examples: Path Analysis Takes set of program paths in P’ expressed as an algebraic expressionManipulates the expression to get a set of cycle-free exemplar paths.Compare such paths in P with P’Tests that traverse modified exemplar paths will be selected
46 Examples: Path Analysis-Inclusiveness Selects only modified paths and omits the cancel and new paths.Not safe.
47 Examples: Path Analysis-Precision It will select all the test cases that are modification-traversing and execute modified exemplar paths.
49 Examples: Path Analysis-Efficiency Exponential in |P| and |P’|
50 Examples: Path Analysis-Generality Assumption: low-level program designs are depicted by language-independent algebraic representations.Does not handle test cases for additions or deletings of code.Does not require any coverage criterian or test generation technique.Require tool for collecting traces at the statement level.
51 ConclusionsFramework for evaluating regression test selection technique that classifies techniques in terms of inclusiveness, precision, efficiency, and generality.Several test selection techiques are evaluated
52 ReferenceG.Rotherel. Efficient, Effective Regression Testing Using Safe TestSelection Techniques.
Your consent to our cookies if you continue to use this website.