1. Regression Methodology Einat Ravid

2. Regression Testing - Definition  The selective retesting of a hardware system that has been modified to ensure that any bugs have been fixed and that no other previously working functions have failed as a result of the reparations and that newly added features have not created problems with previous versions of the hardware. bugs  Also referred to as verification testing, regression testing is initiated after a designer has attempted to fix a recognized problem or has added source code to a program that may have inadvertently introduced errors. It is a quality control measure to ensure that the newly modified code still complies with its specified requirements and that unmodified code has not been affected by the maintenance activity. source codeprogramcodesource codeprogramcode

3. Regression Strategy  Retest all – rerun every test. This may cover all the possibilities but may not be realistic to project time.  Regression test selection – trade off between time and fault detection. pros: Test fixed bugs immediately. Test fixed bugs immediately. Can check side effects of fix (depends on the how broad is the test). Can check side effects of fix (depends on the how broad is the test). Reduce time of rerunning the tests. Reduce time of rerunning the tests.Cons: Require development time. Require development time.

4. Regression Selection Methods  Minimization Method Select a minimum number of tests from entire regression that covers the changes only. Can be tested with coverage on the changes.  Dataflow Method Run tests that activate data flows that the changes affect.  Safe Method Define a set of safety conditions and make sure the tests cover them. I.E – test per changed/ added/ removed statement in the code.  Random Method Run random tests.  Retests All Method

5. Research Results  Fault Detection Minimization Method discovered the least number of faults Minimization Method discovered the least number of faults Safe and retest all were very effective at fault detection Safe and retest all were very effective at fault detection Random method was more effective as the number of test increased. But after a certain number of tests the increase was considerably small. Random method was more effective as the number of test increased. But after a certain number of tests the increase was considerably small.  Cost Benefit ( number of tests V.S. Fault detection) Random Method was very effective. Random Method was very effective. Safe Method had 100% fault detection but reduction was very different for different programs – no rule of thumb. Safe Method had 100% fault detection but reduction was very different for different programs – no rule of thumb. Dataflow Method – was effective but not safe, may still miss faults. Dataflow Method – was effective but not safe, may still miss faults. Minimization Method – few tests but low fault detection percentage. Minimization Method – few tests but low fault detection percentage.

6. What To Consider When Building a Regression?  How to reduce testing cost?  What is the best way to detect faults?  What is the trade off between test selection and fault detection?  How does design type and machine resources effect test selection?  What coverage do we want to achieve?  How easy it would be to maintain this subset of test?  How do we identify obsolete tests?

7. Suggestions  Add a regression test for each bug fix.  Use tests that consistently pass.  Focus on functional issues.  Add tests with boundary conditions.  Make sure there are no duplications.