1. Review for Final – Spring 2019
Jeff Offutt
SWE 437
Software Testing & Maintenance

2. Parameters 10:30 – 1:15 Same room 14 May, next next Tuesday
LAST quiz retake 6-May, 1:00-3:00pm (in Engineering 4201)
Closed book / closed notes / closed neighbor
One sheet of definitions allowed
Hand-written, not typed, double-sided is okay
Comprehensive
You may take breaks (not all together)
Spread out
Break up tables, max 2 per table
437 - Final Review
© Jeff Offutt, 2019

3. Summary: Software Maintenance
Week 2: Overview & evolution
Two essays
Overview of software maintenance and evolution
How the web resuscitated evolutionary design
Week 3: Coding and design for change
Coding for change
Designing for change
437 - Final Review
© Jeff Offutt, 2019

4. Summary: Part 1—Foundations
Why Test Software? (Ch 1)
Fault, error, failure, program state
Test maturity levels
Cost of late testing
Model-Driven Test Design (Ch 2)
RIPR model
Other terms
Test Automation (Ch 3)
Junit
Components of a test case
Developer Testing (Ch 4)
Agile and TDD
Criteria-Based Test Design (Ch 5)
Test requirements and criteria
437 - Final Review
© Jeff Offutt, 2019

5. Cost of Late Testing 60 50 40 30 20 10
Assume $1000 unit cost, per fault, 100 faults
$250K
$360K
Fault origin (%)
Fault detection (%)
Unit cost (X)
$20K
$100K
$13K
$6K
28-Oct-2010, at GTAC, added the animation to demonstrate increasing the number of faults found early, thereby decreasing the number of faults found late, and finally saving money. Lots of it! This animation is fairly complicated … must practice first!!
Requirements
Design
Prog / Unit Test
Integration Test
System Test
Post-Deployment
Software Engineering Institute; Carnegie Mellon University; Handbook CMU/SEI-96-HB-002
437 - Final Review
© Jeff Offutt, 2019

6. Incorrect Program State
RIPR Model
Reachability
Infection
Propagation
Revealability
Test
Final Program State
Observed Final Program State
Reaches
Fault
Incorrect Final State
Infects
Incorrect Program State
The failures can be revealed when the observed final program state has overlap with the incorrect final program state.
The question is: should testers check the entire program state? How to observe the incorrect program state in a cost-effective manner.
Getting the overlap as big as possible and use the cost as small as possible.
Propagates
Reveals
Test Oracles
437 - Final Review
© Jeff Offutt, 2019

7. Model-Driven Test Design–Activities
refined requirements / test specs
Test Design
model / structure
test requirements
DESIGN
ABSTRACTION
LEVEL
IMPLEMENTATION
ABSTRACTION
LEVEL
software artifact
input values
Test Automation
Test Evaluation
pass / fail
test results
Test Execution
test scripts
test cases
437 - Final Review
© Jeff Offutt, 2019

8. Criteria Based on Structures
Structures : Four ways to model software
Input Domain Characterization (sets)
A: {0, 1, >1}
B: {600, 700, 800}
C: {swe, cs, isa, infs}
Graphs
Logical Expressions
(not X or not Y) and A and B
if (x > y)
z = x - y;
else
z = 2 * x;
Syntactic Structures (grammars)
437 - Final Review
© Jeff Offutt, 2019

9. Summary: Part 2 Coverage Criteria (1 of 4)
Input Space Partitioning – Ch 6
Input Domain Modeling (6.1)
Combination Strategies Criteria (6.2)
Skipped 6.3 & 6.4 – not on final exam
Be sure to review the coverage definitions in 6.2
Know how to construct partitions and blocks
Know how to choose values
437 - Final Review
© Jeff Offutt, 2019

10. ISP Coverage Criteria Subsumption
Each Choice Coverage
ECC
All Combinations Coverage
ACoC
T-Wise Coverage
TWC
Multiple Base Choice Coverage
MBCC
Pair-Wise Coverage
PWC
Base Choice Coverage
BCC
437 - Final Review
© Jeff Offutt, 2019

11. Summary: Part 2 Coverage Criteria (2 of 4)
Graph Coverage – Ch 7
Overview (7.1)
Graph Coverage Criteria (7.2)
Graph Coverage for Source Code (7.3)
Skipped 7.4, 7.5, 7.6 – not on final exam
Be sure to review the definitions in 7.2
Know how to generate graphs for various artifacts and apply the criteria to them
Know how to choose values (EC, EPC, PPC, …)
We skipped data flow—not on the final
437 - Final Review
© Jeff Offutt, 2019

12. Graph Coverage Criteria Subsumption
Simple Round Trip Coverage
SRTC
Node Coverage NC
Edge Coverage EC
Edge-Pair Coverage
EPC
Prime Path Coverage
PPC
Complete Path Coverage
CPC
Complete Round Trip Coverage
CRTC
All-DU-Paths Coverage
ADUP
All-uses Coverage
AUC
All-defs Coverage
ADC
437 - Final Review
© Jeff Offutt, 2019

13. Summary: Part 2 Coverage Criteria (3 of 4)
Logic Coverage – Ch 8
Logic Coverage Criteria (Active) (8.1)
Structural Logic Coverage of Programs (8.3)
Skipped 8.2, – not on final exam
Be sure to review the coverage definitions in 8.1
Know how to apply the coverage criteria to programs, specs, and FSMs
Know how to choose values for CACC
437 - Final Review
© Jeff Offutt, 2019

14. Logic Criteria Subsumption
Clause Coverage CC
Predicate Coverage PC
Combinatorial Clause Coverage
COC
Restricted Active Clause Coverage
RACC
Restricted Inactive Clause Coverage
RICC
General Active Clause Coverage
GACC
Correlated Active Clause Coverage
CACC
General Inactive Clause Coverage
GICC
437 - Final Review
© Jeff Offutt, 2019

15. Summary: Part 2 Coverage Criteria (4 of 4)
Syntax-based Testing – Ch 9
Syntax-based Coverage Criteria (9.1)
Input Space Grammars (9.5)
Skipped 9.2, 9.3, 9.4 – not on final exam
Understand how to derive tests from BNF
437 - Final Review
© Jeff Offutt, 2019

16. Representing Input Domains
goal domain
specified domain
This region is a rich source of software errors …
implemented domain
437 - Final Review
© Jeff Offutt, 2019

17. Preparing & Format Re-read the text and handouts
Study the homework assignments
Redo the quizzes
Try exercises in text
Get a good night’s sleep
Possibly different kinds of questions
Mostly problem solving—similar to quizzes, perhaps bigger
Short answer
Maybe a few T / F, M / C, single answer questions
437 - Final Review
© Jeff Offutt, 2019

18. Posting Grades I will post on Blackboard: Last assignment grades
Last quiz grades
Average quiz grades
Average assignment grades
Participation grades
Final exam score
437 - Final Review
© Jeff Offutt, 2019

19. Final Grades
15% participation, 30% quizzes, 20% assignments, 35% final exam
Unbiased grading – compute and sort:
>= 98 : A – Probably only one A+ per class
> 93 : A
> 90 : A-
> 86 : B+
> 80 : B
> 75 : B-
> 65 : C
I will “slide the line” down to an appropriate break point
437 - Final Review
© Jeff Offutt, 2019

20. What I Want You to Take Away
You in 5 years Me
Forgot everything about testing and high quality software
Remember testing, but develop software the “same old way”
Use these concepts to hand-design more effective tests for your software
Apply automation and criteria when possible
Convince your company to develop a quality mindset
437 - Final Review
© Jeff Offutt, 2019