1. Software Quality Assurance

2. Recap What is SQA? What is quality? Different Terminologies
Cost of correcting an error
Elements of SQA

3. SQA Goals, Attributes and Metrics
Number of ambiguous modifiers (e.g., many, large, human-friendly)
Number of TBAs, TBDs
Number of sections/subsections
Number of changes per requirement
Time (by activity) when change is requested
Number of requirements not traceable to design/code
Number of UML models
Number of descriptive pages per model
Number of UML errors
Existence of architectural model
Number of components that trace to architectural model
Complexity of procedural design
Layout appropriateness
Number of patterns used
Goals
Requirement quality
Design quality
Attributes
Ambiguity
Completeness
Understandability
Volatility
Traceability
Model clarity
Architectural integrity
Component completeness
Interface complexity
Patterns

4. SQA Goals, Attributes and Metrics
Cyclomatic complexity
Design factors
Percent internal comments
Variable naming conventions
Percent reused components
Readability index
Staff hour percentage per activity
Actual vs. budgeted completion time
Review metrics
Number of errors found and criticality
Effort required to correct an error
Origin of error
Goals
Code quality
QC effectiveness
Attributes
Complexity
Maintainability
Understandability
Reusability
Documentation
Resource allocation
Completion rate
Review effectiveness
Testing effectiveness

5. SQA plan Management section Documentation section
describes the place of SQA in the structure of the organization
Documentation section
describes each work product produced as part of the software process
Standards, practices, and conventions section
lists all applicable standards/practices applied during the software process and any metrics to be collected as part of the software engineering work
Reviews and audits section
provides an overview of the approach used in the reviews and audits to be conducted during the project
Test section
references the test plan and procedure document and defines test record keeping requirements
Problem reporting and corrective action section
defines procedures for reporting, tracking, and resolving errors or defects, identifies organizational responsibilities for these activities
Other
tools, SQA methods, change control, record keeping, training, and risk management

6. Statistical SQA
Information about software defects is collected and categorized
An attempt is made to trace each defect to its underlying cause
Isolate the vital few causes of the major source of all errors
Then move to correct the problems that have caused the defects

7. Statistical SQA – Categories of errors
Incomplete or erroneous specification (IES)
Misinterpretation of customer comm (MCC)
Intentional deviation from specification (IDS)
Violation of programming standards (VPS)
Error in data representation (EDR)
Inconsistent module interface (IMI)
Error in design logic (EDL)
Incomplete or erroneous testing (IET)
Inaccurate or incomplete documentation (IID)
Error in programming lang. Translation (PLT)
Ambiguous or inconsistent human-computer interface (HCI)
Miscellaneous (MIS)
Most often IES, MCC and EDR are the vital few causes for majority of errors.

8. Identifying the vital few

9. Statistical SQA

10. Example

11. Statistical SQA – Six Sigma
Most widely used strategy for statistical SQA
Three core steps
Define customer requirements, deliverables and project goals via well-defined methods of customer communication
Measure the existing process and its output to determine quality
Analyze defect metrics and determine the vital few causes
If an existing software process is in place, but improvement is required six sigma suggests
Improve the process by eliminating the root causes of defects
Control the process to ensure that future work does not reintroduce the cases of defects
If an organization is developing a software process, the core steps are augmented
Design the process to (1) avoid the root causes of defects and (2) to meet customer requirements
Verify that the process model will, in fact, avoid defects and meet customer requirements

12. Reviews To uncover errors/defects
To uncover errors in function, logic, or implementation for any representation of the software
To verify that software meets its requirements
To ensure that software representation meets predefined standards
To achieve software development in a uniform manner
To make projects more manageable

13. Review Roles Presenter (designer/producer).
Coordinator (not person who hires/fires).
Recorder
records events of meeting
builds paper trail
Reviewers
maintenance oracle
standards bearer
user representative
others

14. Formal Technical Reviews
Involves 3 to 5 people (including reviewers)
Advance preparation (no more than 2 hours per person) required
Duration of review meeting should be less than 2 hours
Focus of review is on a discrete work product
Review leader organizes the review meeting at the producer's request.
Reviewers ask questions that enable the producer to discover his or her own error (the product is under review not the producer)
Producer of the work product walks the reviewers through the product
Recorder writes down any significant issues raised during the review
Reviewers decide to accept or reject the work product and whether to require additional reviews of product or not.

15. Formality and Timing Formal review presentations
resemble conference presentations.
Informal presentations
less detailed, but equally correct.
Early
tend to be informal
may not have enough information
Later
tend to be more formal
Feedback may come too late to avoid rework

16. Formality and Timing Analysis is complete. Design is complete.
After first compilation.
After first test run.
After all test runs.
Any time you complete an activity that produce a complete work product.

17. Why do peer reviews? To improve quality.
Catches 80% of all errors if done properly.
Catches both coding errors and design errors.
Enforce the spirit of any organization standards.
Training and insurance.

18. Review Guidelines.. Review the product, not producer
Set an agenda and maintain it
Limit the debate
Enunciate problem areas, not to solve every problem noted
Take written notes
Allocate resources and time schedule for FTR’s
Use standards to avoid style disagreements.
Let the coordinator run the meeting and maintain order.
Limit the number of participants and insist upon advance preparation
Develop a checklist for each work product to be reviewed
Training for all reviewer’s
Reviewing earlier reviews Keep it short (< 30 minutes).
Don’t schedule two in a row.
Don’t review product fragments.

19. Effectiveness of review  Defect Amplification and Removal
Used to illustrate the generation and detection of errors during design and code generation
Errors passed through
Amplified errors 1:x
Newly identified errors
Percent efficiency for error detection
Errors from previous steps
Errors passed to next step
Development step
Defects
Detection

20. Effectiveness of review  Defect Amplification and Removal
No reviews
With reviews

21. Effectiveness of review  Defect Amplification and Removal

22. Review metrics and their use
Many metrics can be defined for technical reviews
The following can be calculated for each review conducted:
Preparation effort (Ep)
Assessment effort (Ea)
Rework effort (Er)
Work product size (WPS)
Minor errors found (Errminor)
Major errors found (Errmajor)

23. Summary SQA goal, attributes and metrics SQA plan
Formal Technical Review (FTR)
Statistical SQA
Six Sigma
Identifying vital few causes
Review efficiency