1. Week 2 Quality Assurance Quality Assurance in Context
Software Qualit Engineering

2. What is SQA
SQA is a well defined, repeatable process that is integrated with project management and the software development lifecycles to review internal control mechanisms and assure adherence to software standards and procedures. The objective of the process is to assure conformance to requirements, reduce risk, assess internal controls and improve quality while conforming to the stated schedule and budget constraints. (Owens and Khazanchi)

3. More definitions (Owens and Khazanchi)

4. What is QA? QA as Dealing with Defect Defect Prevention
Error blocking
Error source removal
Defect Detection and Removal
Inspection
Testing etc.
Defect Containment
Failure prevention
Failure containment.
Tian defines QA in 3 contexts.. i.e the process to deal with defects, prevent, eliminate, reduce or contain various specific problems associated with different aspects of defects.

5. QA: quality assurance Focus on correctness aspect of Q
QA as dealing with defects
post-release: impact on consumers
pre-release: what producer can do
what: testing & many others
when: earlier ones desirable (lower cost)
how =>classification as follows
Defect prevention and reduction activities directly deal with the competing processes of defect injection and removal.
After release the failure observed and problems reported by customers and users also need to be fixed, which in turn could lead to reduced defect and improved product quality. However, one shouldn’t rely on these post release activities and give up pre release activities. Coz error correction after release can be many time more expensive. Can also leads to damage to the reputation of SW development organisation. Pre release activities may be complimented with ‘Controlled field testing’ commonly referred to as ‘beta testing’, and similar techniques.
Defect containment activities aim at minimizing the –ve impact of these remaining faults during operational use after product release. Defect containment activities involve a lot of duplication and redundancy and requires more effort to design and implement related features. Therefore, they are limited to situation where in field failures are associated with substantial damage. .

6. Error/Fault/Failure & QA
Removal of faults (pre: detection)
inspection: faults discovered/removed
testing: failures trace back to faults
Failure prevention and containment:
local failure !=> global failure
via dynamic measures to tolerate faults
failure impact =>safety assurance

7. QA Classification

8. Defect Prevention Overview
Error blocking
error: missing/incorrect actions
direct intervention to block errors
fault injections prevented
rely on technology/tools/etc.
Error source removal
root cause analysis
identify error sources
removal through education/training/etc.
Focus on Product and domain specific knowledge, SW dev knowledge and expertise, developmental methodologies, development process knowledge
Systematic defect prevention via process improvement.
Education and training to remove human misconceptions. For error source elimination one can focus on

9. Formal Methods Motivation (formal methods =>fault absent)
fault present:
revealed through testing/inspection/etc.
fault absent: formally verify.
(formal methods =>fault absent)
Basic ideas
behavior formally specified:
pre/post conditions, or
as mathematical functions.
verify “correctness":
intermediate states/steps,
axioms and compositional rules.
Approaches: axiomatic/functional/etc.

10. Defect Reduction:Inspection Overview
Artifacts (code/design/test-cases/etc.) from (req./design/coding/testing/etc) phases.
Informal reviews:
self conducted reviews.
independent reviews.
orthogonality of views desirable.
Formal inspections:
Fagan inspection and variations.
process and structure.
individual vs. group inspections.
what/how to check: techniques .

11. Testing Overview Product/Process characteristics:
scale/order:
unit, component, system, : : :
who: self, independent, 3rd party
What to check:
external specifications (black-box)
internal implementation (white/clear-box)
Criteria: when to stop?
coverage of specs/structures.
Reliability: usage-based testing

12. Defect Containment: Fault Tolerance Overview
Idea originated from hardware systems.
Focus is on reliability, availability and dependability.
Motivation
fault present but removal infeasible/impractical
(fault tolerance ) contain defects.
FT techniques: break fault-failure link
Recovery blocks: rollback and redo
NVP: N-version programming
fault blocked/out-voted
RECOVERY blocks: like databases.
N-version programming: Many parallel programs implementing same functionality but using different logic, algorithm..

13. Safety Assurance Overview
Extending FT idea for safety:
fault tolerance to failure “tolerance"
Safety related concepts:
safety: accident free
accident: failure with severe consequences
hazard: precondition to accident
Safety assurance:
hazard analysis
hazard elimination/reduction/control
damage control
Hazard elimination through substitution, simplification, decoupling etc.
Hazard reduction through design for controllability.. For example pressure release valve..
Hazard control through reducing exposure, isolation and containment
Damage control: through escape routes, safe abandonment of product and materials, and devices for limiting physical damage to equipments

14. QA in Context QA and the overall development context
defect handling/resolution
activities in process
alternative perspectives
verification/validation (V&V) view
Now defects are found through various activities outlined earlier. Now the question is what we need do to handle these defect. Handling defect is called defect resolution

15. Defect handling/resolution
status and tracking
Logging
Monitoring
causal (root-cause) analysis
resolution: defect removal/etc.
Many stakeholders: testers, developer, managers, client (except unit testing)
Part of project management activity
improvement: break causal chain

16. Defect Measurement and Analysis
parallel to defect handling
where injected/found?
type/severity/impact?
more detailed classification possible?
consistent interpretation
timely defect reporting
Defect analyses
as follow up to defect handling.
data and historical baselines
goal: assessment/prediction/improvement
causal/risk/reliability/etc. analyses

17. Defect Handling in Different QA Activities.
Among 3 classes of QA activities:
defect detection and removal activities are more closely associated with defect handling.
Various defect prevention activities do not directly deal with defect or discovered faults.
On defect containment the focus is not on fault finding or removal.
So defect handling is not closely associated with prevention and containment activities

18. QA in Software Processes
Development process components:
requirement, specification, design, coding, testing, release.
Process variations:
waterfall development process
iterative development process
spiral development process
lightweight/agile development processes such as XP (extreme programming)
maintenance process too
mixed/synthesized/customized processes
QA important in all processes

19. QA in Waterfall Process

20. QA in Waterfall Process: QA throughout process
defect prevention in early phases
focused defect removal in testing phase
defect containment in late phases
phase transitions: inspection/review/etc.

21. QA in Software Processes
Process variations (: waterfall) and QA:
iterative: QA in iterations/increments
spiral: QA and risk management
How we can do it in agile processes??
mixed/synthesized: case specific
more evenly distributed QA activities
QA in maintenance processes:
focus on defect handling;
some defect containment activities for critical or highly-dependable systems;
data for future QA activities
QA scattered throughout all processes
test-driven development, pair programming, pair inspection, involvement of customer.

22. V&V Core QA activities grouped into V&V.
Validation: w.r.t. requirement (what?)
appropriate-for-use “ doing right thing"?
scenario and usage inspection/testing;
system/acceptance testing;
beta testing and operational support.
Verification: w.r.t. specification/design (how?)
Correct “doing things right"
design as specification for components;
structural and functional testing;
inspections and formal verification.

23. V&V in Software Process

24. V&V vs DC View Two views of QA: Mapping between V&V and DC view:
V&V view
DC (defect-centered) view
Interconnected: mapping possible?
Mapping between V&V and DC view:
V&V after commitment
(defect injected already)
defect removal & containment focus
Verification: more internal focus
Validation: more external focus
In V-model: closer to user (near top) or

25. DC-V&V Mapping

26. Time for a break

27. Software Quality Engineering
SQE: Software Quality Engineering
Key SQE Activities
SQE in Software Process
Since there are diferent expectations by customer and developer, we need to move beyond just focusing on QA activities to SQE by managing these quality expectation as engineering problem. Goal is to meet or exceed these quality expectations through the selection and execution of selected appropriate QA activities while minimizing the cost and risk under the project constraints.

28. QA to SQE QA activities need additional support:
Planning and goal setting
Management:
when to stop?
adjustment and improvement, etc.
all based on assessments/predictions
Assessment of quality/reliability/etc.:
Data collection needed
Analysis and modeling
Providing feedback for management
QA + above
=> software quality engineering (SQE)

29. SQE Process

30. SQE and QIP QIP (quality improvement paradigm):
Step 1: understand baseline
Step 2: change then assess impact
Step 3: package for improvement
SQE as expanding QA to include QIP ideas.

31. Pre-QA Planning Pre-QA planning: Setting quality goals: Quality goal
Overall QA strategy:
QA activities to perform?
measurement/feedback planning
Setting quality goals:
Identify quality views/attributes
Select direct quality measurements
Assess quality expectations vs. cost

32. Setting Quality Goals Identify quality views/attributes
customer/user expectations,
market condition,
product type, etc.
Select direct quality measurements
direct: reliability
defect-based measurement
other measurements
Assess quality expectations vs. cost
cost-of-quality/defect studies

33. Forming QA Strategy QA activity planning
evaluate individual QA alternatives
strength/weakness/cost/applicability/etc.
match against goals
integration/cost considerations
Measurement/feedback planning:
define measurements (defect & others)
planning to collect data
preliminary choices of models/analyses
feedback & followup mechanisms, etc.

34. Analysis and Feedback Measurement: handling process
defect measurement as part of defect
handling process
other data and historical baselines
Analyses: quality/other models
input: above data
output/goal: feedback and follow up
focus on defect/risk/reliability analyses
Feedback and followup:
frequent feedback: assessments/predictions
possible improvement areas
project management and improvement

35. SQE in Software Processes
SQE activities development activities:
quality planning product planning
QA activities development activities
analysis/feedback project management
Fitting SQE in software processes:
different start/end time
different sets of activities, sub-activities,
and focuses
in waterfall process: more staged
(planning, execution, analysis/feedback)
in other processes:
more iterative or other variations

36. SQE Effort Profile QE activity/effort distribution/dynamics:
different focus in different phases
different levels (qualitatively)
different build-up/wind-down patterns
impact of product release deadline (deadline-driven activities)
planning: front heavy
QA: activity mix (early vs. late; peak variability? deadline?)
analysis/feedback: tail heavy (often deadline-driven or decision-driven)

37. SQE Effort in Waterfall Process

38. Enough for this week Reading Tian And lets discuss your past papers..
Chapter 3, 4 and 5.
And lets discuss your past papers..