1. Software Quality : The Elusive Target
Kitchenham B. & Pfleeger S. L.
January, IEEE Software
Zhenyu Wu
Software Quality
CS5391 Spring 2003

2. Introduction Questions:
What do you really mean by software quality (SQ)?
Is your definition adequate?
How to evaluate your product’s SQ ?
How to measure SQ?
How to achieve SQ?
Software Quality
CS5391 Spring 2003

3. Meaning of SQ
A good definition should let us measure quality in a meaningful way
Measurements let us know
if techniques improve quality
how process quality affects product quality
how quality methods affect product quality during use
Is the investment in SQ methods profitable?
Software Quality
CS5391 Spring 2003

4. Meaning of SQ… Depends on how you approach software quality
Different groups involved have different views
Product-based
Process-based
People believe quality is important and can be improved
Software Quality
CS5391 Spring 2003

5. Views of SQ
SQ can be perceived in various domains (philosophy, economics, marketing and operations management)
SQ is complex and multifaceted concept and can be described from five different perspectives
Transcendental view
User view
Manufacturing view
Product view
Value-based view
Software Quality
CS5391 Spring 2003

6. Transcendental View
See quality as something that can be recognized but not defined
Quality as something toward which we strive as an ideal, but may never implement completely
Abstract sense
Software Quality
CS5391 Spring 2003

7. User View
More concrete, grounded in the product characteristics that meet user’s needs
Evaluate product in a task context can be personalized view
Asses product behavior with respect to operational profiles
Related to product usability and reliability
Software Quality
CS5391 Spring 2003

8. Manufacturing View
Focus on product quality during production and after delivery
Concerns whether or not the product was constructed “right the first time”
Can lead to quality assessment independent of the product
Adapted by ISO 9001 and CMM, advocates conformance to process rather than to specification
Software Quality
CS5391 Spring 2003

9. Product View
Consider the product’s inherent characteristics, looks inside
Assumption : measuring and controlling internal product properties will result in improved external product behavior
More research needed to conform this assumption
Software Quality
CS5391 Spring 2003

10. Value-Based View
Different views can be held by different groups involved in software development
user’s requirement for a useful product conflict with manufacturer’s goal of minimizing rework
consider the trade-offs between cost and quality
manage conflict  design to cost
compare with potential benefits
This view tries to bring in compromise between different groups and views
Software Quality
CS5391 Spring 2003

11. Measuring Quality We need to measure quality so we can:
Establish baselines
Predict likely quality
Monitor improvement
Product attributes contribute to user satisfaction are a mixture of:
product’s functions
product’s nonfunctional qualities
constraints
ISO definition of quality: the totality of characteristics of an entity that bear on its ability to satifsfy stated and implied needs.
Software Quality
CS5391 Spring 2003

12. Measuring user's view
Reliability: how long the product functions properly between failures
Usability: including ease of installation, learning and use
Tom Gilb’s technique: characteristics can be measured directly
Example:
Learning time = average elapsed time (in hours) for a typical user to achieve stated level of competence
Software Quality
CS5391 Spring 2003

13. Measuring user's view…
This technique can be generialized to any quality feature
The quality concept is broken down into component parts until each can be stated in terms of directly measurable attributes
Software Quality
CS5391 Spring 2003

14. Measuring the Manufacturer's View
Defects count
Count the number of known defects recorded during development and use
Can compare modules, products or projects (with same way and same time)
Relationship between defects count and operational failures is unclear
Can indicate test efficiency and identify process improvement areas
Software Quality
CS5391 Spring 2003

15. Measuring the Manufacturer's View…
Rework costs
Any additional effort required to find and fix problems after documents and code are formally signed-off as part of configuration management
Count debugging effort during integration and testing
Not count end-phase verification and validation
Pre-release rework: measure manufacturing efficient and process improvement
Post-release rework: measure of deliverd quality
Software Quality
CS5391 Spring 2003

16. Modeling Quality
Several models have been built to understand and measure quality
Old Models
McCall’s model
ISO 9126
New model
Dromey’s model
Software Quality
CS5391 Spring 2003

17. McCall's Model
Defines product qualities as hierarchy of factors, criteria and metrics:
Quality factor represents behavior characteristic of the system
Quality criterion is an attribute of a quality factor that is related to software production and design
Quality metric is a measure that captures some aspect of a quality criterion.
Software Quality
CS5391 Spring 2003

18. McCall's quality Model…
The metrics have to be answered with ‘yes’ or ‘no’ answer
The final quality is assessed by dividing the ‘yes’ answers by the total number of questions chosen to describe the quality
Software Quality
CS5391 Spring 2003

19. McCall’s Model…
Software Quality
CS5391 Spring 2003

20. ISO 9126 Six characteristics, completely hierarchical
Quality charactertistics are divided into sub-characteristics which can be measured using metrics
Recommends measuring the char-acteristics directly, but does not indicate clearly how to do it
Software Quality
CS5391 Spring 2003

21. Quality Characteristic Definition
Functionality
A set of attributes that bear on the existence of a set of functions and their specified propertise. The functions are those that satisfy stated or implied needs.
Relibity
A set of attributes that bear on the capability of software to maintain its performance level under stated conditions for a stated period of time.
Usablity
A set of attributes that bear on the effort needed for use and on the individual assessment of such use by a stated or implied set of users.
Software Quality
CS5391 Spring 2003

22. Efficiency Maintainability Portability
A set of attributes that bear on the relationship between the software’s performance and the amount of resources used under stated conditions.
Maintainability
A set of attributes that bear on the effect needed to make specified modifications (which may include correction, improvements, or adaptations of software to environmental changes and changes in the requirements and functonal specifications).
Portability
A set of attributes that bear on the ability of software to be transferred from one environment to another (this includes the organizational, hardware or software environment).
Software Quality
CS5391 Spring 2003

23. ISO 9126… Functionality Reliebility Usability
Suitability
Accurecy
Interoperability
Secuity
Functionality
Materity
Fault Telerance
Recoverability
Reliebility
Understandability
Learnability
Operability
Usability
Characteristics Sub-characteristics
Software Quality
CS5391 Spring 2003

24. ISO 9126… Effiency Maintainability Portability
Time behavior
Resource behavior
Analyzability
Changeability
Stability
Testability
Maintainability
Adaptability
Installability
Conformance
Replaceability
Portability
Characteristics Sub-characteristics
Software Quality
CS5391 Spring 2003

25. Main Differences between Mccall’s and ISO 9126 model
with different quality frame
quality factor
not hierarchical
ISO 9126
work and terminology
quality characteristic
and subcharacteristics
completely hierarchical
Software Quality
CS5391 Spring 2003

26. Model Problems
Lack rationale for determining which factor should be included in the quality definition
Lack rationale for deciding which criteria relate to a particular factor
The measure of metrics can be too subjective
Does not describe how lower level metrics are composed into an overall assessment of higher level quality characteristics
Software Quality
CS5391 Spring 2003

27. Dromey's models (new model)
Higher level quality attributes cannot be measured or built directly
Higher level quality can be achieved by building components which exhibit a consistent, harmonious and complete set of product properties that result in manifestations of quality attributes
Can allow us to verify models
Software Quality
CS5391 Spring 2003

28. Business Value of Quality
Businesses invest valuable money for obtaining good software, we have to pay them back in good
We should be careful in assessing and providing software with good quality
How much "less than perfect" software are the businesses willing to accept should be determined
Software Quality
CS5391 Spring 2003

29. Conclusion
Quality is a complex concept; It means different things to different people
You must define aspects of quality in which you are interested
Quality should be defined in a measurable way, so that it can be understood and verified
Quality must be related to business goals
Software Quality
CS5391 Spring 2003