1. Quality Exercise 2 Instructions
Quality Analysis
The current software upgrade project has been in progress for a year now and the project should be well into the Coding and Unit Testing phase. The PM believes that something is amiss and has asked your IPT to perform an in-depth analysis to assess the project’s status from a quality perspective. The PM received from the prime contractor the measurement indicators (without any additional comments or information) and has given them to your IPT to analyze (see Contractor Data Presented to PM). The PM has also identified the program’s specific Quality Attributes and is concerned that the data does not allow for a good quality assessment. The PM wants your IPT to complete the enclosed assessment templates.
Tasks:  Within your IPT, review carefully the set of charts to identify problems, assess their impact, project the outcomes, and evaluate alternatives.
Compare/contrast the PM’s Quality Attributes with the Software Quality Model Characteristics (ISO :2001 Software engineering) presented in the lesson material and identify any areas you feel the PM should address.
Compare and contrast the ICM Quality measurable concepts (with indicators and measures) from Exercise 1 with the PM’s Quality Attributes. Identify the differences between the ICM table and the PM’s Quality Attributes.
Assess the data provided by the contractor - does the data provided meet information needs based on the PM’s Quality Attributes? If not, identify the areas where you need additional data to support the PM’s Quality Attributes.
What recommendations would you make to the PM based on the insight gained from your analysis? Be sure to include recommendations for strengthening the overall SW quality program. What additional analyses you would perform to assess and strengthen the SW Quality activities.
SLIDE INFORMATION************************************* *Slide Type (Content or Exercise): Content
*ELO ID: 
ELO Given several process-focused and product-focused software quality assurance methods, describe how each assures quality in a software acquisition
ELO Given a software acquisition scenario, recognize the preferred method for identifying and tracking defects
***************************************************************

2. PM Quality Attributes Quality Attribute. Modularity
Quality Attribute. Interoperability
Attribute Concerns
A. Ease of interfacing with using applications
B. Ease of interfacing with other systems
Quality Attribute. Modularity
Attribute Concerns
A. Replace an architectural component
Loose coupling/high cohesion
Design from a common set of components
Quality Attribute Adaptability
Attribute Concerns
Ability to accommodate new requirements
Ability to accommodate new technologies
Ability to field a subset of the current requirements (functionality)
Ability to support various platforms
Quality Attribute Usability
Attribute Concerns
A. Minimize user training (operator, user, non-signal user)
B. Enable user to perform tasks in allocated time.
C. Minimize number of operators.
D. Consistent GUI (common look and feel)
Quality Attribute. Maintainability
Attribute Concerns
A. Ability to accommodate new technologies
B. Ability to support isolation of faults
C. Minimize training for maintainers

3. Quality Exercise 2 Answer Template
Performance Analysis Worksheet
1. Compare/contrast the PM’s Quality Attributes with the Software Quality Model Characteristics (ISO : Software engineering) presented in the lesson material and identify any areas you feel the PM should address.
2. Compare and contrast the ICM Quality measurable concepts (with indicators and measures) from Exercise 1 with the PM’s Quality Attributes. Identify the differences between the ICM table and the PM’s Quality Attributes.   
SLIDE INFORMATION************************************* *Slide Type (Content or Exercise): Content
*ELO ID: 
ELO Given several process-focused and product-focused software quality assurance methods, describe how each assures quality in a software acquisition
ELO Given a software acquisition scenario, recognize the preferred method for identifying and tracking defects
***************************************************************

4. Quality Exercise 2 Answer Template
Performance Analysis Worksheet
3. Assess the data provided by the contractor - does the data provided meet information needs based on the PM’s Quality Attributes? If not, identify the areas where you need additional data to support the PM’s Quality Attributes.
What recommendations would you make to the PM based on the insight gained from your analysis? Be sure to include recommendations for strengthening the overall SW quality program. What additional analyses you would perform to assess and strengthen the SW Quality activities.

5. Software Quality Model Characteristics
ISO :2001 Software engineering -- Product quality provides a SW quality model that identifies six main quality characteristics:
Functionality
Suitability
Accurateness
Interoperability
Compliance
Security
Reliability
Maturity
Fault tolerance
Recoverability
Usability
Understandability
Learnability
Operability
Efficiency
Time behavior
Resource behavior
Analyzability
Maintainability
Changeability
Stability
Testability
Adaptability
Portability
Installability
Conformance
Replaceability
SLIDE INFORMATION***************************************************************************************************************************
*Policy / Directive / Standard / DTM ID: ISO 9126 Software Engineering – Product Quality
*Supporting ELOs ID: ELO 24.X.X.X Define software quality
*Slide Type: Content (Content or Exercise)
**********************************************************************************************************************************************************
ISO 9126 is an international standard for the evaluation of software. The standard is divided into four parts which addresses, respectively, the following subjects: quality model; external metrics; internal metrics; and quality in use metrics. ISO 9126 Part one, referred to as ISO is an extension of previous work done by McCall (1977), Boehm (1978), FURPS and others in defining a set of software quality characteristics. ISO represents the latest (and ongoing) research into characterizing software for the purposes of software quality control, software quality assurance and software process improvement (SPI). This article defines the characteristics identified by ISO The other parts of ISO 9126, concerning metrics or measurements for these characteristics, are essential for SQC, SQA and SPI but the main concern of this article is the definition of the basic ISO 9126 Quality Model. The ISO software quality model identifies 6 main quality characteristics, namely:
Key Points:
Efficiency
Maintainability
Usability
Reliability
Functionality
Portability
MT Software Quality factors are attributes of the software that improve the efficient use (capabilities provided) and impact the life-cycle costs of the software.
These characteristics are broken down into sub-characteristics, a high level table is shown below under the Terms\Definitions\Acronyms section
How are these “ilities” being put to use in your Software Quality Assurance program?
Key Questions to Ask and Anticipated Answers:
The main characteristics of the ISO quality model, can be defined as follows:
Terms \ Definitions \ Acronyms:
Suitability
This refers to the correctness of the functions, an ATM may provide a cash dispensing function but is the amount correct?
Accurateness
This is the essential Functionality characteristic and refers to the appropriateness (to specification) of the functions of the software
Interoperability
Where appropriate certain industry (or government) laws and guidelines need to be complied with, i.e. SOX. This subcharacteristic addresses the compliant capability of software.
Compliance
A given software component or system does not typically function in isolation. This subcharacteristic concerns the ability of a software component to interact with other components or systems.
This subcharacteristic relates to unauthorized access to the software functions.
 Security
Fault tolerance
This subcharacteristic concerns frequency of failure of the software.
Maturity
The ability of software to withstand (and recover) from component, or environmental, failure.
Understandability
Ability to bring back a failed system to full operation, including data and network connections.
Recoverability
Determines the ease of which the systems functions can be understood, relates to user mental models in Human Computer Interaction methods.
Learning effort for different users, i.e. novice, expert, casual etc.
Learnability
Operability
Ability of the software to be easily operated by a given user in a given environment.
Characterizes response times for a given thru put, i.e. transaction rate.
Time behavior
Resource behavior
Characterizes the ability to identify the root cause of a failure within the software.
Analyzability
Characterizes resources used, i.e. memory, cpu, disk and network usage.
Changeability
Characterizes the sensitivity to change of a given system that is the negative impact that may be caused by system changes.
Stability
Characterizes the amount of effort to change a system.
Testability
Characterizes the ability of the system to change to new specifications or operating environments.
Adaptability
Characterizes the effort needed to verify (test) a system change.
Installability
Similar to compliance for functionality, but this characteristic relates to portability. One example would be Open SQL conformance which relates to portability of database used.
Conformance
Characterizes the effort required to install the software.
Characterizes the plug and play aspect of software components, that is how easy is it to exchange a given software component within a specified environment.
Replaceability
Software Quality Assurance

6. Quality Exercise 1 Potential Solution
Measurable Concepts and
Questions Addressed
Potential Indicators and Measures
Priority
Functionality
Is the product good enough for delivery to the
user?
Are identified problems being resolved?
Defect Profiles / Defect Density
Technical Measurement Trends
System Elements Accepted
Defects by status, severity, priority, distribution, age,etc.
Technical measurement requirement, target, threshold,budget, and actual
System elements verified 2
Reliability
How often is service to users interrupted?
Are failure rates within acceptable bounds?
Mean-Time-to-Failure
Availability
System element failures by severity, priority
System element start, end times 1
Usability
Is the user interface adequate and appropriate for operations?
Are operator errors within acceptable bounds?
User Interface Acceptability
Operator Error Trends
Actions from user interface reviews
Operator errors 5
Efficiency
Does the target system make efficient use of system resources?
Utilization
Throughput
Response Time
System element capacity available, used
Time for function (budget, actual) 6
Maintainability
How much support does the system require?
How difficult is it to support?
How big is and how much change is occurring with the product's functional size, content, or logical characteristics?
Interface number (unique), complexity, growth, approval rates, changes, TBD/TBR closure per plan
Interface Complexity, Interface Compatibility, Lines of Code Trends
Hours to restore
Calendar hours and labor hours to repair
Number of paths through system 4
Portability
To what extent can the functionality be rehosted on different platforms?
Interface Compliance
Interfaces verified 7
Safety and Security
How many vulnerabilities are identified and remediated by life-cycle phase?
How many relevant attack patterns have been covered by test cases?
- Vulnerabilities discovered, remediated
- Cost to fix vulnerabilities
- Test cases developed, verified per attack pattern 3
SLIDE INFORMATION************************************* *Slide Type (Content or Exercise): Content
*ELO ID: 
ELO Given several process-focused and product-focused software quality assurance methods, describe how each assures quality in a software acquisition
ELO Given a software acquisition scenario, recognize the preferred method for identifying and tracking defects
***************************************************************
Instruction version of ICM chart is highlighted to show from where potential answers may originate; prioritization is based on the current problems the program is having – the students can identify other priorities as long as they justify their answers.