1. Software Quality assurance SQA - SWE 333
SQA Planning and Control
Prof. Mohamed Batouche

2. SQA System An SQA Architecture Project Life Cycle SQA components
Development plan
and Quality Plan
Ch.6
Pre-project SQA components
Project Life Cycle SQA components
Formal Design Reviews
Sec. 8.2
Experts Opinion
Sec. 8.5
Peer Reviews
Sec. 8.3
SQA of External Participants
Ch 12
Software Maintenance
Ch. 11
Software Testing
Chs. 9-10
Quality Infrastructure components
Procedures
Ch. 14
Supporting
Devices
Ch. 15
Training
Instruction
Ch. 16
Preventive
Actions
Ch.17
Configuration
Management
Ch. 18
Document-
ation
Control
Ch. 19
Quality Management
Progress
Ch. 20
Software
Quality
Metrics
Ch. 21
Costs
Ch. 22
Standards
Ch. 23
Process
Ch.24
Organizational Base – Human components
Management - Ch. 25
SQA Unit - Sec. 26.1
SQA Committees – Sec. 26.2
SQA Trustees – Sec. 26.2
SQA Forums – Sec 26.4
Contract review
Ch.5
An SQA Architecture

3. SQA Planning and Control
Contract review

4. Contract review stages
Proposal draft review +
Contract draft review
= Contract review

5. Proposal Draft review - Objectives
To make sure that the following activities have been satisfactorily carried out:
Customer requirements clarified and documented
Alternative approaches for carrying out the project examined
Formal aspects of the relationship between the customer and the software firm specified
Development risks identified
Project resources and timetable adequately estimated
The firm’s capacity with respect to the project examined
The customer’s capacity to fulfill his commitments examined
Partner and subcontractor’s participation conditions defined
Protection of proprietary rights defined

6. Contract Draft review - Objectives
To make sure that the following activities have been satisfactorily carried out: 1. No unclarified issues remain in the contract draft 2. All understandings reached subsequent to the proposal are correctly documented 3. No “new” changes, additions, or omissions have entered the contract draft

7. Disadvantages of unclarified issues to customer
Subject
Disadvantages to the customer
(1) Inadequate definition of project requirements
* Implementation deviates from the needed applications
* Low satisfaction
(2) Poor estimate of the required resources
* Unrealistic expectations about project feasibility
(3) Poor timetable
* Missing scheduled dates for beginning the distribution of new products
(4) Inadequate awareness of development risks
* Customer unprepared for project risks and their consequences

8. Disadvantages of unclarified issues to developer
Subject
Disadvantages to the developer
(1) Inadequate definition of project requirements
* Higher change requirements
* Wasted resources due to introducing avoidable changes
(2) Poor estimate of the required resources
* Substantial deviations from budget
* Friction between units induced by requirements for budget additions
(3) Poor timetable
* Development activities are under time pressures and suffer from low quality
* Delays in freeing staff for their next project
(4) Inadequate awareness of development risks
* Tardy initiation of efforts to overcome difficulties

9. SQA Planning and Control
Development and Quality Plans

10. SQA Planning and Control Objectives
Planning is meant to prepare adequate foundations for successful and timely completion of the project. The planning process includes:
1. Scheduling development activities and estimating the required manpower resources and budget
2. Recruiting team members and allocating development resources
3. Resolving development risks
4. Implementing required SQA activities
5. Providing management with data needed for project control

11. Elements of Development Plan
1.   Project products, specifying “deliverables”
 2.   Project interfaces
 3.   Project’s methodology and development tools
 4.   Software development standards and procedures
 5.   Map of the development process
 6.   Project milestones
 7.   Project staff organization and coordination with external participants
 8.   Required development facilities
 9.   Development risks and risk management actions
10.   Control methods
11.   Project cost estimates

12. Elements of Quality Plan
1.         List of quality goals
2.    Review activities
3.    Software tests
4.    Acceptance tests for software externally developed
5. Configuration management plans: tools, procedures and data for version releases

13. SQA Planning and Control
Reviews
As defined by IEEE (1990), a review process is:
“A process or meeting during which a work product, or set of work products,
is presented to project personnel, managers, users, customers, or
other interested parties for comment or approval.”

14. Reviews
Reviews:
The design phase of the development process produces a variety of documents. The printed products include design reports, software test documents, software installation and software manuals, among others . All these documents have to be examined in depth.
Reviews can be categorized as follows:
Formal design reviews (DRs): These documents require formal professional approval of their quality before moving to next step. Committees are composed of senior professionals including project leader. Formal design reviews are authorized to approve the design document so that work on the next stage of the project can begin.
Peer reviews (Inspections and Walkthrough): Directed at reviewing short documents. Reviewers are all peers. The main objective is to detect as many design and programming faults as possible and deviations from standards.

15. Reviews Objectives Direct objectives Indirect objectives
a. To detect analysis and design errors as well as subjects where corrections, changes and completions are required
b. To identify new risks likely to affect the project.
c.  To locate deviations from templates, style procedures and conventions.
d. To approve the analysis or design product. Approval
allows the team to continue on to the next development phase.
Indirect objectives
a. To provide an informal meeting place for exchange of professional knowledge about methods, tools and techniques.
b. To record analysis and design errors that will serve as a basis for future corrective actions.  

16. With Technical Reviews and Inspections:
Defect Creation & Discovery When Validation is the Primary Removal Method:
Requirements Design Coding Documentation Testing Maintenance
Defect Origins
Defect Discovery
Requirements Design Coding Documentation Testing Maintenance
With Technical Reviews and Inspections:
Requirements Design Coding Documentation Testing Maintenance
Defect Origins
Defect Discovery
Requirements Design Coding Documentation Testing Maintenance

17. Reviews
Formal design reviews (DRs): These documents require formal professional approval of their quality before moving to next step. Committees are composed of senior professionals including project leader.

18. Formal Design Reviews (DRs)
Some common DRs: DPR – Development Plan Review SRSR – Software Requirement Specification Review PDR – Preliminary Design Review DDR – Detailed Design Review DBDR – Data Base Design Review TPR – Test Plan Review STPR – Software Test Procedure Review VDR – Version Description Review OMR – Operator Manual Review SMR – Support Manual Review TRR – Test Readiness Review PRR – Product Release Review IPR – Installation Plan Review

19. The Agenda of a DR Session
a.   A short presentation of the design document.
b.  Comments made by members of the review team.
c.  Verification and validation of comments is discussed to determine the required action items (corrections, changes and additions).
d.  Decisions about the design product (document), which determines the project's progress:
       ·      Full approval.
       ·      Partial approval.                      
·      Denial of approval.

20. DR post-review activities
Preparation of the DR report.
The report's major sections:
    ·    A summary of the review discussions.
·    The decision about continuation of the project.
·    A full list of the required action items — corrections, changes and additions. For each action item, completion date and project team member responsible are listed.
·    The name(s) of the review team member(s) assigned to follow up.
b. Follow up performance of the corrections and to examine the corrected sections.

21. Design Review process Major corrections Review Report Next Step
No approval
Approval
Partial approval

22. Design Review Report

23. Pressman’s 13 Golden Guidelines for a successful DR
Design Review Infrastructure
    1   Develop checklists for common types of design documents.
    2   Train senior professionals serve as a reservoir for DR teams.
    3   Periodically analyze past DR effectiveness.
    4  Schedule the DRs as part of the project plan.
The Design Review Team
  5  Review teams size should be limited, with 3–5 members being the optimum.

24. Pressman’s 13 Golden Guidelines for a Successful DR
The Design Review Session
Discuss professional issues in a constructive way refraining from personalizing the issues.
Keep to the review agenda.
Focus on detection of defects by verifying and validating the participants' comments. Refrain from discussing possible solutions.
In cases of disagreement about an error - end the debate by noting the issue and shifting its discussion to another forum.
Properly document the discussed comments, and the results of their verification and validation.
The duration of a review session should not exceed two hours.
 Post-Review Activities
Prepare the review report, including the action items
Establish follow-up to ensure the satisfactory performance of all the list of action items

25. Reviews
Peer reviews (Inspections and Walkthrough): Directed at reviewing short documents. Reviewers are all peers. The main objective is to detect as many design and programming faults as possible.

26. Candidates for Reviews and Inspections
Strategic Plans
Contracts
Requirements
High Level Designs
Detailed Designs
Architectural Documentation
Code
Test Plans
Test Designs
Test Source
User Documentation
Project Plans, etc.

27. Participants of peer reviews
Inspection
Walkthrough
Review leader
The author
Specialized professionals:
Designer
Coder or implementer
Tester
Review leader
The author
Specialized professionals:
Standards enforcer
Maintenance expert
User representative

28. peer reviews: Inspections
Inspections Reduce Project
Costs & Schedule
Project Cost
Time Schedule
Savings
With Inspections Without Inspections

29. peer reviews: Inspections

30. What are Inspections? An inspection is a structured peer review:
That Provides: To:
Defect information Author
Other perspectives on work Author
Workproduct information Peers and other groups
Accurate project status Product Management
Generic defects (trends) Management

31. Inspection roles

32. Inspection checklists
Checklist of common errors should be used to drive the inspection.
Error checklists are programming language dependent and reflect the characteristic errors that are likely to arise in the language.
In general, the 'weaker' the type checking, the larger the checklist.
Examples: Initialisation, Constant naming, loop termination, array bounds, etc.

33. Inspection checks 1

34. Inspection checks 2

35. Inspection Session Findings report

36. Inspection Session Summary Report

37. Code Inspection effectiveness at Fujitso (Cusumano)
Year
Defect detection method
Defects per 1000 lines of maintained code
Test %
Design review
Code inspection
1977 85
--- 15
0.19
1978 80 5
0.13
1979 70 10 20
0.06
1980 60 25
0.05
1981 40 30
0.04
1982
0.02

38. Walkthrough Task Kickoff Meeting Task Kickoff Meeting Standards,
templates, rules
and checklists
Walkthrough
Standards,
templates, rules
and checklists
Walkthrough E N T R Y E N T R Y E X I T
Output E X I T
Output...
Program
Element
Creation
Program
Element
Creation
Input
Input
Task
Kickoff
Meeting
Task
Kickoff
Meeting
SW Inspection
SW Inspection

39. Walkthrough Process Led by author
Reduce the amount of time needed for re-work by reviewing work completed to date and address questions and concerns.
Led by author
Discussion of deliverable style and format
Examine technical alternatives
Evaluate checklists, standards, and rules.
Capture undocumented issues
Forum for learning

40. Inspection vs. Walkthrough

41. Peer Review Methods Desk Check SW Inspection Walkthrough Methods
Typical Goals
Typical Attributes
Minimal overhead
Quick turnaround
Little/no preparation
Informal process
No measurement
No Meetings
Done by the author
Desk Check
Minimal overhead
Developer training
Quick turnaround
Little/no preparation
Informal process
Meetings
No measurement
Led by the author
Walkthrough
Detect and report all defects efficiently and effectively.
Usage of deliverable
Formal process
Known Coverage Rate
Moderator
Checklists
Customer Reviewers
Measurement
SW Inspection

42. Comparison of Review methodologies Process
Properties
Design review
Inspection
Walkthrough
Overview meeting No
Yes
Participant’s preparations
Yes - thorough
Yes - brief
Review session
Follow-up of corrections
Formal training of participants
Participant’s use of checklists
Error-related data collection
Not formally required
Formally required
Review documentation
Formal design review report
1) Inspection session findings report
2) Inspection session summary report

43. Benefits of Use of Templates
For development teams: * Facilitates the process of preparing documents. * Documents prepared by developer are more complete. * Provides for easier integration of new team members. * Facilitates review of documents. For software maintenance teams: * Enables easier location of the information.

44. Benefits of Use of Checklists
To development teams:
     * Helps developers carrying out self-checks of documents or software code prior completion.
     * Assists developers in their preparations for tasks.
To review teams:
     * Assures completeness of document reviews by review team members.
     * Facilitates improves efficiency of review sessions.

46. Expert’s participation in reviews
· Insufficient in-house professional capabilities in a specialized area. · Temporary lack of in-house professionals for review team. · Indecisiveness caused by major disagreements among the organization’s senior professionals. · In small organizations, where the number of suitable candidates for a review team is insufficient.

47. Audit Process
Software Quality Assurance is to provide management with appropriate visibility into the process being used by the software developers and of the products being built.
Determine:
Conformance to policies, procedures, and standards
Adequacy of policies, procedures, and standards
Effectiveness and efficiency of policies, procedures, and standards
Assess personnel familiarity to requirements and documentation
Assure availability, use and adherence to software standards

48. Audit Process Developers Auditor Auditor & Project Manager Start END
Checklist
Audit
Kickoff
Meeting
Start
Produce
Product
Identify
Requirements
Conduct
Audit
Write-up
Report &
Findings
Review
with
Manager NO
Un-
satisfactory
Report?
YES
Corrective
Actions
Close
Audit &
File
END OK
Re-Work
Follow-up
Audit 14

49. SQA Planning and Control
Management of Software Development Risks

50. What is a Risk?
Risks are potential problems that may affect successful completion of a software project.
Risks involve uncertainty and potential losses.
Risk analysis and management are intended to help a software team understand and manage uncertainty during the development process.

51. Classes of Software Development Risks
1. Scheduling and timing risks
2. System functionality risks
3. Subcontracting risks
4. Requirement management risks
5. Resource usage and performance risks
6. Personnel management risks

52. Boehm and Ross’s Top 10 Software Risk items
Developing wrong software functions
Unrealistic schedules and budgets
Developing wrong user interface
Gold plating
Continuing stream of requirement changes
Shortfalls in externally furnished components
Shortfalls in externally performed tasks
Personnel shortfalls
Real-time performance shortfalls
Straining computer science capabilities

53. The Software Management Risk Process
Pre -
project
Risk identification
and assessment
Planning risk
management activities
New
Planning and updating risk
Implementing
risk management actions
(risk resolution)
Monitoring software risk
Identifying and
assessing new software
risks
Ongoing
projects
Evaluate
monitoring
results
Required results
achieved
Unsatisfactory results

54. References
Williaw E. Lewis, “Software Testing And Continuous Quality Improvement”, Third Edition, CRC Press, 2009.
K. Naik and P. Tripathy: “Software Testing and Quality Assurance”, Wiley, 2008.
Ian Sommerville, Software Engineering, 8th edition, 2006.
Aditya P. Mathur,“Foundations of Software Testing”, Pearson Education, 2009.
D. Galin, “Software Quality Assurance: From Theory to Implementation”, Pearson Education, 2004
David Gustafson, “Theory and Problems of Software Engineering”, Schaum’s Outline Series, McGRAW-HILL, 2002.

55. Q & A