1. CHAPTER 4 Life-Cycle Components

2. 2 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser 2 Learning Objectives: To discuss: Integrating quality activities in the project life cycle Reviews Software testing Integrating quality in software maintenance

3. 3 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Integrating Quality Activities In The Project Life Cycle Classic and other software development methodologies Factors affecting intensity of quality assurance activities in the development process Model for SQA defect removal effectiveness and cost

4. 4 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Integrating Quality Activities In The Project Life Cycle SQA can be planned according to the software development process and activities. Therefore, SQA professionals should be acquainted with the various software engineering models in order to be able to prepare a quality plane that is properly integrated into the project plan.

5. 5 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Classic and other software development methodologies SDLC Prototype Spiral Object-oriented

6. 6 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Prototype model

7. 7 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Source: After Boehm 1988 (© 1988 IEEE) Spiral model

8. 8 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Object-oriented model

9. 9 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Factors affecting intensity of quality assurance activities in the development process Project factors Team factors

10. 10 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Project factors Magnitude of the project Technical complexity and difficulty Extent of reusable software components Severity of failure outcomes if the project fails Factors affecting intensity of quality assurance activities in the development process

11. 11 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Team factors Professional qualification of the team members Team acquaintance with the project and its experience in the area Availability of staff members who can professionally support the team Familiarity with the team members, in other words the percentage of new staff members in the team. Factors affecting intensity of quality assurance activities in the development process

12. 12 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Verification – The process of evaluating a system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase Validation - The process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements Qualification - The process used to determine whether a system or component is suitable for operational use IEEE Std 610.12-1990 (IEEE 1990) Model for SQA defect removal effectiveness and cost

13. 13 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser A model for SQA defect removal effectiveness and cost The model deals with two quantitative aspects of an SQA plan consisting of several defect detection activities: The plan’s total effectiveness in removing project defects. The total costs of removal of project defects. The plan itself is to be integrated within a project’s development process.

14. 14 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Defect origin distribution – from Boehm (1981) and Jones (1996) Software development phase Average % of defects originating in phase, POD Requirement specification 15% Design35% Unit coding30% Integration coding10% Documentation10% System testing----- Operation-----

15. 15 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Defect removal effectiveness – from Boehm (1981) and Jones (1996) Quality assurance activity Defects removal effectiveness rate for standard SQA plan Defects removal effectiveness rate for comprehensive SQA plan Specification requirement review 50%60% Design inspection-----70% Design review50%60% Code inspection-----70% Unit test50%40% Integration tests50%60% Documentation review50%60% System test50%60% Operation phase detection100%

16. 16 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Cost of defect removal – based on Boehm (1981) and Pressman (2000) Software development phase Average % of defects originating in phase Average relative defect removal cost Requirement specification 15%1 day Design35%2.5 Unit coding30%6.5 Integration coding10%16 Documentation10%40 System testing-----40 Operation-----110

17. 17 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser The model Based on the following assumptions: The development process is linear and sequential (waterfall model). A number of “new” defects are introduced in each development phase. Various review and test software assurance activities serve as filters, removing a percentage of the entering defects while allowing the rest to pass to the next software assurance activity.

18. 18 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser The model (cont) Incoming defects are the sum of defects passed from the former quality assurance activity together with “new” defects created in the current development phase. The cost of defect removal is calculated by multiplying the number of defects removed by the relative cost of removing a defect. Defects passed to the customer will be detected by him or her; their full removal at this phase will incur heavy costs.

19. Defect removal phaseDefect removal effectiveness Average relative defect removal cost {cost unit} Defect origination phase ReqDesUniIntDoc Requirement specification (Req) 50%1--- Design (Des)50%2.51--- Unit coding (Uni)50%6.52.61--- Integration (Int) System documentation (Doc) 50% 16 6.4 2.5 1 1 System testing / Acceptance testing (Sys) 50%40166.22.5 Opertion by customer (after release) 100%11044176.9

20. · POD = Phase Originated Defects ·PD = Passed Defects (from former phase or former quality assurance activity) ·%FE = % of Filtering Effectiveness (also termed % screening effectiveness) ·RD = Removed Defects ·CDR = Cost of Defect Removal ·TRC = Total Removal Cost. TRC = RD x CDR.

21. D doc D int D uni D des D req Total ID 15 PD 7.5 %FE=50 RD 7.5 RDRC 1TRC >7.5 cu Req, specification(POD=15) D doc D int D uni D des D req Total ID 357.542.5 PD 17.53.821.3%FE=50 RD 17.53.721.2 RDRC 12.5TRC >26.8 cu D doc D int D uni D des D req Total ID 3017.53.851.3 PD 158.81.925.7%FE=50 RD 158.71.925.6 RDRC 12.66.5TRC >50 cu D doc D int D uni D des D req Total ID 10158.81.935.7 PD 57.54.41.017.9%FE=50 RD 57.54.40.917.8 RDRC 12.56.416TRC >66.3 cu D doc D int D uni D des D req Total ID 1057.54.41.027.9 PD 52.53.82.20.514.0%FE=50 RD 52.53.72.20.513.9 RDRC 112.56.416TRC >38.9 cu D doc D int D uni D des D req Total ID 52.53.82.20.514.0 PD 2.51.21.91.10.37.0%FE=50 RD 2.51.31.91.10.27.0 RDRC 2.5 6.21640TRC >50.9 cu D doc D int D uni D des D req Total ID 2.51.21.91.10.37.0 PD 000000%FE=50 RD 2.51.21.91.10.37.0 RDRC 6.9 1744110TRC >139.2 cu Design(POD=35) Unit tests(POD=30) Integration tests(POD=10) Operation bycustomer (POD=0) System tests(POD=0) Documentationreviews (POD=10) 21.3 25.7 17.9 7.5 14.0 7.0 Slide 7.12a – relates to updated section 7.4

22. Reviews

23. 23 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Reviews Review objectives Formal design reviews (DRs) Peer reviews A comparison of the team review methods Expert opinion

24. 24 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Review objectives Direct objectives Indirect objectives

25. 25 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Direct review objectives To detect analysis and design errors as well as subjects where corrections, changes and completions are required with respect to the original specifications and approved changes. To identify new risks likely to affect completion of the project. To locate deviations from templates and style procedures and conventions. Corrections of these deviations is expected to contribute to improved communication and coordination resulting from greater uniformity of methods and documentation style. To approve the analysis or design product. Approval allows the team to continue to the next development phase.

26. 26 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Indirect review objectives To provide an informal meeting place for exchange of professional knowledge about development methods, tools and techniques. To record analysis and design errors that will serve as a basis for future corrective actions. The corrective actions are expected to improve development methods by increasing effectiveness and quality, among other product features.

27. 27 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Formal design reviews (DRs) Variously called “design reviews” / ”DRs” / ”formal technical reviews (FTR)”. Without this approval, the development team cannot continue to the next phase of the software development project. May be conducted at any development milestone requiring completion of an analysis or design document, whether that document is a requirement specification or an installation plan.

28. 28 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Some common formal design reviews DPR – Development Plan Review SRSR – Software Requirement Specification Review PDR – Preliminary Design Review DDR – Detailed Design Review DBDR – Data Base Design Review TPR – Test Plan Review STPR – Software Test Procedure Review VDR – Version Description Review OMR – Operator Manual Review SMR – Support Manual Review TRR – Test Readiness Review PRR – Product Release Review IPR – Installation Plan Review

29. 29 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Participants in a DR The review leader The review team

30. 30 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Characteristics of a DR leader Knowledge and experience in development of projects of the type reviewed. Preliminary acquaintance with the current project is not necessary. Seniority at a level similar if not higher than that of the project leader. A good relationship with the project leader and his team. A position external to the project team

31. 31 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser The review team Member: Senior members of the project team, together with senior professionals assigned to other projects/departments. Customer-user representatives Software development consultants. Size: efficient team – 3 to 5 members. Large size  problems?

32. 32 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Preparation for a DR Main tasks of the review leader in preparation stage are: To appoint the team members To schedule the review sessions To distribute the design document among the team members.

33. 33 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Preparation for a DR Review team members preparations: Review the design and list their comments prior to the review session. Prepare checklist. Department team preparation: Short presentation of the design document – focus on the main professional issues awaiting approval rather than description of the project in general.

34. 34 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser The DR session A short presentation of the design document. Comments made by members of the review team. Verification and validation of comments is discussed to determine the required action items (corrections, changes and additions). Decisions about the design product (document), which determines the project's progress: Full approval Partial approval Denial of approval

35. 35 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Post-review activities The DR report DR leader is responsible to issue the DR report immediately after the DR session. Content: A summary of the review discussions. The decision about continuation of the project. A full list of the required action items — corrections, changes and additions. For each action item, completion date and project team member responsible are listed. The name(s) of the review team member(s) assigned to follow up. Follow-up performance of the corrections and to examine the corrected sections.

36. Design Review Infrastructure · Develop checklists for common types of design documents. · Train senior professionals serve as a reservoir for DR teams. · Periodically analyze past DR effectiveness. · Schedule the DRs as part of the project plan. The Design Review Team. Review teams size should be limited, with 3–5 members being the optimum.

37. The Design Review Session Discuss professional issues in a constructive way refraining from personalizing the issues. Keep to the review agenda. Focus on detection of defects by verifying and validating the participants' comments. Refrain from discussing possible solutions. In cases of disagreement about an error - end the debate by noting the issue and shifting its discussion to another forum. Properly document the discussed comments, and the results of their verification and validation. The duration of a review session should not exceed two hours. Post-Review Activities Prepare the review report, including the action items Establish follow-up to ensure the satisfactory performance of all the list of action items

38. 38 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Peer reviews Major difference between formal design reviews and peer reviews is in their participants: DR – superior positions Peer reviews – project leaders equals, member of his/her department and other units. Degree of authority and objectives: DR – approve the design document Peer reviews – detect errors and deviations from standards (abnormality).

39. 39 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Peer review method Inspection More formal, corrective action, effort to improve development method  considered to contribute more significantly to the general level of SQA. Walkthrough Findings are limited to comments on the document reviewed.

40. 40 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Review leader The author Specialized professionals: Designer Coder or implementer Tester Review leader The author Specialized professionals: Standards enforcer – expert in standards and procedures Maintenance expert User representative Participants of peer review

41. 41 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Peer review leader Be well versed in development of projects of the current type and familiar with its technologies. Preliminary acquaintance with the current project is not necessary. Maintain good relationships with the author and the development team. Come from outside the project team. Display proven experience in coordination and leadership of professional meetings. For inspections, training as a moderator is also required.

42. 42 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Team assignments The presenter Chosen by moderator (review leader). Inspection: Not the document’s author, must understand the technical part of the reviewed document. Walkthrough: The document’s author. The scribe Usually team leader, record the noted defects. More procedural: requires thorough professional understanding of the issues discussed.

43. 43 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Preparation of a peer review session Peer review leader To determine (together with the author, which sections of the design document are to be reviewed – the most difficult and complex sections, critical sections, where any defect can cause severe damage, sections prone to defects. To select team members. To schedule the peer review sessions. To distribute the document to team members prior to the review session.

44. 44 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Preparation of a peer review session Peer review team Inspection Thorough Team members are expected to read the document sections to be reviewed and list comments before the inspection session begins. Checklist. Walkthrough Brief Team member briefly read the material to get general overview of the section to be reviewed, the project and its environment. Not required to prepare comments.

45. 45 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Peer review session Presenter reads a section of the document and adds if needed, a brief explanation of the issues involved in his/her own words. The participant either deliver their comments to the document. Identification of errors – not deal with tentative solutions. Walkthrough – start with the author’s short presentation or overview of the project and the design sections to be reviewed. During the session, the scribe should document each error recognized by location and description, type and character (incorrect, missing parts or extra parts). Inspection – will add the estimated severity of each defect, a factor to be used in the statistical analysis of defects found and for the formulation of preventive and corrective actions. Duration for peer review session – not more than 2 hours each session, not more than twice daily.

46. 46 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Session documentation Documentation produced after the inspection session is more comprehensive than a walkthrough session. Two documents to be produced: Inspection session findings report Produced by the scribe, should be completed and distributed immediately after the session’s closing. Main purpose – documentation of identified errors for correction. Inspection summary report To be compiled by the inspection leader shortly after the session. Summarizes the inspection findings and the resources invested in the inspection – basic quality and efficiency metrics. Input for analysis aimed at inspection process improvement and corrective actions. Copies of error documentation will be handed to the development team and the session participants.

47. 47 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser The efficiency of peer review activities Common metrics applied to estimate the efficiency of peer reviews: Peer review detection efficiency (average hours worked per detect detected). Peer review defect detection density (average number of defects detected per page of the design document). Internal peer review effectiveness (% of defects detected by peer review as a percentage of total defects by the developer).

49. Sections recommended for inclusion Sections of complicated logic Critical sections, where defects severely damage essential system capability Sections dealing with new environments Sections designed by new or inexperienced team members Sections recommended for omission “Straightforward” sections (no complications) Sections of a type already reviewed by the team in similar past projects Sections that, if faulty, are not expected to effect functionality Reused design and code Repeated parts of the design and code

50. PropertiesDesign reviewInspectionWalkthrough Overview meetingNoYesNo Participant’s preparations Yes - thorough Yes - brief Review sessionYes Follow-up of corrections Yes No Formal training of participants NoYesNo Participant’s use of checklists NoYesNo Error-related data collection Not formally required Formally required Not formally required Review documentation Formal design review report 1) Inspection session findings report 2) Inspection session summary report

51. 51 ESGD5125 SEM II 2009/2010 Dr. Samy Abu Naser Expert opinion Prepared by outside experts, support quality evaluation by introducing additional capabilities to the internal review staff. Outside experts transmit their expertise by either: Preparing an expert’s judgement about a document or a code section. Participating as a member of an internal design review, inspection or walkthrough team.

52. · Insufficient in-house professional capabilities in a specialized area. · Temporary lack of in-house professionals for review team. · Indecisiveness caused by major disagreements among the organization’s senior professionals. · In small organizations, where the number of suitable candidates for a review team is insufficient.