2. Thomas L. Gilchrist tomg@tomgtomg.com Testing Basics Set 5: CM & Peer Reviews By Thomas L. Gilchrist CSQE,CSQA 2009

3. Thomas L. Gilchrist tomg@tomgtomg.com SCCM Software Change & Configuration Management

4. Thomas L. Gilchrist tomg@tomgtomg.com SCCM (software change and configuration management) Basics Identification: How does and organization identify and manage the many existing versions of a program and its documentation in a manner that will enable change to be accommodated efficiently? Control: How does an organization control changes before and after the software (and supporting documentation) is released to the customer? Status Accounting: Who has responsibility for approving and ranking changes Verification: How can we ensure that changes have been made properly? Reporting: What mechanism is used to appraise others of changes that have been made (or being made). Roger Pressman…”Software Engineering, Fifth Edition”

5. Thomas L. Gilchrist tomg@tomgtomg.com CM High Level View Project CM Repository SW Eng Tasks New SCI Tech Review Approved SCI SCM Controls SCI = Software Configuration Item

6. Thomas L. Gilchrist tomg@tomgtomg.com CM High Level View Project CM Repository SW Eng Tasks New SCI Tech Review Approved SCI SCM Controls Read Only Copy Ver 1 SCI Ver 1 SCI SCI = Software Configuration Item Ver 1 SCI Ver 1 SCI Project CM Repository Tech Review

7. Thomas L. Gilchrist tomg@tomgtomg.com CM High Level View Project CM Repository SCM Controls Request For Change Read Only Copy Ver 1 Ver 1 Ver 1 SCI Ver 1 SCI Ver 1 SCI Ver 1 SCI Project CM Repository

8. Thomas L. Gilchrist tomg@tomgtomg.com CM High Level View Project CM Repository SW Eng Tasks Current Baseline SCI Approved to Change SCM Controls Request For Change Read Only Copy Ver 1 Ver 1 Ver 1 SCI Ver 1 SCI Ver 1 SCI Ver 1 SCI Project CM Repository

9. Thomas L. Gilchrist tomg@tomgtomg.com CM High Level View Project CM Repository SW Eng Tasks SCM Controls Request For Change Read Only Copy Ver 1 Ver 1 Ver 1 SCI Ver 1 SCI Ver 1 SCI Ver 1 SCI Current Baseline SCI Approved to Change Project CM Repository

10. Thomas L. Gilchrist tomg@tomgtomg.com CM High Level View Project CM Repository SW Eng Tasks Modified SCI Tech Review SCM Controls SCM Controls Request For Change Read Only Copy Ver 1 Ver 1 Approved SCI Ver 1 SCI Ver 1 SCI Ver 1 SCI Ver 1 SCI Current Baseline SCI Approved to Change Project CM Repository Tech Review

11. Thomas L. Gilchrist tomg@tomgtomg.com CM High Level View Project CM Repository SW Eng Tasks Tech Review SCM Controls SCM Controls Request For Change Read Only Copy Modified SCI Approved SCI ? ? Ver 2 SCI Ver 2 SCI Current Baseline SCI Approved to Change Project CM Repository Tech Review

12. Thomas L. Gilchrist tomg@tomgtomg.com Peer Reviews

13. Thomas L. Gilchrist tomg@tomgtomg.com Reviews Vs. Testing Dynamic Vs. Static Testing Peer Reviews can be done before code can be tested. Testing evaluates product while performing function in target environment. Not mutually exclusive

14. Thomas L. Gilchrist tomg@tomgtomg.com Peer Reviews Intended as a method of static verification for software by the producers’ peers to identify defects and areas where changes are needed. Method is not intended to replace dynamic testing, but rather to augment it.

15. Thomas L. Gilchrist tomg@tomgtomg.com What Peer Review Is Not A “silver bullet.” A tool for personnel evaluation. A quality assurance program.

16. Thomas L. Gilchrist tomg@tomgtomg.com Static Verification Techniques Desk Checking Walkthroughs Formal Technical Reviews Inspection (Fagan) Management Reviews Audits Static Program Analysis Formal (Mathematical) Verification

17. Thomas L. Gilchrist tomg@tomgtomg.com Peer Review Methods Walkthroughs MethodsTypical GoalsTypical Attributes Minimal overhead Developer training Quick turnaround Little/no preparation Informal process Meetings No measurement Not Inspection! Inspections Detect and report all defects efficiently and effectively. Formal process Checklists Structured Meetings Measurements Verify phase Desk Checks Minimal overhead Quick turnaround Little/no preparation Informal process No measurement No Meetings Not Inspection! Copyright © 1998 Philip M. Johnson www.ics-hawaii.edu/~johnson used with permission

18. Thomas L. Gilchrist tomg@tomgtomg.com Initial Level Informal Checking Do Work Down- stream Customers Re-Work Input Source Materials Errors Found Process Product Entry Criteria Loosely Defined Exit Criteria For Quality Not Defined Work Process ENTRYENTRY EXITEXIT

19. Thomas L. Gilchrist tomg@tomgtomg.com Software Inspections

20. Thomas L. Gilchrist tomg@tomgtomg.com Measure and Improve Product Quality Do Work Down- stream Customers Re- Work Errors Found Process Product Crossfunctional Checklists Software Inspection Input Source Materials Entry Criteria Loosely Defined Exit Criteria For Quality Defined ENTRYENTRY EXITEXIT Work Process

21. Thomas L. Gilchrist tomg@tomgtomg.com Improve Product Quality and Reduce Variation Do Work Down- stream Customers Re- Work Errors Found Process Product Crossfunctional Checklists Input Source Materials Entry Criteria Defined Exit Criteria For Quality Defined ENTRYENTRY EXITEXIT Software Inspection Work Process

22. Thomas L. Gilchrist tomg@tomgtomg.com To Improve Product AND Process (Entry Criteria Met) Do Work Down- stream Customers Re- Work Input Source Materials Errors Found Process Product Crossfunctional Checklists Exit Criteria Met Software Inspection Repeatable Work Process

23. Thomas L. Gilchrist tomg@tomgtomg.com To Improve Product AND Process (Entry Criteria Met) Do Work Down- stream Customers Re- Work Input Source Materials Errors Found Process Product Crossfunctional Checklists Exit Criteria Met Statistical Data Analysis and Causal Analysis Software Inspection Repeatable Work Process

24. Thomas L. Gilchrist tomg@tomgtomg.com To Improve Product AND Process (Entry Criteria Met) Do Work Down- stream Customers Re- Work Input Source Materials Errors Found Process Product Crossfunctional Checklists Exit Criteria Met Statistical Data Analysis and Causal Analysis P D C A Deming Cycle (Defect Prevention Process) Software Inspection Repeatable Work Process

25. Thomas L. Gilchrist tomg@tomgtomg.com Inspection Characteristics Works for any kind of technical material - plans, specifications, reports, design representations, code, etc. Structured, rigorous, repeatable method of crossfunctional review. Works in "low process maturity" environments: Product Improvement Works in "high process maturity" environments: Process Improvement Indicated when the material is important

26. Thomas L. Gilchrist tomg@tomgtomg.com Software Inspections Software Inspection is a method for the improvement of technical materials with an eye to the identification and reporting of defects as early as possible. 50

27. Thomas L. Gilchrist tomg@tomgtomg.com Swimlane Process Tool The Team The Process Flow Solid flow lines go down. Dashed flow lines go up. R TeamFlow Process Tool

28. Thomas L. Gilchrist tomg@tomgtomg.com Swimlane Process Tool

29. Thomas L. Gilchrist tomg@tomgtomg.com Swimlane Process Tool

30. Thomas L. Gilchrist tomg@tomgtomg.com Swimlane Process Tool

31. Thomas L. Gilchrist tomg@tomgtomg.com Inspection Process Flow See Handout

32. Thomas L. Gilchrist tomg@tomgtomg.com Entry/Exit Materials Checklists –A list of criteria to facilitate the identification of major errors. –Crossfunctional in nature because contents of deliverable must address the entry-level materials as well as satisfy all the downstream customers. High-level Materials –The source materials entered into the process in which the deliverable was produced. Standards –Rules and / or guidelines established to improve the quality of a broad range of customer sets.

33. Thomas L. Gilchrist tomg@tomgtomg.com Inspection Materials Inspection Process Target Document

34. Thomas L. Gilchrist tomg@tomgtomg.com Inspection Materials Inspection Process Checklists Target Document

35. Thomas L. Gilchrist tomg@tomgtomg.com Inspection Materials Inspection Process High Level Documents Checklists Target Document

36. Thomas L. Gilchrist tomg@tomgtomg.com Inspection Materials High Level Documents Standards Checklists Target Document Inspection Process

37. Thomas L. Gilchrist tomg@tomgtomg.com Inspection Materials High Level Documents Standards Checklists Target Document Inspection Process Metrics

38. Thomas L. Gilchrist tomg@tomgtomg.com Inspection Process See Handout

39. Thomas L. Gilchrist tomg@tomgtomg.com Inspection Process See Handout

40. Thomas L. Gilchrist tomg@tomgtomg.com Software Inspection Entry Criteria oHas a moderator/team leader been identified and formally trained in the SW inspection technique? oHas all exit criteria found in the activity task description (process) been satisfied? oHave the necessary upstream documents and standards been identified and are they available? oIs the author confident that the deliverable is of high quality (he knows of no errors in the deliverable to be inspected).

41. Thomas L. Gilchrist tomg@tomgtomg.com oAre there qualified people available to serve on the inspection team? oDoes the authoring group supervision and management aware that they will not be able to participate in the inspection logging meetings (SW inspections are not to be used for evaluating people)? oIs there the possibility that there are major errors? oIs there a written process and rules for conducting the software inspection? Software Inspection Entry Criteria

42. Thomas L. Gilchrist tomg@tomgtomg.com Software Inspection Exit Criteria oDid the inspection team spend sufficient time finding major errors? oDid the inspection team spend sufficient time reporting and logging errors found (but not more that 2 hours per session)? oWere concerns and questions documented? oWas an individual or resource identified as a contact point for each question and/or issue? oWere action items documented and assigned?

43. Thomas L. Gilchrist tomg@tomgtomg.com oWas the current health of the inspection process indicated? oHave Action Items been resolved? oHave all errors been addressed by the author? oHas the inspection team decided on the disposition of the inspected deliverable (release or re-inspect)? Software Inspection Exit Criteria

44. Thomas L. Gilchrist tomg@tomgtomg.com Basic Rules of Software Inspection oNo meeting will last over 2 hours. oChecklist will be used to stimulate error detection. oThe emphasis is on finding and reporting major errors. oOne and only one person from the authoring group is on the inspection team. oFocus is on team effort in assisting the author to improve product (attacking the author is not permitted).

45. Thomas L. Gilchrist tomg@tomgtomg.com Basic Rules of Software Inspection oInspection team management/supervision does not participate in the inspection processes (inspections are not used to evaluate personnel). oA trained team leader (moderator) is used to facilitate the process and the meetings. oTeam members are trained and assigned specific “roles”. oRationale (i.e.,design/style) is not questioned or discussed during the logging meetings.

46. Thomas L. Gilchrist tomg@tomgtomg.com Basic Rules of Software Inspection oError detection is maximized by using optimum material coverage (i.e.,10-15 typewritten pages per 2 hour chunk). oThe number of major and minor errors, number of questions/concerns, the number of pages (or LOC) inspected, number of team members, and the time spent finding, reporting, and addressing errors will be collected for each chunk. oDocumented inspection rules are followed.

47. Thomas L. Gilchrist tomg@tomgtomg.com oIf there are more than 10-15 type written pages to be inspected, then the inspection will be divided into “chunks”. Basic Rules of Software Inspection

48. Thomas L. Gilchrist tomg@tomgtomg.com Roles and Responsibilities Moderator Author/Developer/Engineer/Producer Inspector(s) –Key inspector: if absent or unprepared, inspection meeting will be rescheduled. –Optional inspector: if absent or unprepared, inspection meeting will be held as scheduled. Scribe Reader –Key inspector Supervisors of authors are specifically excluded from participating in inspections but are kept informed of results.

49. Thomas L. Gilchrist tomg@tomgtomg.com Overview Meeting Distribution of Target Document Distribution of Checklists, Standards, High Level Documents Assignment and Clarification of Roles Special Instructions Understand Expectations of Moderator/Process Acceptance of Materials Agreement to Proceed ModeratorTeam

50. Thomas L. Gilchrist tomg@tomgtomg.com Individual Preparation ModeratorTeam Available for Questions Self-Study of Documentation (Up to 2 hours) Identify and Describe Errors and Questions (Individually)

51. Thomas L. Gilchrist tomg@tomgtomg.com Error Logging Meeting Moderator/Scribe*Team Record Prep Times* Facilitate Reporting of Errors Log Errors/Questions* "Playback" Error Log* Record Logging Times* Determine Disposition of Errors Kickoff Next Chunk Process Check* Report Prep Time Report and Classify Errors Build on Synergy Process Feedback