Thomas L. Gilchrist Testing Basics Set 5: CM & Peer Reviews By Thomas L. Gilchrist CSQE,CSQA 2009

Thomas L. Gilchrist SCCM Software Change & Configuration Management

Thomas L. Gilchrist SCCM (software change and configuration management) Basics Identification: How does and organization identify and manage the many existing versions of a program and its documentation in a manner that will enable change to be accommodated efficiently? Control: How does an organization control changes before and after the software (and supporting documentation) is released to the customer? Status Accounting: Who has responsibility for approving and ranking changes Verification: How can we ensure that changes have been made properly? Reporting: What mechanism is used to appraise others of changes that have been made (or being made). Roger Pressman…”Software Engineering, Fifth Edition”

Thomas L. Gilchrist CM High Level View Project CM Repository SW Eng Tasks New SCI Tech Review Approved SCI SCM Controls SCI = Software Configuration Item

Thomas L. Gilchrist CM High Level View Project CM Repository SW Eng Tasks New SCI Tech Review Approved SCI SCM Controls Read Only Copy Ver 1 SCI Ver 1 SCI SCI = Software Configuration Item Ver 1 SCI Ver 1 SCI Project CM Repository Tech Review

Thomas L. Gilchrist CM High Level View Project CM Repository SCM Controls Request For Change Read Only Copy Ver 1 Ver 1 Ver 1 SCI Ver 1 SCI Ver 1 SCI Ver 1 SCI Project CM Repository

Thomas L. Gilchrist CM High Level View Project CM Repository SW Eng Tasks Current Baseline SCI Approved to Change SCM Controls Request For Change Read Only Copy Ver 1 Ver 1 Ver 1 SCI Ver 1 SCI Ver 1 SCI Ver 1 SCI Project CM Repository

Thomas L. Gilchrist CM High Level View Project CM Repository SW Eng Tasks SCM Controls Request For Change Read Only Copy Ver 1 Ver 1 Ver 1 SCI Ver 1 SCI Ver 1 SCI Ver 1 SCI Current Baseline SCI Approved to Change Project CM Repository

Thomas L. Gilchrist CM High Level View Project CM Repository SW Eng Tasks Modified SCI Tech Review SCM Controls SCM Controls Request For Change Read Only Copy Ver 1 Ver 1 Approved SCI Ver 1 SCI Ver 1 SCI Ver 1 SCI Ver 1 SCI Current Baseline SCI Approved to Change Project CM Repository Tech Review

Thomas L. Gilchrist CM High Level View Project CM Repository SW Eng Tasks Tech Review SCM Controls SCM Controls Request For Change Read Only Copy Modified SCI Approved SCI ? ? Ver 2 SCI Ver 2 SCI Current Baseline SCI Approved to Change Project CM Repository Tech Review

Thomas L. Gilchrist Peer Reviews

Thomas L. Gilchrist Reviews Vs. Testing Dynamic Vs. Static Testing Peer Reviews can be done before code can be tested. Testing evaluates product while performing function in target environment. Not mutually exclusive

Thomas L. Gilchrist Peer Reviews Intended as a method of static verification for software by the producers’ peers to identify defects and areas where changes are needed. Method is not intended to replace dynamic testing, but rather to augment it.

Thomas L. Gilchrist What Peer Review Is Not A “silver bullet.” A tool for personnel evaluation. A quality assurance program.

Thomas L. Gilchrist Static Verification Techniques Desk Checking Walkthroughs Formal Technical Reviews Inspection (Fagan) Management Reviews Audits Static Program Analysis Formal (Mathematical) Verification

Thomas L. Gilchrist Peer Review Methods Walkthroughs MethodsTypical GoalsTypical Attributes Minimal overhead Developer training Quick turnaround Little/no preparation Informal process Meetings No measurement Not Inspection! Inspections Detect and report all defects efficiently and effectively. Formal process Checklists Structured Meetings Measurements Verify phase Desk Checks Minimal overhead Quick turnaround Little/no preparation Informal process No measurement No Meetings Not Inspection! Copyright © 1998 Philip M. Johnson used with permission

Thomas L. Gilchrist Initial Level Informal Checking Do Work Down- stream Customers Re-Work Input Source Materials Errors Found Process Product Entry Criteria Loosely Defined Exit Criteria For Quality Not Defined Work Process ENTRYENTRY EXITEXIT

Thomas L. Gilchrist Software Inspections

Thomas L. Gilchrist Measure and Improve Product Quality Do Work Down- stream Customers Re- Work Errors Found Process Product Crossfunctional Checklists Software Inspection Input Source Materials Entry Criteria Loosely Defined Exit Criteria For Quality Defined ENTRYENTRY EXITEXIT Work Process

Thomas L. Gilchrist Improve Product Quality and Reduce Variation Do Work Down- stream Customers Re- Work Errors Found Process Product Crossfunctional Checklists Input Source Materials Entry Criteria Defined Exit Criteria For Quality Defined ENTRYENTRY EXITEXIT Software Inspection Work Process

Thomas L. Gilchrist To Improve Product AND Process (Entry Criteria Met) Do Work Down- stream Customers Re- Work Input Source Materials Errors Found Process Product Crossfunctional Checklists Exit Criteria Met Software Inspection Repeatable Work Process

Thomas L. Gilchrist To Improve Product AND Process (Entry Criteria Met) Do Work Down- stream Customers Re- Work Input Source Materials Errors Found Process Product Crossfunctional Checklists Exit Criteria Met Statistical Data Analysis and Causal Analysis Software Inspection Repeatable Work Process

Thomas L. Gilchrist To Improve Product AND Process (Entry Criteria Met) Do Work Down- stream Customers Re- Work Input Source Materials Errors Found Process Product Crossfunctional Checklists Exit Criteria Met Statistical Data Analysis and Causal Analysis P D C A Deming Cycle (Defect Prevention Process) Software Inspection Repeatable Work Process

Thomas L. Gilchrist Inspection Characteristics Works for any kind of technical material - plans, specifications, reports, design representations, code, etc. Structured, rigorous, repeatable method of crossfunctional review. Works in "low process maturity" environments: Product Improvement Works in "high process maturity" environments: Process Improvement Indicated when the material is important

Thomas L. Gilchrist Software Inspections Software Inspection is a method for the improvement of technical materials with an eye to the identification and reporting of defects as early as possible. 50

Thomas L. Gilchrist Swimlane Process Tool The Team The Process Flow Solid flow lines go down. Dashed flow lines go up. R TeamFlow Process Tool

Thomas L. Gilchrist Swimlane Process Tool

Thomas L. Gilchrist Swimlane Process Tool

Thomas L. Gilchrist Swimlane Process Tool

Thomas L. Gilchrist Inspection Process Flow See Handout

Thomas L. Gilchrist Entry/Exit Materials Checklists –A list of criteria to facilitate the identification of major errors. –Crossfunctional in nature because contents of deliverable must address the entry-level materials as well as satisfy all the downstream customers. High-level Materials –The source materials entered into the process in which the deliverable was produced. Standards –Rules and / or guidelines established to improve the quality of a broad range of customer sets.

Thomas L. Gilchrist Inspection Materials Inspection Process Target Document

Thomas L. Gilchrist Inspection Materials Inspection Process Checklists Target Document

Thomas L. Gilchrist Inspection Materials Inspection Process High Level Documents Checklists Target Document

Thomas L. Gilchrist Inspection Materials High Level Documents Standards Checklists Target Document Inspection Process

Thomas L. Gilchrist Inspection Materials High Level Documents Standards Checklists Target Document Inspection Process Metrics

Thomas L. Gilchrist Inspection Process See Handout

Thomas L. Gilchrist Inspection Process See Handout

Thomas L. Gilchrist Software Inspection Entry Criteria oHas a moderator/team leader been identified and formally trained in the SW inspection technique? oHas all exit criteria found in the activity task description (process) been satisfied? oHave the necessary upstream documents and standards been identified and are they available? oIs the author confident that the deliverable is of high quality (he knows of no errors in the deliverable to be inspected).

Thomas L. Gilchrist oAre there qualified people available to serve on the inspection team? oDoes the authoring group supervision and management aware that they will not be able to participate in the inspection logging meetings (SW inspections are not to be used for evaluating people)? oIs there the possibility that there are major errors? oIs there a written process and rules for conducting the software inspection? Software Inspection Entry Criteria

Thomas L. Gilchrist Software Inspection Exit Criteria oDid the inspection team spend sufficient time finding major errors? oDid the inspection team spend sufficient time reporting and logging errors found (but not more that 2 hours per session)? oWere concerns and questions documented? oWas an individual or resource identified as a contact point for each question and/or issue? oWere action items documented and assigned?

Thomas L. Gilchrist oWas the current health of the inspection process indicated? oHave Action Items been resolved? oHave all errors been addressed by the author? oHas the inspection team decided on the disposition of the inspected deliverable (release or re-inspect)? Software Inspection Exit Criteria

Thomas L. Gilchrist Basic Rules of Software Inspection oNo meeting will last over 2 hours. oChecklist will be used to stimulate error detection. oThe emphasis is on finding and reporting major errors. oOne and only one person from the authoring group is on the inspection team. oFocus is on team effort in assisting the author to improve product (attacking the author is not permitted).

Thomas L. Gilchrist Basic Rules of Software Inspection oInspection team management/supervision does not participate in the inspection processes (inspections are not used to evaluate personnel). oA trained team leader (moderator) is used to facilitate the process and the meetings. oTeam members are trained and assigned specific “roles”. oRationale (i.e.,design/style) is not questioned or discussed during the logging meetings.

Thomas L. Gilchrist Basic Rules of Software Inspection oError detection is maximized by using optimum material coverage (i.e.,10-15 typewritten pages per 2 hour chunk). oThe number of major and minor errors, number of questions/concerns, the number of pages (or LOC) inspected, number of team members, and the time spent finding, reporting, and addressing errors will be collected for each chunk. oDocumented inspection rules are followed.

Thomas L. Gilchrist oIf there are more than type written pages to be inspected, then the inspection will be divided into “chunks”. Basic Rules of Software Inspection

Thomas L. Gilchrist Roles and Responsibilities Moderator Author/Developer/Engineer/Producer Inspector(s) –Key inspector: if absent or unprepared, inspection meeting will be rescheduled. –Optional inspector: if absent or unprepared, inspection meeting will be held as scheduled. Scribe Reader –Key inspector Supervisors of authors are specifically excluded from participating in inspections but are kept informed of results.

Thomas L. Gilchrist Overview Meeting Distribution of Target Document Distribution of Checklists, Standards, High Level Documents Assignment and Clarification of Roles Special Instructions Understand Expectations of Moderator/Process Acceptance of Materials Agreement to Proceed ModeratorTeam

Thomas L. Gilchrist Individual Preparation ModeratorTeam Available for Questions Self-Study of Documentation (Up to 2 hours) Identify and Describe Errors and Questions (Individually)

Thomas L. Gilchrist Error Logging Meeting Moderator/Scribe*Team Record Prep Times* Facilitate Reporting of Errors Log Errors/Questions* "Playback" Error Log* Record Logging Times* Determine Disposition of Errors Kickoff Next Chunk Process Check* Report Prep Time Report and Classify Errors Build on Synergy Process Feedback