1. Tom Gilchrist, CQA, CSQE tomg@halcyon.com Quality Assurance in ISD and Maintenance Projects How do you do QA when the time it takes is longer that the time to do the work?

2. 10/13/00.1 tomg@halcyon.com Disclaimers The views and opinions expressed in this presentation are not necessarily those of my employer. No financial connection with vendors mentioned. Your results may vary.

3. 10/13/00.1 tomg@halcyon.com Improvement PROCESS Major determinants of product cost, schedule, and quality performance.

4. 10/13/00.1 tomg@halcyon.com Improvement PROCESSTECHNOLOGY Major determinants of product cost, schedule, and quality performance.

5. 10/13/00.1 tomg@halcyon.com Improvement PEOPLE PROCESSTECHNOLOGY Major determinants of product cost, schedule, and quality performance.

6. 10/13/00.1 tomg@halcyon.com Goals and Objectives Understand…. How ISD and maintenance SDLC work. How to Do SQA in ISD Projects Deployment Sharpen Your SQA Skills

7. 10/13/00.1 tomg@halcyon.com Some Level Setting What is Quality? What is SQA? What are Peer Reviews? What is a “Customer”? What are the goals of SQA?

8. 10/13/00.1 tomg@halcyon.com Low Quality in Software Unneeded Complexity Rework -Wrong -Unclear -Missing -Extra

9. 10/13/00.1 tomg@halcyon.com Software Quality Assurance Check software products and processes to verify that they comply with the applicable procedures and standards. (Process Checks/Audits) Review and measure the quality of software products and processes throughout development. (Peer Reviews) Provide software project management (and other appropriate parties) with the results of reviews and process checks. Work with the software project during early stages to establish plans, standards, and procedures to keep errors from occurring in the first place.

10. 10/13/00.1 tomg@halcyon.com Peer Review Methods Walkthroughs Minimal overhead Developer training Quick turnaround MethodsTypical GoalsTypical Attributes Little/no preparation Informal process Meetings No measurement Not a Formal Review! Formal Reviews Detect and report all defects efficiently and effectively. Formal process Known Coverage Rate Moderator Checklists Customer Reviewers Measurement Desk Checks Minimal overhead Quick turnaround Little/no preparation Informal process No measurement No Meetings Not a Formal Review!

11. 10/13/00.1 tomg@halcyon.com SQA Plan…Peer Reviews (exit criteria)

12. 10/13/00.1 tomg@halcyon.com Customer vs. End User End User: -The person who uses the software product in the value stream Customer: -Anyone who is affected by the end product or intermediate software engineering deliverables being produced (reviewed).

13. 10/13/00.1 tomg@halcyon.com Software Development Model ENTRYENTRY EXITEXIT Input Output ENTRYENTRY EXITEXIT Input Program Element Creation Program Element Creation Standards, templates, rules and checklists Standards, templates, rules and checklists Time

14. 10/13/00.1 tomg@halcyon.com Exit Criteria Cost Schedule Quality ENTRYENTRY EXITEXIT Input Output ENTRYENTRY EXITEXIT Input Program Element Creation Program Element Creation Standards, templates, rules and checklists Standards, templates, rules and checklists

15. 10/13/00.1 tomg@halcyon.com How SDLC’s Work Waterfall SDLC Waterfall SQA Techniques Maintenance & Sustaining SDLC ISD SDLC

16. 10/13/00.1 tomg@halcyon.com Waterfall SDLC Feasibility Analysis Requirements Definition Alternatives Analysis Functional Specification Preliminary Design Detailed Design & Construction Verification Installation Software Development Lifecycle

17. 10/13/00.1 tomg@halcyon.com Waterfall SDLC Feasibility Analysis Requirements Definition Alternatives Analysis Functional Specification Preliminary Design Detailed Design & Construction Verification Installation Software Development Lifecycle All planning is oriented toward a single delivery date. All analysis and design are done before coding and testing. Model or document driven.

18. 10/13/00.1 tomg@halcyon.com The Project Requirements Finished Product SpecsDesignCodeTest Time Project Plan

19. 10/13/00.1 tomg@halcyon.com When Waterfall Works Solution is well understood. Developers are well practiced. End-user process being automated is stable and well understood. Mature project management.

20. 10/13/00.1 tomg@halcyon.com Idealized Waterfall Sequence Time Requirements Design Code Requirements Design Code Managed by intermediate software engineering deliverable

21. 10/13/00.1 tomg@halcyon.com Development Models The new revolution BANG! The old system exists New system being developed Adopted from Tom Gilb, Principles of Software Engineering Management

22. 10/13/00.1 tomg@halcyon.com Development Models The new revolution BANG! The old system exists New system being developed Old system New System Adopted from Tom Gilb, Principles of Software Engineering Management

23. 10/13/00.1 tomg@halcyon.com The “Plan” Usable Results POC

24. 10/13/00.1 tomg@halcyon.com The “Result” Usable Results POC Inc 1

25. 10/13/00.1 tomg@halcyon.com The “Final Results” Inc 1 Inc 2 Inc 3 Inc n Inc n+1 POC

26. 10/13/00.1 tomg@halcyon.com Phased Development Feasibility Analysis Requirements Definition Alternatives Analysis Functional Specification Preliminary Design Detailed Design & Construction Verification Installation Software Released Incrementally Detailed Design & Construction Verification Installation Detailed Design & Construction Verification Installation Construction Release 1 Construction Release 2 Construction Release 3 This is NOT ISD

27. 10/13/00.1 tomg@halcyon.com ISD Phases Approval Proof of Concept Increment

28. 10/13/00.1 tomg@halcyon.com ISD SDLC Software Development Lifecycle ApprovalPlan Dev Initial Product Propose Review Grow Product Plan APRINCPOCINC Review Propose Grow Product Plan Review Propose Proof-of-ConceptIncrement 2Increment 1 Plan DoCheck Act POC INC 2 INC 1

29. 10/13/00.1 tomg@halcyon.com Idealized ISD/Sustaining Sequence INC Time RequirementsDesignCode INC Incremental Development or Sustaining Original Waterfall Development Triggered by CR’s and/or PR’s

30. 10/13/00.1 tomg@halcyon.com INCPOCINC POC Real-World ISD/Sustaining How can one spend the flow time necessary to do Peer Reviews when the flow time to do them is sometimes longer than the increment?

31. 10/13/00.1 tomg@halcyon.com SQA in ISD Projects Tools of SQA Strategies & Goals Process vs. Product Creative SQA Opportunities

32. 10/13/00.1 tomg@halcyon.com Peer Review Methods Walkthroughs Minimal overhead Developer training Quick turnaround MethodsTypical GoalsTypical Attributes Little/no preparation Informal process Meetings No measurement Not a Formal Review! Formal Reviews Detect and report all defects efficiently and effectively. Formal process Known Coverage Rate Moderator Checklists Customer Reviewers Measurement Desk Checks Minimal overhead Quick turnaround Little/no preparation Informal process No measurement No Meetings Not a Formal Review!

33. 10/13/00.1 tomg@halcyon.com Peer Reviews ENTRYENTRY EXITEXIT Output... ENTRYENTRY EXITEXIT Output Input Standards, templates, rules and checklists Standards, templates, rules and checklists Walkthrough Desk Check Walkthrough Desk Check Program Element Creation Program Element Creation Formal Review Formal Walkthrough Formal Desk Check Formal Review Formal Walkthrough Formal Desk Check Formal Review Formal Walkthrough Formal Desk Check Formal Review Formal Walkthrough Formal Desk Check Walkthrough Desk Check Walkthrough Desk Check Kickoff Readiness Kickoff Readiness Kickoff Readiness Kickoff Readiness

34. 10/13/00.1 tomg@halcyon.com SQA Interventions Output ENTRYENTRY EXITEXIT Input Program Element Creation StartDuringEnd Kickoff Meeting Readiness Review

35. 10/13/00.1 tomg@halcyon.com SQA Interventions Output ENTRYENTRY EXITEXIT Input Program Element Creation StartDuringEnd Walkthrough Desk Check Kickoff Meeting Readiness Review

36. 10/13/00.1 tomg@halcyon.com SQA Interventions Output ENTRYENTRY EXITEXIT Input Program Element Creation StartDuringEnd Kickoff Meeting Readiness Review Walkthrough Desk Check Formal Review Formal Walkthrough Formal Desk Check

37. 10/13/00.1 tomg@halcyon.com Software Development Model ENTRYENTRY EXITEXIT Input Output ENTRYENTRY EXITEXIT Input Program Element Creation Program Element Creation Standards, templates, rules and checklists Standards, templates, rules and checklists Time

38. 10/13/00.1 tomg@halcyon.com Fan-In vs Fan-out

39. 10/13/00.1 tomg@halcyon.com Fan-In vs Fan-out

40. 10/13/00.1 tomg@halcyon.com Fan-In vs Fan-out

41. 10/13/00.1 tomg@halcyon.com Readiness Review Identify people/organizations producing product for the process Identify the people/organization who will do the work Identify and document potential major errors (build/modify checklists) Confirm task/process entry conditions Identify resources Forum for learning Reduce the amount of time needed by the author(s) to produce a deliverable by making sure things are ready to do the job.

42. 10/13/00.1 tomg@halcyon.com Generic Document Worklist Before creating the deliverable... -Hold Kickoff Meeting Invite the right participants Plan for a successful document Build/Refine Checklists While working on the deliverable… -Use the checklists -Use Walkthroughs and Desk Checks as needed to reduce rework (major errors). When finished with the deliverable… -As per SQA plan, perform the appropriate Peer review -Send peer review data summary to SQA focal

43. 10/13/00.1 tomg@halcyon.com CR or PR Maintenance CR or PR

44. 10/13/00.1 tomg@halcyon.com Approved CR or PR Approved CR or PR Maintenance Approved CR or PR

45. 10/13/00.1 tomg@halcyon.com Approved CR or PR Targeted to a Release Approved CR or PR Approved CR or PR Maintenance Approved CR or PR Approved CR or PR Targeted to a Release Approved CR or PR Targeted to a Release

46. 10/13/00.1 tomg@halcyon.com Maintenance Change Approved CR or PR Targeted to a Release Approved CR or PR Targeted to a Release Integrated Test

47. 10/13/00.1 tomg@halcyon.com SQA Sustaining/ISD Strategies Ignore On the Schedule -No Change -Demotion -Creative Techniques -Sampling Off the Schedule -Sampling with Process Improvement

48. 10/13/00.1 tomg@halcyon.com Demotion Trigger for new/first creation Peer Review Define less rigorous Peer Review for Updates to deliverables Tracking/Reporting System

49. 10/13/00.1 tomg@halcyon.com SQA Plan…Peer Reviews (Exit Criteria)

50. 10/13/00.1 tomg@halcyon.com Creative Techniques Demo Peer Reviews Risk Assessment Others...

51. 10/13/00.1 tomg@halcyon.com Deliverable Maintenance Output Standards, templates, Rules and Checklists Program Element Modification ENTRYENTRY EXITEXIT Program Element Documentation Input

52. 10/13/00.1 tomg@halcyon.com Maintenance Approved CR or PR

53. 10/13/00.1 tomg@halcyon.com Maintenance Requirements Approved CR or PR

54. 10/13/00.1 tomg@halcyon.com Maintenance RequirementsDesign Approved CR or PR

55. 10/13/00.1 tomg@halcyon.com Maintenance RequirementsDesignCode Approved CR or PR

56. 10/13/00.1 tomg@halcyon.com Maintenance RequirementsDesignCode Approved CR or PR Integrated Tests

57. 10/13/00.1 tomg@halcyon.com RequirementsDesignCode Approved CR or PR Integrated Tests Demo Walkthrough Maintenance

58. 10/13/00.1 tomg@halcyon.com “Demo” Peer Reviews Attendees: -Authors with changed documents -Customer requesting change -Downstream Customers (like testing, data conversion, implementation, etc) Process Used: Walkthrough or Formal Review Trigger: Ready for Test

59. 10/13/00.1 tomg@halcyon.com Sampling On Schedule Randomness Timing of Decision Type of Peer Review Risk

60. 10/13/00.1 tomg@halcyon.com Sample With Improvement INC Design Doc & Checklist INC Design Doc & Checklist Design Doc & Checklist

61. 10/13/00.1 tomg@halcyon.com Sample With Improvement INC Design Doc & Checklist INC Formal Review Design Doc & Checklist Design Doc & Checklist

62. 10/13/00.1 tomg@halcyon.com Sample With Improvement INC Design Doc & Checklist INC Address Errors Found? Design Doc & Checklist Design Doc & Checklist Formal Review

63. 10/13/00.1 tomg@halcyon.com Sample With Improvement INC Design Doc & Checklist INC Process Improvement (Defect Prevention Process) Address Errors Found? Design Doc & Checklist Design Doc & Checklist Formal Review

64. 10/13/00.1 tomg@halcyon.com Sample With Improvement INC Design Doc & Checklist INC Process Improvement (Defect Prevention Process) Address Errors Found? Design Doc & Checklist Design Doc & Checklist Formal Review

65. 10/13/00.1 tomg@halcyon.com Benefits... Can be “off schedule” Reduce reliance on 100% inspection Manageable Measure Results -Development capability -Improvement capability

66. 10/13/00.1 tomg@halcyon.com Possible Triggers Number or Major Errors found. Testing Results Calendar Customer Feedback

67. 10/13/00.1 tomg@halcyon.com Beware of “Open Secrets" - these indicate goal “stated” vs “actual” misalignments. Doing things right is punished Nobody takes notice Doing it wrong is rewarded How to Fail...Impede Performance

68. 10/13/00.1 tomg@halcyon.com Goals and Objectives Understand…. How ISD and maintenance SDLC work. How to Do SQA in ISD Projects Deployment Sharpen Your SQA Skills

69. 10/13/00.1 tomg@halcyon.com