1. 5 Feb 2008 Roberta Allsman LSST Corporation LSST Data Management DC2 Post-mortem

2. 2 Software Quality Control Transitioning from DC2 to DC3 Software Quality Control Introduction DC2: The Way It Was DC2: The Regrets and Promise of Change DC2 Transition to DC3

3. 3 Software Quality Control (SQC) SQC –Verifies thru reviews: deliverables satisfy standards –Validates thru testing: deliverables satisfy requirements Standards and Requirements include Product requirements specification (use case driven) Design document (UML) Coding, Documentation and Testing standards

4. 4 DC2 SQC: The Way It Was Requirements, Standards, Policies –Project requirements and science use cases –Design standards (DM UML model and DC2 specialization) –Coding standards (language, best practices, layout) –Documentation standards (in-line) –Toolset policy (sanctioned support tools & nominations of) –Product release policy

5. 5 DC2 SQC: The Way It Was Quality Control Procedures –Change Control Board reviewed need and feasibility of feature additions –Code Review Procedure Defined responsibilities of developer and code reviewer Specified task workflow –Tasks were not forgotten and progressed to completion Specified first validation testing: unittest

6. 6 DC2 SQC: The Way It Was Support –Software Developers Start-up Guide Training for new developers in setting up LSST DM installation –Component Design Overviews Background implementation overview –Trac web-based software project management and issue tracking system

7. 7 DC2 SQC: The Way It Was TRAC WIKI Web-based Document Manager –Easy to use –Enabled issues to be recorded and debated –Made project decisions/mandates readily accessible Ticket Management –Current state of each issue (aka ticket) readily available –At-a-glance summary reports based on tickets attributes Source Browser –On-line browsing of source and its change history

8. 8 DC2: The Regrets and Promise of Change Standards Code Review Procedure defined a workflow which created a bottleneck at point of design resynchronization with implementation. A proposed DC3 workflow alleviates this. Unit test requirements have not been adequately documented. Uniform naming syntax for DB columns is missing. Code Review As a group, code reviewers were inconsistent in application of language and documentation requirements. Using an automated standards checker would simplify review process.

9. 9 DC2: The Regrets and Promise of Change Trac WIKI Add new V0.11 blog feature to foster copy-cat of Becker's popular 'how-did' blog WIKI's intrinsic flat hierachy leads to orphaned pages –Concept: Modify 'New Page' function to require user to enter page locator into Table-of-Contents page SQC A testbed needs to be setup on each supported hardware configuration for integration tests

10. 10 DC2: The Regrets and Promise of Change Source Management Remove superfluous named branch: BRANCHES UML/source sync is too manpower intensive –Change source layout to be 'better' for EA –Define uniform naming of swig generated file – Setup EA user environment for consistent source location Testing More formalized testing was needed –Will be addressed in DC3 SQC Standards upgrade

11. 11 DC2: The Regrets and Promise of Change Trac Ticket Management More numerous DC2 milestones were needed –Define finer grained milestones –Accurately assign tickets to milestones –Generate reports showing ticket/milestone progress Ticket workflow wasn't detailed enough –New Trac V0.11 has flexible workflow definition

12. 12 DC3 transition: Upgrade Standards Documents Select/develop new use cases to define requirements Complete/revise list of configurations to support Develop Test Standards document Specify categories of testing required Specify which testing categories are required for –No-guarantees release (TICKETS branch) –Beta release (TRUNK branch) –Production release (TAGS branch) Refine development lifecycle (aka workflow) to include new testing

13. 13 DC3 transition: Upgrade SQC Procedures Acquire configurable language and coding standards checker Examples: ProQA, CodeCheck, Apex, QStudio Develop new workflow procedure document Develop procedures for each new testing category Use Trac project management features more effectively Track ticket time to resolution Track milestone progress Assign priorities to tickets based on milestones & dependencies Expand ticket tracking workflow

14. 14 DC3 transition: Preliminary DC3 Workflow Change Review Board Requirements review for new functionality Task assignment Implementation in TICKETS branch Software development Code review Language/documentation/best practices review Testing Unit test, Component test

15. 15 DC3 transition: Preliminary DC3 Workflow Merge from TICKETS into TRUNK branch Conflict resolution due to source merge Testing Unit test, Component test, Integration test Copy from TRUNK into TAGS branch Testing End-to-End test, Platform test, Distribution test, Stress test Synchronize UML design with implementation Release of verified and validated product