3 Potential Response The job of testing is to find defects.
4 Myth Exposed Myth: The Job of Testing Is to Find Defects 1 1 The Seven Principles of Lean Software Development by Mary Poppendieck
5 Quality is everyones responsibility Myth: The Job of Testing Is to Find Defects The job of tests, and the people that develop and run tests, is to prevent defects, not find them. A quality assurance organization should champion processes that build quality into the code from the start rather than test quality in later. 1 1 The Seven Principles of Lean Software Development by Mary Poppendieck
6 At a recent software seminar The common attributes of successful software projects are Heroes Chaos Re-work Untracked OT
7 Todays Discussion Case Study – 4 projects 2 Definitions Testing Metrics (what to collect & analyze) Metrics from first 2 projects Process Improvement Steps Success Story – Projects 3 & 4 Some trends & conclusions
8 Project Background Legacy mature system (mainframe) Industry regulatory enhancements Time-boxed: 3 to 4 months Waterfall SDLC First 2 Projects Project 1 – Dec09 – 1X Project 2 – June10 – 2.3X
9 Definition: Re-work Re-work – AD time to fix and testing time to report and re-test System Testing and Production defects
10 Definition: Static Testing Static Testing - Testing of a component or system at specification or implementation level without execution of that software, e.g., reviews or static code analysis.
11 Static vs. Dynamic Testing From System Testing with an Attitude, p.14, by Nathan Petschenik
12 Testing Defect Analysis When was the defect introduced (SDLC phase) When was the defect discovered (testing phase)
25 Process Improvement Steps 1. Developed framework in Quality Center to track COR 2. Presentations to business partners & executives 3. Lively discussion during presentations 4. AD appointed a new AD Lead 5. AD improved their internal code review 6. More emphasis on and participation during reviews, walk-throughs – Static Testing
26 Process Improvement Steps 7. More collaboration between AD and testing teams 8. Increased emphasis on unit testing 9. Risk-based testing approach 10. Improved regression work 11. Staff increased in domain knowledge
33 Defect Discovery Rate DDR = % of Total Defects discovered during an SDLC or Test Phase Static Testing System Testing Transition Goal - Trying to find out if we are finding defects earlier.
34 DDR Trends ----------------Coding----------------- As DDR for Static Testing increases, DDR for System Testing and Production Defects decreases
35 When is Unit Testing complete? A necessary criterion for completion of developer testing is that the developers believe that the software is ready to be delivered to the users on the day it is delivered to the system-test team. From System Testing with an Attitude, p. 33, by Nathan Petschenik
36 Todays Discussion Defect metrics – powerful story Static Testing works It is a collaborative effort The positive effect ripples into Unit Testing, System Testing, UAT, and implementation No shortcuts here – influence your project to do it right
37 Some Final Thoughts Training needed for our staff (testing, AD, business, PM community) - how to be more effective in document reviews and walk-throughs. Perspectives Tester – static testing > a paradise (a new maturity level)