Dinesh Rawat , Software Test Manager Test Management & Test Strategy Best Practices Dinesh Rawat , Software Test Manager ABB

Abstract Different test tools are available in market today and the test techniques are no longer differentiators. How we manage and what strategy we use for better testing is the way forward. This presentation aims at: Highlighting few best practices we used. Share the knowledge gained from our experiences.

Best Practices Presentation Flow Test Planning User Story Discussion Testing Execution Monitoring Optimization Reporting Summary

Test Planning Best Practices 5 Year Test Improvement Goals

User Story Discussion Best Practices State Transition Diagrams State Transition Diagrams are used for better understanding of new functionalities to improve test case quality. Challenges Addressed: One liner User story, more discussions needed Invalid defects. Benefits: Good collaboration between development/TPM, test team & technical writers. Lesser invalid defects and review time. Source: Teach-ICT.com

Testing Best Practices Risk Based Testing Risk Based Testing is used to prioritize testing efforts based on Risk Priority Number (RPN). Challenges Addressed Late finding, Test quality, Time constraint. Test selection & Focus, Test coverage. Dependency on TPM and development – Impact analysis

Testing Best Practices Risk Based Testing

Risk Based Testing Continued

Execution Best Practices Documentation as part of test team Technical writer is included in the review of test descriptions, state transition diagrams and daily issue report. Challenges Addressed Technical writer had to wait till the end of the test cycle for product manual inputs & Manuals were delayed and were getting more comments. Competency level of the technical writer. Over dependency on Subject Matter Expert (SME).

Execution Best Practices Daily issues reporting During a test cycle(i.e. QTT/PTT), the update of the defect/issues are reported to the product and project team on a daily basis. Challenges Addressed Less number of deferred/known issues. Better dev/TPM/PM/Test collaboration, issues are highlighted at all levels at once and quality of development get more attention. Lesser invalid defects logged.

Monitoring Best Practices TFS(Team Foundation Server) Board TFS Board is a inbuilt feature in TFS web client used for tracking the development process of the requirements in real time. Not started - Work items development not yet started. Ready for test - - Work item ready for test. - Update product manuals for the new functionality. Tested - - Work item passed and working fine. - Review and approval of product manuals with release notes. Active - - Works items are taken into development. - Test team starts understanding the work item and prepares test description for the same. Starts creating State Transition Diagrams. - Test descriptions and State transition diagrams are verified and approved. - Test team prioritizes the work items based on Risk Based Testing Techniques. Failed - Failed work items. No run - Blocked tests due to dependency or some other reasons.

Monitoring Best Practices Following metrics can be used by Test team : TFS Reports Metrics Following metrics can be used by Test team : Customer cases count Defects by software area Deferred or Known issues Defect Classification Invalid Defects % Requirement Coverage Defect Removal Efficiency

Reporting Best Practices <Project Name> Test Project Report <Project Name> Overall Status G0 G1 G2 G3 G4 G5 Objective: Project Activity 1 Project Activity 2 Project Activity 3 ---------------------- Customer value/ABB value: Goals Improvements -------------------- Project Status (Monthly Progress) Testing Module 1 Testing Module 2 Testing Module 3 ----------------------- Plan for the next period (Next month) Upcoming testing module 1 Upcoming testing module 2 --------------------------- Deliverables Plan Actual Milestone 1 WkX WkA Milestone 2 WkY WkB Main Risks (Top 3): Priority Risk 1 Priority Risk 2 Gate Status: Plan Actual G2 Wk-- Wk-- G5 Wk-- Wk-- Project KPI’s: Target Actual Defect Validity >80% xx Test Effectiveness >80% xx Post release defects <10% xx Product Quality: Cost (KUSD): Budget Forecast Actual XX K USD xx K USD xx K USD Time (Milestones) Plan Actual Milestone 1 WkX WkA Milestone 2 WkY WkB Resources: BudgetFY YTD Resources xx.y x.yy Productivity Initiative Goal vs Actions Targets from 5 Years Plan Project Manager: <Project Manager Name> CHTET: <CHTET No> Month: Month Legend changes since last report To be updated for individual Project

Optimization Best Practices Value Stream Mapping A technique used to Analyze the current Workflow, Identify Non-Required/ repetitive activities and Develop Action plan to bring a better/optimized workflow. Identify the steps where no real value is added and eliminate/optimize these types of activities. Develop an Action plan to implement Future State VSM which may include various Improvement initiatives like best practices, Test Optimization Ideas, Test Automation etc. E.g. Risk Based Testing for Test Execution

Summary Earlier - Testing 3 products at a time, Description Action Result Test Planning Stream lining Productivity improvement ~25%. User Story Discussion Clear and elaborate user stories Faster time to market ~ 30%. Testing & Execution Implemented Risk Based Testing Faster time to market ~20% Monitoring, Optimization & Reporting Effective planning and structuring Reduced COPQ ~20% Earlier - Testing 3 products at a time, Now - Testing 5 products with same team & time. Cycle time - Reduced to half approx.

Thank You!!!