1. Effective Test Planning: Scope, Estimates, and Schedule Presented By: Shaun Bradshaw shaun.bradshaw@zenergytechnologies.com

2. Agenda Why Plan? Whats in a test plan? Define – Objectives – Scope – Priorities Estimate work effort Develop schedule Software Peace of Mind in ActionSlide 2

3. Why Plan? Planning is imperfect Involves assumptions & estimates Too many variables to control But… Allows us to think about what we need Determine our abilities & constraints Develop roadmap for activities And… We are told to do it Software Peace of Mind in ActionSlide 3

4. Whats in a Test Plan? Most test plan templates available are derived from IEEE standards The template presented here is a minimalist version of that template Generally, a test plan includes: – Objectives – Scope – Risks & Contingencies – Focus / Priorities – Work Effort Estimation – Go/No Go Criteria Software Peace of Mind in ActionSlide 4

5. Sections of the Document Software Peace of Mind in ActionSlide 5

6. Software Peace of Mind in ActionSlide 6 Sections of the Document

7. Software Peace of Mind in ActionSlide 7 Sections of the Document

8. Software Peace of Mind in ActionSlide 8 Sections of the Document

9. Software Peace of Mind in ActionSlide 9 Sections of the Document

10. Test Objectives Test Objectives should be based on the needs of the business and established organizational processes & goals – Determine short-term and long-term business needs – Is the business most concerned about qualitative or quantitative objectives – How can our test objectives help the business achieve its objectives – Test plan objectives should tie into business objectives Software Peace of Mind in ActionSlide 10

11. Lets identify test plan objectives related to each of the following business objectives Increase customer satisfaction with the applicationImprove market reputation and industry recognitionReduce the number of user calls to the help deskEnsure key functionality is delivered on time Software Peace of Mind in ActionSlide 11 Test Objectives

12. Scope Identify the scope and how the test effort will allow the business to meet its objectives – Determine functionality, processes, and systems to be validated – Establish types of testing necessary to perform the validation – Identify supporting systems that should be in scope – if not in scope then document it – Read available documentation – identify potential anomalies – Examine existing applications & processes – Get involved EARLY!! Software Peace of Mind in ActionSlide 12

13. Risk Risk is the probability of an undesirable event occurring and its subsequent impact to a project Software Peace of Mind in ActionSlide 13

14. Types of Risk Technical Risk Do available resources have the appropriate technical or business knowledge to effectively perform testing? Environmental Risk Is there an adequate test environment? Are there appropriate controls and access restrictions? Cost Risk Do sufficient funds exist to cover the costs necessary to complete the testing? Schedule Risk Is there sufficient time to complete testing activities? Software Peace of Mind in ActionSlide 14

15. Contingency Planning Because risks tend to have a domino effect, develop contingency plans for each Contingency should align with the probability and cost of the risk Contingency should address the impact of the risk Software Peace of Mind in ActionSlide 15

16. Prioritization Prioritization of application components assists in determining what functionality has precedence when organizing the test effort. Creates a systematic approach for identifying what to test first, what can be minimally tested, and what can be eliminated from testing, if necessary Determine criticality and impact of application components based on discussions with SMEs Software Peace of Mind in ActionSlide 16

17. Prioritization Consider the following factors for prioritization Customer Needs Customer Use Monetary Value Interdependencies Historical Fail Rates Software Peace of Mind in ActionSlide 17

18. Risk/Impact Analysis Use a Prioritization Table to determine which components have the highest priority – Set Risk value (probability of failure) – Set Impact value (impact of failure) – Calculate Total and sort highest to lowest Software Peace of Mind in ActionSlide 18

19. Estimating the Work Effort Producing estimates using a defined process provides greater confidence in its potential accuracy – Is an accurate estimate an oxymoron? – Think about The Price Is Right Following a process for estimation allows us to review and identify information necessary to complete the project – it is NOT wasted time The problem is not in doing an estimate, but in the assumption that an estimate is automatically accurate and therefore a promise!! Software Peace of Mind in ActionSlide 19

20. Estimating Size Provides direct correlation to the number of tests necessary to validate the component Use existing documentation, conversations, and historical data to create estimate Estimate size based on amount of functionality in each component Review business rules Identify logical branches Consider inputs and outputs Software Peace of Mind in ActionSlide 20

21. Estimating Complexity The more complex an application, the longer it will take to analyze, create, and execute the tests. Some items to consider when determining complexity: Real-time vs. batch processing Number & types of interfaces/integrated systems User interface Data model OS / platform / technology Data quality Software Peace of Mind in ActionSlide 21

22. Estimating Reliability Determine reliability based on: Defect rates on previous, similar projects Responsiveness of developers in correcting defects in previous projects. Did corrections result in additional defects? Use historical analysis to evaluate the likelihood that the development process produces complete, stable, high-quality applications. Software Peace of Mind in ActionSlide 22

23. Estimation Example Step 1 Estimate size by counting the number of business rules, logical branches, inputs and outputs Software Peace of Mind in ActionSlide 23

24. Estimation Example Step 2 Estimate complexity & reliability for each component using a 5-point scale of 0.25, 0.5, 1.0, 1.5, and 2.0. Note that the scales for complexity and reliability are reversed, i.e. a 0.25 for complexity indicates very low complexity, but a 0.25 for reliability indicates a highly reliable component. Software Peace of Mind in ActionSlide 24

25. Estimation Example Step 3 Based on historical information or industry standards, determine: Average TC Analysis & Creation Time1.5 hours Average TC Execution Time1.0 hours Average Defect Rate35% Productive Hours/Day6.5 hours Software Peace of Mind in ActionSlide 25

26. Estimation Example Step 4 Calculate component estimates Calculated Factor (CF) = Size * Complexity * Reliability Analysis & Creation Time (ACT) = CF * Avg TC Analysis & Creation Time ÷ Productive Hrs/Day Execution Time (ET) = CF * Avg Time to Execute a TC ÷ Productive Hrs/Day Re-execution Time (RT) = ET * Avg Defect Rate

27. Test Schedule Develop a work schedule that assigns testing resources and timeframes to each test activity and identifies relationships/dependencies between them – Schedules help leads determine if the project is adequately staffed – Remember to match the experience of your resources to the level of work necessary, but look for opportunities to challenge them and cultivate their skills Software Peace of Mind in ActionSlide 27

28. Contingency Time Time considerations when developing the test schedule Holidays Special events Environment downtime Environment setup Environment/resource sharing Application response time Defect correction time Team location Software Peace of Mind in ActionSlide 28

29. Primary Test Activities There are 5 primary test activities – Test planning – Test analysis – Test creation – Test execution (includes defect tracking) – Test summary Each of these activities may be combined or duplicated for each type of test performed Each activity should have associated deliverables identified in the schedule Software Peace of Mind in ActionSlide 29

30. Sample Gantt Chart Software Peace of Mind in ActionSlide 30

31. Conclusion Careful planning lays the groundwork for a successful test effort by providing opportunities to: – Learn the scope of the project and the testing needs – Research and analyze, early in the project, the business needs and environment – Identify the relative size, complexity, and reliability of the application Software Peace of Mind in ActionSlide 31

32. Questions Software Peace of Mind in ActionSlide 32