Presentation is loading. Please wait.

Presentation is loading. Please wait.

Agile Test Paul Gerrard gerrardconsulting.com.

Similar presentations


Presentation on theme: "Agile Test Paul Gerrard gerrardconsulting.com."— Presentation transcript:

1 Agile Test Paul Gerrard gerrardconsulting.com

2 Overview What is Agile Test Strategy? Project Profiling (Test Strategy as) Agile Interventions Test Automation Whats Left? Summary Q&A Intelligent Definition and AssuranceSlide 2

3 What is Agile Test Strategy? Agile Strategy – an oxymoron?

4 Agile Test Strategy What do we mean by this? 1.AGILE Test Strategy – how to create a test strategy in an Agile way? 2.AGILE Test Strategy – a test strategy for an Agile project? Well look at how we created an Agile approach to strategy, but well spend more time on strategy for an Agile project. Intelligent Definition and AssuranceSlide 4

5 Google Agile Test Strategy There are plenty of recipes out there Most offer a selection of techniques but dont provide much guidance on how to blend them We need to know how to make choices, not just know what choices exist Strategy is a thought process, not a document – Although you might document the ideas for reuse, as a reminder or for the record. Intelligent Definition and AssuranceSlide 5

6 Agile governance Governance is the act of governing. It relates to decisions that define expectations, grant power, or verify performance Wikipedia Define expectations – DEFINITION of need Grant power – DELEGATION to a project team Verify performance – ASSURANCE of solution. Intelligent Definition and AssuranceSlide 6

7 Strategy helps you decide what to do A.The strategy presents some decisions that can be made ahead of time B.Defines the process or method or information that will allow decisions to be made (in project) C.Sets out the principles (or process) to follow for uncertain situations or unplanned events In a structured/waterfall environment, most questions answered off-the-shelf – A-style, ready for anything In an Agile environment – might have some ready-made policies but we manage scope and adapt (mostly C?) Intelligent Definition and AssuranceSlide 7

8 Contexts of Test Strategy Test Strategy Test Strategy Risks Goals Constraints Human resource Environment Timescales Process (lack of?) Contract Culture Opportunities User involvement Automation De- Duplication Early Testing Skills Communication Axioms Artefacts

9 Traditional v Agile test strategy Traditional – structured, goal/risk-driven – Identify stakeholders; what are their goals? – Product risk analysis – Allocate risks/goals to test stages – Formulate test stage definitions (entry/exit criteria, environments, tools etc. etc. Agile – interventionist, consensus-driven – Project profiling to set the testing theme – Identify testing interventions (perhaps better, contributions) in the Agile process – Test policy overlays the process; catches exceptions. Intelligent Definition and AssuranceSlide 9

10 Project Profiling Select a profile for your project first, then choose the aspects of test strategy that suite your project

11 Template-driven? Bah! So this is just a template copy and edit process? Wont you always end up with the same document? Profiling doesnt need to be prescriptive – No need to write a document if you dont need to – But if company policy or common sense dictates certain approaches, save yourself some time – Create a set of deeper, more detailed questions to be answered (Pocketbook) Profilers are really just checklists: heuristic guidelines designed to help you make choices and trade-offs. Intelligent Definition and AssuranceSlide 11

12 Cerise Orange Green Test Plan Items Product Risks Project Profiler Risk Profiler Project Plan Test Assurance Project Manager Business, Project Team and Boards Consultation Blue Unknowns Using a Project Profiler to Derive a Test Strategy and Project Plan (A government client example) The Project Profiler (with Test Assurance) helps Project Managers to: Select a project style that fits (Waterfall or Agile) Identify the product risks that need testing Identify test activities to include in project plans Carefully define the scope of the project Environment Story Guideline Tools Incident Mgt. Waterfall Test Strategy SCRUM/Agile Test Strategy Intelligent Definition and Assurance12

13 Project profiling process Task 1Have the Information you need to hand 2Project Profiler (section 3): Select the statements that best match your project context. The Blue column indicates that you need more information – consult your stakeholders, team or relevant Board(s). 3Generic Risk Profiler (section 4): Consider the generic project risks – which are significant for your project? Can testing help? 4Product Risk Profiler (Section 5): Consider the functional and non-functional risks that most concern your stakeholders – should they be tested? 5Actions and Test Activities (Section 6): Consider the actions that prompt your next steps and the test activities that should be incorporated into your project plan. 6Create your Test Strategy from the Test Strategy Framework Template 7Incorporate the activities from stage 5 and identified in 6 into your Project Plan. Intelligent Definition and Assurance13

14 Project Profiler (part of section 3) Project AspectCeriseOrangeGreenBlue Responsibility for Acceptance Users will take responsibility for UAT; they have UAT experience Users will be responsible for UAT but have no test experience Users will take part in UAT or witness tests at critical periods, and will review the outcome Users are unwilling/unable to take part in UAT; reluctant to make the acceptance decision or not known Requirements (Sources of Knowledge) New system replaces a well-understood existing system; users have clear vision of system goals and prefer to document their requirements up-front Users want to collaborate to jointly define requirements and meet them incrementally or iteratively Users put the onus of requirements elicitation on the project; requirements and the solution will evolve Inexperienced users who are unable or unwilling to collaborate with requirements gathering Requirements StabilityNew system is a functional replacement of an existing system or a well-defined process (requirements can be fixed early on) New system replaces an existing system with enhancements or an established (but not necessarily documented process) New system supports a new business need; business process exists but will change/evolve; users have experience of requirements New system supports a new business need; business process is not yet known; users have no experience or requirements Visibility, FormalityHigh visibility/risk to general public; formal progress reporting required at board level; fixed scope and deliverables; formal approvals and sign-offs High visibility/risk to business; formal progress reporting required; some defined deliverables, some deliverables will emerge/evolve; some approvals and sign-offs Relatively low business- risk; informal progress reporting is acceptable; partial solution may suffice, incremental/iterative delivery Potentially, high visibility, high risk project, uncertain impact on the business External DependenciesMore than one or new external suppliers responsible for development (and supplier testing) Single, known supplier responsible for development (and supplier testing) In-house development, no external dependencies Dependencies on external suppliers, their responsibilities or competence not yet known Etc. Intelligent Definition and Assurance14

15 Project types - examples CeriseStructured, waterfall style of project (and includes COTS projects) OrangeIterative/prototyping style of project using SCRUM in a formal way and having dedicated resources for the Business Analyst and Tester roles. GreenA project using SCRUM in a less formal way but not having dedicated resources for the Business Analyst and/or the Tester roles. BlueBlue column statements describe where there is insufficient information available to identify the style of project and the recommendation must be that some further investigation is required. Intelligent Definition and Assurance15

16 (Test Strategy as) Agile Interventions Im using Scrum/Sprint terminology, but you dont have to of course

17 Interventions (government client example) On the following slides, we highlight 8 interventions Some are test phases, but some arent No.ActivityWhen? 1Story ChallengeAs stories are added to the Product Backlog 2Story DefinitionAs stories are added to a Sprint Backlog These activities are repeated for each Sprint iteration 3Daily Stand-UpOnce per day during the Sprint 4Story RefinementOccurs throughout the Sprint as new information emerges 5Developer TestingOccurs throughout the Sprint as the developer codes the stories 6Integration (and incremental System) Testing During and at the end of each sprint, including the final sprint 7System TestingAt the end of each sprint, including the final sprint 8User Acceptance Testing At the end of each sprint, including the final sprint 9Non-functional Testing and Pre- Production Testing Expected to take place on an as-needs basis. Intelligent Definition and AssuranceSlide 17

18 Integration into Existing Code base Automated testing New Code 8. User Test 7. System Test Sprint 1 Developed Stories Sprint 3Sprint 2 Sprint Backlog 1.Story Challenge Suggest what-ifs to challenge new stories and define story headlines Increasing Scope of Sys. Test and UAT Increasing Scope of Integration, System and Users Testing 2. Story Definition Introduce scenarios to enhance the Acceptance Criteria Complete Tests after Final Sprint Project Level Test Activities (This diagram shows three sprints, but there could be more or fewer) 6. Integration Test

19 Integration into Existing Code base Automated testing New Code 8. User Test 7. System Test Sprint 1 Developed Stories Sprint 3Sprint 2 Sprint Backlog 1.Story Challenge Suggest what-ifs to challenge new stories and define story headlines Increasing Scope of Sys. Test and UAT Increasing Scope of Integration, System and Users Testing 2. Story Definition Introduce scenarios to enhance the Acceptance Criteria Complete Tests after Final Sprint Project Level Test Activities (This diagram shows three sprints, but there could be more or fewer) 6. Integration Test

20 Integration into Existing Code base Automated testing New Code 8. User Test 7. System Test Sprint 1 Developed Stories Sprint 3Sprint 2 Sprint Backlog 1.Story Challenge Suggest what-ifs to challenge new stories and define story headlines Increasing Scope of Sys. Test and UAT Increasing Scope of Integration, System and Users Testing 2. Story Definition Introduce scenarios to enhance the Acceptance Criteria Complete Tests after Final Sprint Project Level Test Activities (This diagram shows three sprints, but there could be more or fewer) 6. Integration Test

21 Integration into Existing Code base Automated testing New Code 8. User Test 7. System Test Sprint 1 Developed Stories Sprint 3Sprint 2 Sprint Backlog 1.Story Challenge Suggest what-ifs to challenge new stories and define story headlines Increasing Scope of Int. Sys. and UAT Increasing Scope of Integration, System and Users Testing 2. Story Definition Introduce scenarios to enhance the Acceptance Criteria Complete Tests after Final Sprint Project Level Test Activities (This diagram shows three sprints, but there could be more or fewer) 6. Integration Test

22 Integration into Existing Code base Automated testing New Code 8. User Test 7. System Test Sprint 1 Developed Stories Sprint 3Sprint 2 Sprint Backlog 1.Story Challenge Suggest what-ifs to challenge new stories and define story headlines Increasing Scope of Int. Sys. and UAT Increasing Scope of Integration, System and Users Testing 2. Story Definition Introduce scenarios to enhance the Acceptance Criteria Complete Tests after Final Sprint Project Level Test Activities (This diagram shows three sprints, but there could be more or fewer) 6. Integration Test

23 Daily Scrum Stand-Up Meeting 24 Hours 2-4 Weeks Backlog tasks expanded by team Potentially Shippable Product increment Product backlog As prioritised by Product Owner Sprint Backlog 4. Story Refinement Refine scenarios to enhance story definition, create system tests as stories, as required 6) Integration/System Testing Incorporate automated unit tests into the CI regime. On weekly basis and at end of Sprint, deploy to System test environment and tester runs system tests. 3. Daily Stand-Up Report anomalies found, stories tested, amended, created 5) Developer Testing Private ad-hoc tests and build/run automated unit tests Test Activities in the Sprint Intelligent Definition and Assurance23

24 Daily Scrum Stand-Up Meeting 24 Hours 2-4 Weeks Backlog tasks expanded by team Potentially Shippable Product increment Product backlog As prioritised by Product Owner Sprint Backlog 4. Story Refinement Refine scenarios to enhance story definition, create system tests as stories, as required 6) Integration/System Testing Incorporate automated unit tests into the CI regime. On weekly basis and at end of Sprint, deploy to System test environment and tester runs system tests. 3. Daily Stand-Up Report anomalies found, stories tested, amended, created 5) Developer Testing Private ad-hoc tests and build/run automated unit tests Test Activities in the Sprint Intelligent Definition and Assurance24

25 Daily Scrum Stand-Up Meeting 24 Hours 2-4 Weeks Backlog tasks expanded by team Potentially Shippable Product increment Product backlog As prioritised by Product Owner Sprint Backlog 4. Story Refinement Refine scenarios to enhance story definition, create system tests as stories, as required 6) Integration/System Testing Incorporate automated unit tests into the CI regime. On weekly basis and at end of Sprint, deploy to System test environment and tester runs system tests. 3. Daily Stand-Up Report anomalies found, stories tested, amended, created 5) Developer Testing Private ad-hoc tests and build/run automated unit tests Test Activities in the Sprint Intelligent Definition and Assurance25

26 Daily Scrum Stand-Up Meeting 24 Hours 2-4 Weeks Backlog tasks expanded by team Potentially Shippable Product increment Product backlog As prioritised by Product Owner Sprint Backlog 4. Story Refinement Refine scenarios to enhance story definition, create system tests as stories, as required 6) Integration/System Testing Incorporate automated unit tests into the CI regime. On weekly basis and at end of Sprint, deploy to System test environment and tester runs system tests. 3. Daily Stand-Up Report anomalies found, stories tested, amended, created 5) Developer Testing Private ad-hoc tests and build/run automated unit tests Test Activities in the Sprint Intelligent Definition and Assurance26

27 4. Story Refinement (example definition) Objectives To define acceptance criteria for all stories that are included in a Sprint as they are worked on by development To define scenarios that describe the tests and expected behaviours of the System Improve understanding of the requirement and communicate anomalies to developers To identify System Tests that exercise functionality of multiple stories that can be system tested in this sprint To assure the completeness for stories in the current Sprint Whats being tested? Stories to be included in the current Sprint Deliverables Refined story definitions with defined acceptance criteria and scenarios, where appropriate System tests Responsibilities (Orange) Tester – challenges stories by suggesting potential scenarios, new stories, story merges and splits; performs ad-hoc testing with/on behalf of developers; assures completeness of stories. Developers – considers stories, evaluates impact on development Product Owner or Analyst – collates feedback and decisions on stories Product Owner – approves changes to stories, accepts completeness of stories Scrum Master – monitors progress; evaluates impact on resources and schedules Responsibilities (Green) Not performed in Green projects Baseline Story Guideline (reference 3) Entry Criteria On commencement of the Sprint Exit Criteria When all stories within a Sprint are considered complete Queries, anomalies, discrepancies and inconsistencies have been eliminated or explained System Tests appropriate to the Sprint have been defined Definition of acceptance is agreed with Product Owner Intelligent Definition and Assurance27

28 Test Automation Could you create an Agile Test Strategy without automation?

29 Brian Maricks Testing quadrants Intelligent Definition and Assurance29

30 Test Automation Pyramid – Lisa Crispins version (Google others) Pyramid reflects the relative numbers of tests Focus on unit/component – Acceptance of Services GUI are end-to-end Manual checking the exception? Intelligent Definition and Assurance30

31 GUI Test Framework GUI Test Tool Browser Inter/Intranet HTTP/S Web Server App. Server DB Server HTTP/S HTTP Driver Stories/ Scenarios Unit Test Framework Test Code HTTP/S Unit Test Framework Test Code API 4. Programmers write low level HTTP GET/POST calls through a driver that simulates a browser 3. Non-Technical testers write scripts Tools Experts write interface 2. Technical Testers code scripts directly 1. Programmers write unit tests or execute embedded unit tests using a unit test framework to test components Where do you automate?

32 Distributed testing Use business stories and scenarios/acceptance criteria to validate requirements Reuse those stories to feed Acceptance- Driven Development BDD/TDD process Automated tests are an Anti-Regression tactic Automated tests dont replicate manual tests; think of them as a change trip-wire that triggers an alarm, if tripped. Intelligent Definition and Assurance32

33 Deriving scenarios to test To understand feature scope? To get stakeholder to accept? To validate the requirement? To estimate the work to build this feature? To system test this feature? To unit test this feature? Scenarios are created to meet several goals Story Challenge Story Refinement Story Definition Iteration Planning System Testing Developer Testing Intelligent Definition and Assurance33

34 Whats Left?

35 Other aspects of test policy Definitions (of done etc.) Incident management Test automation Story format e.g. Gherkin Environment request and management Regression testing (at what levels) Test deliverables and documentation. Intelligent Definition and AssuranceSlide 35

36 The Three Amigos Business Analyst – Liaises and manages stakeholders and their needs – Transforms business requirements into specification (at multiple levels) Developer – Scopes, designs, builds, tests and delivers features Tester – Challenges the thinking on the project – Performs Assurance in the small. Intelligent Definition and AssuranceSlide 36

37 The testers contribution to testing _________ feature/story acceptance criteria _________ the developers to unit test (auto) _________ feature/story acceptance _________ acceptance test _________ service/UI level automation Scope: from low-level detail to system integration Liaison with integration testers and feedback Fill in the blanks yourself; negotiate with your team. Intelligent Definition and AssuranceSlide 37

38 Summary

39 Close Agile test strategy has its place and many aspects of test can be pre-defined Importantly, we use a principled approach to deal with the unexpected Project profiling can help Testing as interventions, rather than test phases The testing role is split at least three ways – the tester doesnt own testing – think TESTMASTER Test automation in the context of Specification by Example, requirements validation, BDD, TDD. Intelligent Definition and AssuranceSlide 39

40 Agile Test Paul Gerrard gerrardconsulting.com


Download ppt "Agile Test Paul Gerrard gerrardconsulting.com."

Similar presentations


Ads by Google