1. Assurance: the Evolution of Test Management
Assurance, Re-Training, Mentoring
11-Nov-18
Assurance: the Evolution of Test Management
Paul Gerrard
Web: gerrardconsulting.com
Intelligent Definition and Assurance

2. Intelligent Definition and Assurance
Agenda
Defining Assurance
Scope and Role of Assurance
Is Assurance for Large Projects Only?
From Test Management to Assurance
Will You get into Assurance?
Intelligent Definition and Assurance

3. Intelligent Definition and Assurance
Intro
Agile has no clear role for test managers
Some test managers reverted to testers
Some specialise (test automation, performance or security)
Some moved upstream to be business analysts or product owners
Some left the business completely
Test Manager -> Virtual Test Management
Although Test Managers are under threat
Assurance is emerging and is a natural progression for delivery-focused professionals.
Intelligent Definition and Assurance

4. Assurance (aka test-, business- or delivery-assurance)
A way to improve the relationship with software suppliers, provide support and police them
Reports to the board and speaks truth to power
Requires coaching and persuasion skills and a deep knowledge of software and organisational dynamics
Digital is taking over everything – now is the time for Digital Assurance.
Intelligent Definition and Assurance

5. We need an Assurance vision
For twenty years or so, I’ve been involved in Assurance projects
Perhaps one meaty project each year
Mike Jarred and I are convinced Assurance is an important, emerging discipline
The TMF -> ALF rebrand is a natural evolution
Assurance is a discipline, a role in business
Today, I want to explore and discuss:
A definition of Assurance
What role it plays in business and software projects
How it presents an attractive career progression.
Intelligent Definition and Assurance

6. “Assuring Software Delivery is Successful”
Assurance Defined
“Assuring Software Delivery is Successful”

7. Assurance (from dictionary.com)

8. Missing from those definitions
We assure delivery
Not about process and box-ticking, it’s about improving (the chances of) successful delivery
Independence
Requires independence – no responsibility for delivery
Advisory
Advising projects on approaches, methods, plans, tools, techniques
Critical
Not here to make friends, we critically evaluate the performance of all activities and participants.
Intelligent Definition and Assurance

9. Missing from those definitions 2
Uber alles
Anything that has a bearing on successful delivery is potentially in scope
Stakeholder focused
We interpret the status and performance of projects and suppliers to inform stakeholders; we pass on stakeholder concerns to projects
People
Are the right people in place? Appropriate/adequate capabilities? Motivation, incentives, support available?
Governance
Are the right people involved? Are appropriate checks and balances in place?
Tech
Requirements, development, test, implementation, cutover approaches.
Intelligent Definition and Assurance

10. Scope, Role and Other Aspects

11. Is Assurance for Large Projects Only?No

12. Intelligent Definition and Assurance
Project scale
$200m SAP migration, £150m government infrastructure
Flash: Possible involvement in a £1.2bn project next week
But Assurance is scaleable to smaller and Agile projects
Consider the tester’s role in an Agile Scrum project
Testers might not do much testing; rather they act on intervention points.
Intelligent Definition and Assurance

13. (Test Strategy as) Agile Interventions
I’m using Scrum/Sprint terminology, but you don’t have to of course

14. These activities are repeated for each Sprint iteration
Interventions
No.
Activity
When? 1
Story Challenge
As stories are added to the Product Backlog 2
Story Definition
As stories are added to a Sprint Backlog
These activities are repeated for each Sprint iteration 3
Daily Stand-Up
Once per day during the Sprint 4
Story Refinement
Occurs throughout the Sprint as new information emerges 5
Developer Testing
Occurs throughout the Sprint as the developer codes the stories 6
Integration (and incremental System) Testing
During and at the end of each sprint, including the final sprint 7
System Testing
At the end of each sprint, including the final sprint 8
User Acceptance Testing 9
Non-functional Testing and Pre- Production Testing
Expected to take place on an as-needs basis.
On the following slides, we highlight 8 interventions
Some are test phases, but some aren’t
Intelligent Definition and Assurance

15. Project Level Test Activities
(This diagram shows three sprints, but there could be more or fewer)
Story Challenge
Suggest ‘what-ifs’ to challenge new stories and define story headlines
Sprint Backlog
Sprint Backlog
Sprint Backlog
2. Story Definition
Introduce scenarios to enhance the Acceptance Criteria
Sprint 1
Sprint 2
Sprint 3
Developed Stories
Developed Stories
Developed Stories
New Code
Integration into
Existing Code base
Automated testing
6. Integration Test
6. Integration Test
6. Integration Test
Increasing Scope of Sys. Test and UAT
7. System Test
8. User Test
Increasing Scope of Integration, System and Users Testing
Complete Tests after Final Sprint
Intelligent Definition and Assurance

16. Project Level Test Activities
(This diagram shows three sprints, but there could be more or fewer)
Story Challenge
Suggest ‘what-ifs’ to challenge new stories and define story headlines
Sprint Backlog
Sprint Backlog
Sprint Backlog
2. Story Definition
Introduce scenarios to enhance the Acceptance Criteria
Sprint 1
Sprint 2
Sprint 3
Developed Stories
Developed Stories
Developed Stories
New Code
Integration into
Existing Code base
Automated testing
6. Integration Test
6. Integration Test
6. Integration Test
Increasing Scope of Sys. Test and UAT
7. System Test
8. User Test
Increasing Scope of Integration, System and Users Testing
Complete Tests after Final Sprint
Intelligent Definition and Assurance

17. Project Level Test Activities
(This diagram shows three sprints, but there could be more or fewer)
Story Challenge
Suggest ‘what-ifs’ to challenge new stories and define story headlines
Sprint Backlog
Sprint Backlog
Sprint Backlog
2. Story Definition
Introduce scenarios to enhance the Acceptance Criteria
Sprint 1
Sprint 2
Sprint 3
Developed Stories
Developed Stories
Developed Stories
New Code
Integration into
Existing Code base
Automated testing
6. Integration Test
6. Integration Test
6. Integration Test
Increasing Scope of Sys. Test and UAT
7. System Test
8. User Test
Increasing Scope of Integration, System and Users Testing
Complete Tests after Final Sprint
Intelligent Definition and Assurance

18. Project Level Test Activities
(This diagram shows three sprints, but there could be more or fewer)
Story Challenge
Suggest ‘what-ifs’ to challenge new stories and define story headlines
Sprint Backlog
Sprint Backlog
Sprint Backlog
2. Story Definition
Introduce scenarios to enhance the Acceptance Criteria
Sprint 1
Sprint 2
Sprint 3
Developed Stories
Developed Stories
Developed Stories
New Code
Integration into
Existing Code base
Automated testing
6. Integration Test
6. Integration Test
6. Integration Test
Increasing Scope of Int. Sys. and UAT
7. System Test
8. User Test
Increasing Scope of Integration, System and Users Testing
Complete Tests after Final Sprint
Intelligent Definition and Assurance

19. Project Level Test Activities
(This diagram shows three sprints, but there could be more or fewer)
Story Challenge
Suggest ‘what-ifs’ to challenge new stories and define story headlines
Sprint Backlog
Sprint Backlog
Sprint Backlog
2. Story Definition
Introduce scenarios to enhance the Acceptance Criteria
Sprint 1
Sprint 2
Sprint 3
Developed Stories
Developed Stories
Developed Stories
New Code
Integration into
Existing Code base
Automated testing
6. Integration Test
6. Integration Test
6. Integration Test
Increasing Scope of Int. Sys. and UAT
7. System Test
8. User Test
Increasing Scope of Integration, System and Users Testing
Complete Tests after Final Sprint
Intelligent Definition and Assurance

20. Test Activities in the Sprint
3. Daily Stand-Up
Report anomalies found, stories tested, amended, created
5) Developer Testing
Private ad-hoc tests and build/run automated unit tests
Daily Scrum
Stand-Up
Meeting
24 Hours
4. Story Refinement
Refine scenarios to enhance story definition, create system tests as stories, as required
6) Integration/System Testing
Incorporate automated unit tests into the CI regime.
On weekly basis and at end of Sprint, deploy to System test environment and tester runs system tests.
2-4 Weeks
Backlog tasks
expanded by team
Sprint Backlog
Potentially Shippable Product increment
Product backlog
As prioritised by Product Owner
Intelligent Definition and Assurance

21. Test Activities in the Sprint
3. Daily Stand-Up
Report anomalies found, stories tested, amended, created
5) Developer Testing
Private ad-hoc tests and build/run automated unit tests
Daily Scrum
Stand-Up
Meeting
24 Hours
4. Story Refinement
Refine scenarios to enhance story definition, create system tests as stories, as required
6) Integration/System Testing
Incorporate automated unit tests into the CI regime.
On weekly basis and at end of Sprint, deploy to System test environment and tester runs system tests.
2-4 Weeks
Backlog tasks
expanded by team
Sprint Backlog
Potentially Shippable Product increment
Product backlog
As prioritised by Product Owner
Intelligent Definition and Assurance

22. Test Activities in the Sprint
3. Daily Stand-Up
Report anomalies found, stories tested, amended, created
5) Developer Testing
Private ad-hoc tests and build/run automated unit tests
Daily Scrum
Stand-Up
Meeting
24 Hours
4. Story Refinement
Refine scenarios to enhance story definition, create system tests as stories, as required
6) Integration/System Testing
Incorporate automated unit tests into the CI regime.
On weekly basis and at end of Sprint, deploy to System test environment and tester runs system tests.
2-4 Weeks
Backlog tasks
expanded by team
Sprint Backlog
Potentially Shippable Product increment
Product backlog
As prioritised by Product Owner
Intelligent Definition and Assurance

23. Test Activities in the Sprint
3. Daily Stand-Up
Report anomalies found, stories tested, amended, created
5) Developer Testing
Private ad-hoc tests and build/run automated unit tests
Daily Scrum
Stand-Up
Meeting
24 Hours
4. Story Refinement
Refine scenarios to enhance story definition, create system tests as stories, as required
6) Integration/System Testing
Incorporate automated unit tests into the CI regime.
On weekly basis and at end of Sprint, deploy to System test environment and tester runs system tests.
2-4 Weeks
Backlog tasks
expanded by team
Sprint Backlog
Potentially Shippable Product increment
Product backlog
As prioritised by Product Owner
Intelligent Definition and Assurance

24. The tester’s Assurance contribution
Think of testing as interventions, not stages
The testing role is redistributed and split
The tester doesn’t own testing – think TESTMASTER – ASSURANCE in the small
Identify the intervention points and negotiate with your team.
Intelligent Definition and Assurance

25. From Test Management to Assurance

26. Intelligent Definition and Assurance
Test Managers
You might have been a tester, then a test lead
As a lead you become a coach, mentor to other testers, but still spend time testing
As a Test Manager, things change
You are accountable for the performance of the team
You represent the team to Project and Stakeholder management
You have to plan, organise, manage, control
Justify, defend testing, timescales, scope, coverage…
Intelligent Definition and Assurance

27. Testing Measures Achievement
Although you don’t test, you have to coach, cajole, command the team to test
Project managers can push people around and provide graphs of money and time spent
Inputs give a rear-view mirror perspective
Testing reveals the truth:
Achievement: what works and what does not
Risk: what is the view of the risk of release.
Intelligent Definition and Assurance

28. Testing is at the heart of Assurance
Information on achievement and risks is pretty much what stakeholders need
You are responsible, accountable for providing that information
Assurance, being independent, depends on the outcomes of testing to a large degree
(But it’s not everything of course)
Test Managers are well placed to take the role.
Intelligent Definition and Assurance

29. My Challenge to You

30. Will you get into assurance?
In your current project(s), would an assurance role help?
There is no test manager, but test management happens
Who keeps an eye on the virtual test manager?
You might not have all the skills required, but how will you acquire them?
First opportunity is likely to be a technical one.
Intelligent Definition and Assurance

31. Assurance: the Evolution of Test Management
Assurance, Re-Training, Mentoring
11-Nov-18
Assurance: the Evolution of Test Management
Paul Gerrard
Web: gerrardconsulting.com
Intelligent Definition and Assurance