Test Management for Very Large Programs A Survival Kit aka Program Test Management - A Survival Kit Graham Thomas Independent Software Testing Consultant STARWEST, Anaheim, October 2008

A BSTRACT In large organizations with multiple, simultaneous, and related projects, how do you coordinate testing efforts for better utilization and higher quality? Some organizations have opened Program Test Management offices to oversee the multiple streams of testing projects and activities, each with its own test manager. Should the Program Test Manager be an über-manager in control of everything, or is this office more of an aggregation and reporting function? Graham Thomas examines the spectrum of possible duties and powers of this position. He also shares the critical factors for successful program test management, including oversight of the testing products and deliverables; matrix management of test managers; stakeholder, milestone, resource and dependency management; and the softer but vital skills of influence and negotiation with very senior managers. Relating experience gained on several large testing programs, Graham shares a practical model—covering the key test management areas of organization, people, process, tools, and metrics—that your organization can adapt for its needs. 2© Graham Thomas 2008

A GENDA  Introduction  Definitions  Role  Attributes & Skill Set  Survival Kit o Organisation o People o Process o Tools o Metrics  Critical Success Factors  Lessons Learnt  Summary © Graham Thomas 20083

I NTRODUCTION  Why Program Test Management? o Large Organisations o Big Programs of work o Many inter-related projects and programs  Why Program Test Managers? o To represent testing at the program level o To manage multiple streams of testing projects and activities o At a level which is outside the sphere of influence for individual projects or testing activities 4© Graham Thomas 2008

D EFINITION T EST M ANAGEMENT planning analysis design preparation execution validation reporting “The direct management of testing activities” 5© Graham Thomas 2008

D EFINITION P ROGRAM M ANAGEMENT  What is a Program? “A broad effort encompassing a number of projects and/or functional activities with a common purpose.” Association for Project Managers - Web site Definitions  What is Program Management? “The management of multiple streams of development activities (including testing), each with their own teams carrying out detailed planning, management and execution activities.” 6© Graham Thomas 2008

D EFINITION P ROGRAM T EST M ANAGEMENT “The management of multiple streams of testing, each with their own test management in place which is responsible for detailed planning, management and control activities.” 7© Graham Thomas 2008

PgTM R OLE C ONTINUUM  Über Test Manager o Micro-management o Disempowering o Decision making bottleneck o Stifle creativity o Perpetuates hero culture  Aggregating View o Detached and remote o Spy in the camp o Collation and aggregation o Viewed as overhead o No ‘test’ value add © Graham Thomas 20088

I DEAL PgTM R OLE R ESPONSIBILITIES  PgTMs are not responsible for managing the testing activities. That is what we have Test Managers for!  So, what are PgTMs responsible for if they are not managing the testing?  Responsibilities vary from program to program:  Testing oversight  Analogous to Program Management  Looking for gaps across the program  Getting the development groups to work together with the testers  Defining and managing the program testing approach 9© Graham Thomas 2008

A TTRIBUTES OF A GOOD PgTM  Honesty, Integrity, Experience, Credibility  Pro-active, (Dynamic - if you think rushing round helps without causing panic)  Politically aware and astute  Focussed on the program and organisational goals  Independence 10© Graham Thomas 2008

E SSENTIAL S KILL S ET  In addition to: o Good test management o Good project management skills  Stakeholder management o Influence & negotiation skills  Program Management o Milestone management o Resource & Dependency management o Delivery through others o Oversight and awareness  Motivate and enthuse 11© Graham Thomas 2008

S URVIVAL T OOLKIT  3 Good Things  Swiss Army Knife - That tool that no-one knows what it is for!  What to avoid doing, is as important to know, as what to do 12© Graham Thomas 2008

13 Video 1

T HREE G OOD T HINGS A few years ago I was invited to a corporate Clay Pigeon shooting day. I had never shot before, but it sounded like fun so I thought that I would give it a go. In the morning we had a practice at the six disciplines. I had never shot before so I was put into the absolute beginners group. I was hopeless. I couldn’t hit anything. The coach was very good. He had a look at what I was doing, and for each discipline, just told me 3 simple things, that I could relate to, that would help me. In the afternoon I won the individual competition – with a high score, and two 6’s. And that got me to thinking about 3 good things. If only we could do that for Software Testing. 14

O RGANISATION B EST P RACTISE  Define clear test organisation structure with matrixed relationships  Clear and agreed interface with stakeholders and sponsors  Maintain independent reporting for testing A NTI -P RACTISE  Testing only reports into delivery  Testing perceived as secondary activity 15© Graham Thomas 2008 CIO Bus.Dev.Test PO

16 Video 2

P EOPLE B EST P RACTISE  Ideally want people who want to be testers  Identify your best people and give them the right opportunities  Create a collaborative environment to share test knowledge and experience A NTI -P RACTISE  Unclear testing roles  Lack of training and investment in testers 17© Graham Thomas 2008

18 Video 3

P ROCESS B EST P RACTISE  Align development and testing activities  Consistent processes - standards, methods and approach  Fit for purpose, flexible and effective A NTI -P RACTISE  Testing in silos – UT, ST, SIT, UAT, OAT etc.  Quality police 19© Graham Thomas 2008

T OOLS B EST P RACTISE  You will get as big a benefit from common process  Match tools and processes e.g. Test Planning  Not just automation - but Config, Release & Change Mgmt A NTI -P RACTISE  Developers get tools, testers don’t  You have Quality Centre, what more can you want?  No Silver bullets 20© Graham Thomas 2008

M ETRICS B EST P RACTISE  Align your measures and metrics to your stakeholders and sponsors needs  Keep your measures and metrics consistent  Trend measures are very powerful A NTI -P RACTISE  If it moves measure it!  Inappropriate use of metrics can drive blame culture 21© Graham Thomas 2008

C RITICAL S UCCESS F ACTORS  To ensure that you put in place a testing organisation that meets the testing needs of the program  To ensure that you have an efficient and effective testing process that support and integrate with your development approach  To ensure that your stakeholders and sponsors clearly understand what testing is doing for them  To be able to articulate the program testing message 22© Graham Thomas 2008

S UMMARY & Q UESTIONS  Defined the Term Program Test Management  Examined the PTM role continuum  Looked at responsibilities, attributes and skills  Worked through a model for Program Test Management  Identified Critical Success Factors  Any Questions 23© Graham Thomas 2008

C ONTACT D ETAILS 24 Graham Thomas Independent Software Testing Consultant   © Graham Thomas 2008