Presentation is loading. Please wait.

Presentation is loading. Please wait.

SoftTest IrelandCopyright 2008 ElectroMind Ltd1 People Challenges for Test Teams Stephen K. Allott.

Similar presentations


Presentation on theme: "SoftTest IrelandCopyright 2008 ElectroMind Ltd1 People Challenges for Test Teams Stephen K. Allott."— Presentation transcript:

1 SoftTest IrelandCopyright 2008 ElectroMind Ltd1 People Challenges for Test Teams Stephen K. Allott

2 SoftTest Ireland Copyright 2008 ElectroMind Ltd 2

3 SoftTest Ireland Copyright 2008 ElectroMind Ltd 3

4 SoftTest Ireland Copyright 2008 ElectroMind Ltd 4

5 SoftTest Ireland Copyright 2008 ElectroMind Ltd 5 Introductions My background Aims of this workshop Explain some common testing challenges based on recent experience Ask for your experiences Agree a list of challenges we all face as testers Try to come up with some quick wins & longer term approaches to solutions

6 SoftTest Ireland Copyright 2008 ElectroMind Ltd 6 My recent clients Investment bank Airline web site Travel company web site Mobile phone operator Retail web site Travel company back office Recruitment web site Includes assignments done in association with TestHouse (London) and Neueda Technologies (Antrim & Dublin)

7 SoftTest Ireland Copyright 2008 ElectroMind Ltd 7 Their technologies Microsoft Dot (.) NET 1/2/3 Microsoft VSTS / TFS Oracle MS SQL Server Windows XP Citrix Metaframe Quality Centre / QTP

8 SoftTest Ireland Copyright 2008 ElectroMind Ltd 8 Number of testers Typically 12 to 14 in the test team A couple of clients had only 1, some have none!

9 SoftTest Ireland Copyright 2008 ElectroMind Ltd 9 Scale Key Area Controlled EfficientOptimising Test strategy A B C D Life-cycle model A B Moment of involvement A B C D Estimating and planning A B Test specification techniques A B Static test techniques A B Metrics A B C D Test automation A B C Test environment A B C Office environment A Commitment and motivation A B C Test functions and training A B C Scope of methodology A B C Communication A B C Reporting A B C D Defect management A B C Testware management A B C D Test process management A B C Evaluation A B Low-level testing A B C Composite TPI Test Maturity Matrix from several clients

10 SoftTest Ireland Copyright 2008 ElectroMind Ltd 10 Summary of TPI results No documented test process No formal test policy, strategy or plan Many tests, no formal design No metrics No proper test environment Some tested in live No documented test basis Not involved early enough

11 SoftTest Ireland Copyright 2008 ElectroMind Ltd 11 Some common observations Constantly changing business priorities Increasing (frantic) demand for new functionality Weekly, sometimes daily release cycles Attempt to use automation to solve all the problems No numbers to explain testings added value Small important teams yet with little influence

12 SoftTest Ireland Copyright 2008 ElectroMind Ltd 12 People challenges – behaviour Motivation A few are resigned to how it is, no hope? Some expect spoon feeding Some adopt a negative attitude Many are willing to make improvements, if there was time Turnover – very high offshore Culture Some managers get too involved in detail Co-dependency – testers must push back It hasnt broken yet so...

13 SoftTest Ireland Copyright 2008 ElectroMind Ltd 13 People challenges – skills Testing Skills Processes Standards Techniques Tools IT Skills Programming Databases Infrastructure Web Domain Knowledge Industry specific Application knowledge E-Business Soft Skills Communication Negotiation Managing technical teams

14 SoftTest Ireland Copyright 2008 ElectroMind Ltd 14 People challenges – perception In general testers are not valued Testers are often blamed Project delays Poor quality Managing expectations with RBT Lack of awareness of what the test team does

15 SoftTest Ireland Copyright 2008 ElectroMind Ltd 15 People challenges – managers Demand for certified staff Demand for skills & knowledge Pressure to cut costs and use offshore resources Developing good test leads Role of testers in an agile project Selling the value of testing to the organisation

16 SoftTest Ireland Copyright 2008 ElectroMind Ltd 16 Themes Facilitated discussion bearing in mind the following themes: 1. Relentless business pressure for new functionality 2. Constant and last minute changes to requirements 3. Introduction of new development / testing methodologies e.g. agile 4. Very frequent releases, often weekly, sometimes daily! 5. Increasing application complexity over time 6. Knowledge gap (e.g. what lies beyond the browser?) 7. Steep learning curve (business as well as technical) 8. Use of contract and offshore resources (loss of in-house expertise) 9. Little or no documentation against which to test 10. Lack of attention to all testing requirements (e.g. middleware, back end database, non-functional quality characteristics)

17 SoftTest Ireland Copyright 2008 ElectroMind Ltd 17 Discussion – how do we respond? Business Pressures New functionality Frequent change Maturity Technology Complex Rapid change Interfaces Offshore Cheaper Plentiful Expertise Behaviour Motivation Attitude Culture Skills Analysis IT / Technical Business Testing Soft skills Managers Costs Time to market Agile

18 SoftTest Ireland Copyright 2008 ElectroMind Ltd 18 Conclusions In my opinion testers of the future must... Measure / appreciate application complexity Be able to accept and manage last minute changes Be flexible but push back with estimates Increase their technical knowledge Learn more about testing, especially techniques Apply the tried and tested techniques Capture metrics to help prove the business case Get involved in the projects much earlier Recruit / train / certify / develop / coach = Retain Share your views with the group

19 SoftTest Ireland Copyright 2008 ElectroMind Ltd 19 Final Thoughts / Questions?

20 SoftTest IrelandCopyright 2008 ElectroMind Ltd20 People Challenges for Test Teams +44(0)

21 SoftTest IrelandCopyright 2008 ElectroMind Ltd21 People Challenges for Test Teams Appendix – Bonus Slides

22 SoftTest Ireland Copyright 2008 ElectroMind Ltd 22 A typical project in trouble

23 SoftTest Ireland Copyright 2008 ElectroMind Ltd 23 Bug snapshot by priority Sev 1Sev 2Sev 3Sev 4Total Pri Pri Pri Pri 4213 None Total

24 SoftTest Ireland Copyright 2008 ElectroMind Ltd 24 Bug snapshot by resolution Sev 1Sev 2Sev 3Sev 4Total By design Duplicate External Fixed Not repro Postponed Will not fix Total

25 SoftTest Ireland Copyright 2008 ElectroMind Ltd 25 Impact of the improvements

26 Events Systems 1.Vendor 2.User Reg. 3.User Shop Sales and Marketing Systems 1.Consulting Practice Mgt. 2.Vendor Account Mgt. 3.Event Mgt. 4.Leads and Referral Databases 1.Profile 2.Common 3.Event OLTP 4.Transaction OLTP 5.Contact History OLTP 6.Domain Reference World-wide Marketing DB 1.Profile OLAP 2.Event OLAP 3.Analytical Reporting 4.Contract History OLAP 5.Domain Corporate Sales and Marketing Systems System Interfaces 1.Event Orders 2.Billing & Financials 3.Credit Card (real-time) 4.Event Communication / FAX 5.Direct service 6.Direct mail Postal interface 7.Leads & Referrals 8.OLTP Staging Interface 9.OLAP Staging Interface 10.SAP 11.Licensing 12.Call Center Source: Reality Test Inc.

27 SoftTest Ireland Copyright 2008 ElectroMind Ltd 27 Basic complexity calculation Factors are weighted from a testing perspective

28 SoftTest Ireland Copyright 2008 ElectroMind Ltd 28 Comparison across applications

29 SoftTest Ireland Copyright 2008 ElectroMind Ltd 29 Complexity comparison

30 SoftTest Ireland Copyright 2008 ElectroMind Ltd 30 Actual staffing comparison

31 SoftTest Ireland Copyright 2008 ElectroMind Ltd 31 Organisation evaluation Organisation within the company Process definition and execution Teamwork Ability to do the job

32 SoftTest Ireland Copyright 2008 ElectroMind Ltd 32 Organisation within the company SectionItem being evaluatedMaxScore 1.1Reporting structure50 1.2Professionalism Organisational churn aStaffing – permanence30 1.4bStaffing – stability35 1.4cStaffing – morale50 1.4dStaffing – number of testers40 1.4eStaffing – organisation type40 1.5Budget control30

33 SoftTest Ireland Copyright 2008 ElectroMind Ltd 33 Process definition and execution SectionItem being evaluatedMaxScore 2.1Software development life-cycle60 2.2Design reviews and documentation70 2.3Early participation in development50 2.4Ability to influence schedule deadlines60 2.5Development standards60 2.6Defined testing approach & procedures Metrics reporting60 2.8Churn90 2.9Patch releases Post-project reviews60

34 SoftTest Ireland Copyright 2008 ElectroMind Ltd 34 Teamwork SectionItem being evaluatedMaxScore 3.1Support from production support60 3.2Support from management60 3.3Correct level of quality90 3.4Resource allocation80

35 SoftTest Ireland Copyright 2008 ElectroMind Ltd 35 Ability to do the job SectionItem being evaluatedMaxScore 4.1Adequate equipment / lab Automated testing70 4.3Programming ability70 4.4Non-testing activities50

36 SoftTest Ireland Copyright 2008 ElectroMind Ltd 36 Total evaluation score

37 SoftTest Ireland Copyright 2008 ElectroMind Ltd 37 Organisational Evaluation

38 SoftTest Ireland Copyright 2008 ElectroMind Ltd 38 Components of a centre of excellence Resource pool Test lab Test Managers SIG Consultants Knowledge Base R&D Centre Tool Experts Project 1 Project 2 Project 3 Project 4

39 SoftTest Ireland Copyright 2008 ElectroMind Ltd 39 Benefits of a centre of excellence Independent testing Breadth & depth Fewer product recalls Fewer production failures Central reference point Knowledgeable staff Expertise across org. Structured career path Variety of work Identity, pride Morale, professionalism Extra revenue stream Consistent approach Multi-methodologies Earlier involvement


Download ppt "SoftTest IrelandCopyright 2008 ElectroMind Ltd1 People Challenges for Test Teams Stephen K. Allott."

Similar presentations


Ads by Google