SoftTest Ireland Copyright 2008 ElectroMind Ltd 5 Introductions My background Aims of this workshop Explain some common testing challenges based on recent experience Ask for your experiences Agree a list of challenges we all face as testers Try to come up with some quick wins & longer term approaches to solutions
SoftTest Ireland Copyright 2008 ElectroMind Ltd 6 My recent clients Investment bank Airline web site Travel company web site Mobile phone operator Retail web site Travel company back office Recruitment web site Includes assignments done in association with TestHouse (London) and Neueda Technologies (Antrim & Dublin)
SoftTest Ireland Copyright 2008 ElectroMind Ltd 7 Their technologies Microsoft Dot (.) NET 1/2/3 Microsoft VSTS / TFS Oracle MS SQL Server Windows XP Citrix Metaframe Quality Centre / QTP
SoftTest Ireland Copyright 2008 ElectroMind Ltd 8 Number of testers Typically 12 to 14 in the test team A couple of clients had only 1, some have none!
SoftTest Ireland Copyright 2008 ElectroMind Ltd 9 Scale012345678910111213 Key Area Controlled EfficientOptimising Test strategy A B C D Life-cycle model A B Moment of involvement A B C D Estimating and planning A B Test specification techniques A B Static test techniques A B Metrics A B C D Test automation A B C Test environment A B C Office environment A Commitment and motivation A B C Test functions and training A B C Scope of methodology A B C Communication A B C Reporting A B C D Defect management A B C Testware management A B C D Test process management A B C Evaluation A B Low-level testing A B C Composite TPI Test Maturity Matrix from several clients
SoftTest Ireland Copyright 2008 ElectroMind Ltd 10 Summary of TPI results No documented test process No formal test policy, strategy or plan Many tests, no formal design No metrics No proper test environment Some tested in live No documented test basis Not involved early enough
SoftTest Ireland Copyright 2008 ElectroMind Ltd 11 Some common observations Constantly changing business priorities Increasing (frantic) demand for new functionality Weekly, sometimes daily release cycles Attempt to use automation to solve all the problems No numbers to explain testings added value Small important teams yet with little influence
SoftTest Ireland Copyright 2008 ElectroMind Ltd 12 People challenges – behaviour Motivation A few are resigned to how it is, no hope? Some expect spoon feeding Some adopt a negative attitude Many are willing to make improvements, if there was time Turnover – very high offshore Culture Some managers get too involved in detail Co-dependency – testers must push back It hasnt broken yet so...
SoftTest Ireland Copyright 2008 ElectroMind Ltd 13 People challenges – skills Testing Skills Processes Standards Techniques Tools IT Skills Programming Databases Infrastructure Web Domain Knowledge Industry specific Application knowledge E-Business Soft Skills Communication Negotiation Managing technical teams
SoftTest Ireland Copyright 2008 ElectroMind Ltd 14 People challenges – perception In general testers are not valued Testers are often blamed Project delays Poor quality Managing expectations with RBT Lack of awareness of what the test team does
SoftTest Ireland Copyright 2008 ElectroMind Ltd 15 People challenges – managers Demand for certified staff Demand for skills & knowledge Pressure to cut costs and use offshore resources Developing good test leads Role of testers in an agile project Selling the value of testing to the organisation
SoftTest Ireland Copyright 2008 ElectroMind Ltd 16 Themes Facilitated discussion bearing in mind the following themes: 1. Relentless business pressure for new functionality 2. Constant and last minute changes to requirements 3. Introduction of new development / testing methodologies e.g. agile 4. Very frequent releases, often weekly, sometimes daily! 5. Increasing application complexity over time 6. Knowledge gap (e.g. what lies beyond the browser?) 7. Steep learning curve (business as well as technical) 8. Use of contract and offshore resources (loss of in-house expertise) 9. Little or no documentation against which to test 10. Lack of attention to all testing requirements (e.g. middleware, back end database, non-functional quality characteristics)
SoftTest Ireland Copyright 2008 ElectroMind Ltd 17 Discussion – how do we respond? Business Pressures New functionality Frequent change Maturity Technology Complex Rapid change Interfaces Offshore Cheaper Plentiful Expertise Behaviour Motivation Attitude Culture Skills Analysis IT / Technical Business Testing Soft skills Managers Costs Time to market Agile
SoftTest Ireland Copyright 2008 ElectroMind Ltd 18 Conclusions In my opinion testers of the future must... Measure / appreciate application complexity Be able to accept and manage last minute changes Be flexible but push back with estimates Increase their technical knowledge Learn more about testing, especially techniques Apply the tried and tested techniques Capture metrics to help prove the business case Get involved in the projects much earlier Recruit / train / certify / develop / coach = Retain Share your views with the group
SoftTest IrelandCopyright 2008 ElectroMind Ltd20 People Challenges for Test Teams firstname.lastname@example.org +44(0) 773 476 1363 www.electromind.com
SoftTest IrelandCopyright 2008 ElectroMind Ltd21 People Challenges for Test Teams Appendix – Bonus Slides
SoftTest Ireland Copyright 2008 ElectroMind Ltd 22 A typical project in trouble
SoftTest Ireland Copyright 2008 ElectroMind Ltd 23 Bug snapshot by priority Sev 1Sev 2Sev 3Sev 4Total Pri 1578114152 Pri 2195212184 Pri 36141030 Pri 4213 None6828811410480 Total14642815421749
SoftTest Ireland Copyright 2008 ElectroMind Ltd 24 Bug snapshot by resolution Sev 1Sev 2Sev 3Sev 4Total By design103729480 Duplicate6337147 External2164022 Fixed862549512447 Not repro29564089 Postponed12176439 Will not fix1159025 Total14642815421749
SoftTest Ireland Copyright 2008 ElectroMind Ltd 25 Impact of the improvements
Events Systems 1.Vendor 2.User Reg. 3.User Shop Sales and Marketing Systems 1.Consulting Practice Mgt. 2.Vendor Account Mgt. 3.Event Mgt. 4.Leads and Referral Databases 1.Profile 2.Common 3.Event OLTP 4.Transaction OLTP 5.Contact History OLTP 6.Domain Reference World-wide Marketing DB 1.Profile OLAP 2.Event OLAP 3.Analytical Reporting 4.Contract History OLAP 5.Domain Corporate Sales and Marketing Systems System Interfaces 1.Event Orders 2.Billing & Financials 3.Credit Card (real-time) 4.Event Communication / FAX 5.Direct email service 6.Direct mail Postal interface 7.Leads & Referrals 8.OLTP Staging Interface 9.OLAP Staging Interface 10.SAP 11.Licensing 12.Call Center Source: Reality Test Inc.
SoftTest Ireland Copyright 2008 ElectroMind Ltd 27 Basic complexity calculation Factors are weighted from a testing perspective
SoftTest Ireland Copyright 2008 ElectroMind Ltd 31 Organisation evaluation Organisation within the company Process definition and execution Teamwork Ability to do the job
SoftTest Ireland Copyright 2008 ElectroMind Ltd 32 Organisation within the company SectionItem being evaluatedMaxScore 1.1Reporting structure50 1.2Professionalism170 1.3Organisational churn135 1.4aStaffing – permanence30 1.4bStaffing – stability35 1.4cStaffing – morale50 1.4dStaffing – number of testers40 1.4eStaffing – organisation type40 1.5Budget control30
SoftTest Ireland Copyright 2008 ElectroMind Ltd 33 Process definition and execution SectionItem being evaluatedMaxScore 2.1Software development life-cycle60 2.2Design reviews and documentation70 2.3Early participation in development50 2.4Ability to influence schedule deadlines60 2.5Development standards60 2.6Defined testing approach & procedures190 2.7Metrics reporting60 2.8Churn90 2.9Patch releases40 2.10Post-project reviews60
SoftTest Ireland Copyright 2008 ElectroMind Ltd 34 Teamwork SectionItem being evaluatedMaxScore 3.1Support from production support60 3.2Support from management60 3.3Correct level of quality90 3.4Resource allocation80
SoftTest Ireland Copyright 2008 ElectroMind Ltd 35 Ability to do the job SectionItem being evaluatedMaxScore 4.1Adequate equipment / lab200 4.2Automated testing70 4.3Programming ability70 4.4Non-testing activities50
SoftTest Ireland Copyright 2008 ElectroMind Ltd 38 Components of a centre of excellence Resource pool Test lab Test Managers SIG Consultants Knowledge Base R&D Centre Tool Experts Project 1 Project 2 Project 3 Project 4
SoftTest Ireland Copyright 2008 ElectroMind Ltd 39 Benefits of a centre of excellence Independent testing Breadth & depth Fewer product recalls Fewer production failures Central reference point Knowledgeable staff Expertise across org. Structured career path Variety of work Identity, pride Morale, professionalism Extra revenue stream Consistent approach Multi-methodologies Earlier involvement
Your consent to our cookies if you continue to use this website.