Presentation is loading. Please wait.

Presentation is loading. Please wait.

Classic Testing Mistakes: Revisited Matthew Heusser Presented at the Indiana QA Conference Indianapolis, Oct 13 th, 2006 Contributing.

Similar presentations


Presentation on theme: "Classic Testing Mistakes: Revisited Matthew Heusser Presented at the Indiana QA Conference Indianapolis, Oct 13 th, 2006 Contributing."— Presentation transcript:

1 Classic Testing Mistakes: Revisited Matthew Heusser Presented at the Indiana QA Conference Indianapolis, Oct 13 th, 2006 Contributing peer reviewers: James Bach Paul Carvalho Michael Kelly Harry Robinson

2 Organization Classic Mistakes: A different approach The mistakes enumerated –Test Management Mistakes –Test Automation Mistakes –Development Mistakes –Test Strategy Mistakes Root Causes What to do tomorrow

3

4

5 Classic Mistake #1: De-humanize the test process Test Management Mistakes AKA Management by Spreadsheet, Management by , Management by MS Project …

6 Classic Mistake #2: Testers Responsible for Quality “It’s strange that QA let that bug slip through” Test Management Mistakes

7 Classic Mistake #3: IV&V Determines ship date Do they really? Test Management Mistakes

8 Classic Mistake #4: Task-based status reporting Examples: –Testing is “on schedule” –Testing “should be done by Tuesday” Consequences –Loss of credibility –Bad information for decision makers Test Management Mistakes

9 Classic Mistake #5: Evaluating testers by bugs found … and developers by number of bugs injected Consequences: –Friction –Focus on easy-to-find yet trivial bugs (usability) –Information hiding Test Management Mistakes

10 Classic Mistakes #6 Inappropriate Models for Test Improvement NO

11 Classic Mistake #7: Lack of test training for developers Testing is a skill. It won’t appear like magic. Development Testing Mistakes

12 Classic Mistake #8: Separate devs and testers To create friction, emphasize division Anything that increases the length of the feedback loop is bad. To improve get rid of waste and tighten the feedback loop. Development Testing Mistakes

13 Mistake #9: When late, add Test Automation Someone has to learn the tool Someone has to record the scripts Test Automation Mistakes

14 Mistake #10: Mine Field Test Automation Test Automation Mistakes

15 Mistake #11: Hiring for test tool skills Technology skills can be taught Talent can’t The “Hit the Ground Running” Argument Test Automation Mistakes

16 Classic Mistake #12: Insufficient diversity in test strategy Examples: –Only requirements based testing –Only coverage testing Consequence: –Missing entire classifications of defects Test Strategy Mistakes

17 Classic Mistake #13: Over-reliance on scripted testing All the testing we did, meticulously pulling down every menu and seeing if it worked right, didn't uncover the showstoppers that made it impossible to do what the product was intended to allow. Trying to use the product, as a customer would, found these showstoppers in a minute. - Joel Spolsky, JoelOnSoftware.com Test Strategy Mistakes

18 Mistake #14: Untrained exploratory Testing Test Strategy Mistakes “Just think creatively” “Try to break it” Exploratory Testing is a discipline

19 Classic Mistake #15: Test ‘Engineers’ and ‘Executors’ Test Strategy Mistakes

20 Classic Mistake #16: Vacuous Documentation Examples: –The issue resolution document –Physical signoff/check marks –Elaborate test case templates Consequence: –Time spent documenting is time not spent testing Test Strategy Mistakes

21 Mistake #17: Trying to fix things beyond your reach The Meta-Mistake

22 Root Cause #1 Lack of Systems Thinking in Testing The law of unintended consequences Root Causes

23 Root Cause #2: Translation Problems Example: - “You need to completely test this module” Root Causes

24 Root Cause #3: Process Myopia Example: –The [in]famous Issue Resolution Document –“We don’t do things that way here” –Elevating process over skills Solutions: –The ear of the king / History Lessons –If your boss doesn’t care – ignore it Root Causes

25 A ‘new’ methodology Root Cause Analysis Pareto Analysis Drive out waste/tighten the feedback loop Then worry about better practices (Image from Rapid Development, (c) 1996 by Steve McConnell. Used with permission from the author) What to do tomorrow

26 Discuss Q&A

27 Where to go for more “Classic Mistakes in Software Testing”, Brian Marick, STQE, 1997 Rapid Development, Steve McConnell An Introduction to general systems thinking, Gerold Weinberg, 1975 Lessons Learned in Software Testing, Kaner, Bach, Pettichord

28 Bonus Section

29 Root Cause #4: Technology Myopia Example –“Use XML on the next project” –“I just bought 5 copies of WinRunner …” Solution: –If you’re technical, they need you to do it –If you’re a manager, focus on business impact and risk Root Causes

30 Root Cause #5: Fred Taylor in the organization Examples: –Factory Mentality –High Specialization –Mixing of skill sets is verboten Solutions: –Peopleware, or anything by Weinberg –First Break all the rules – Jim Collins –Lead. Insulate your team. Root Causes

31 Root Cause #6: Pressure for short-term results Example: –“Ship to make 4 th quarter numbers” Putting off problems instead of addressing them Solution: –Save your team –Professionalism means something –The Quake Example Root Causes


Download ppt "Classic Testing Mistakes: Revisited Matthew Heusser Presented at the Indiana QA Conference Indianapolis, Oct 13 th, 2006 Contributing."

Similar presentations


Ads by Google