1. Ideas for Rapid Test Management Jon Bach QA Manager 2/19/09 SASQAG

2. What is Rapid Testing? “The skill of testing any software, any time, under any conditions, such that your work stands up to scrutiny. The closest thing in the business to a martial art of software testing.”

3. Skills Some Exploration Skills and Tactics Modeling Resourcing Questioning Recording Reporting Chartering Observing Manipulating Pairing Generating/Elaborating Refocusing Alternating Branching/Backtracking Conjecturing “MR.Q COMP GRABC R&R?”

4. More time than ALL of your time Everything is pri 1 Multi-tasking No specs No time for training Cubicles / team doesn’t share info with each other Heavy tools Interruptions Conditions

5. Scrutiny Programmers Business Analysts Test Managers Directors Customers Other Testers You

6. Backstory Start Date Projects Staff Mission Values

7. Problems Who’s on your team? How do you find time to learn? How does exploration fit? What’s your test approach? What are you doing right now? Where are we? How we doin’? How did you do that? Can we do this more smartly?

8. Ideas Surveys Open-Book Testing Dawn Patrol – SBTM Color-aided Test Design Testing Pyramid Hybrid Start / End daily status Drive-by learning Lightning talks Tools: WME, SWP, PK

9. Ideas Surveys Open-Book Testing Dawn Patrol – SBTM Color-aided Test Design Testing Pyramid Hybrid Start / End daily status Drive-by learning Lightning talks Tools: WME, SWP, PK

10. Surveys… What time works for you for a one-on-one with me this week? Do you have any ideas of how to dogfood Concordance? What are your working hours? What’s you biggest annoyance when working? What’s the name of your spouse / children? What are 3 aspects of your ideal manager? What’s the best business / testing book you’ve read? What’s the best overall book you’ve read? What one project problem do you wished was solved NOW? What aspect of working here is most important to you?

11. Survey Says… Team meeting consisting of revealing the answers together -- Family Feud style

12. Speed round… 1) Name a LexisNexis customer that tends to be mentioned a lot in meetings. 2) Jon has one full row of books on the bookshelf in his cube (standard size). How many books are on it? 3) Name the genre of book that most of the staff says they like. 4) Name the most interesting project codename we’ve used (any project). 5) Name a popular first name for staff on all four project teams. 6) Name a test management tool we use. 7) Name a neighboring state to Ohio. 8) What would you say is the average amount of emails each person on Jon’s staff gets per day? 9) Name another state (other than Washington) where LexisNexis Home-Based Employees (LAW, Concordance, TL, or CaseMap) are based? 10) Name a resolution category for a bug that would be assigned back to you.

13. Ideas Surveys Open-Book Testing Dawn Patrol – SBTM Color-aided Test Design Testing Pyramid Hybrid Start / End daily status Drive-by learning Lightning talks Tools: WME, SWP, PK

14. What is Open-Book Testing? The act of assigning testers and students open-ended questions such that…

15. TestersInstructors and managers …are immersed in the product right away, building a model or mind map. …create test “charters”, or missions that seek to reveal many bugs instead of one (akin to using scenarios and personas). …learn how they are provoked into critical thinking by being exposed to many types of questions (test ideas). …learn how their students are provoked to know if they can perform critical thinking. … quickly find bugs and raise issues in answering the questions they are given …evaluate tester skill and test coverage.

16. Excerpts from Flight Sim Ground School exam: Which airplanes are equipped with slotted flaps? Describe the steps needed to start the engine of the DC3. If the vacuum system failed in the 172, what instruments would no longer function? Joe, an aviation history fanatic, buys Combat FS and complains to tech support that the P-51 flight model is “all wrong” because in a dogfight, “the airplane stalls at 200 MPH.” Joe flies with the Realism Settings set to Hard and he knows that the P-51 should stall at about 100 MPH. Explain to Joe how it is possible for an airplane to stall at twice its published stalling speed.

17. Closed-book exams test knowledge. Open-book exams test resourcefulness. “Imagination is more important than knowledge” -- Einstein

18. The answer may matter less than how it was derived. For example, Which airplanes are equipped with slotted flaps? …can lead to the following strategies to find the answer: Select aircraft / spot plane view should show me the type Aircraft details: should tell me the type of flaps Flap function (shortcut key, panel icon, keyboard) can be tested to verify the flap animation is consistent with model Flight model affect (whether the flaps behave as slotted) can be tested in flight

19. OBT questionThe spirit of the test Which airplanes are equipped with slotted flaps? Confirm or refute a rumor that the texture maps for slotted flap animations weren’t ready, but were dropped into the build anyway Describe the steps needed to start the engine of the DC3. Investigate a claim from the Developer, a DC3 pilot, that omissions in the “pilot checklist” feature are now fixed If the vacuum system failed in the 172, what instruments would no longer function? When enabled, does the “system failures” feature show that the altimeter and airspeed indicators are incorrect? Can a 737 stall at a speed that’s twice as fast as its published stalling speed? Any plane should stall at any speed if the attitude of the angle of attack is too steep.

20. Ideas Surveys Open-Book Testing Dawn Patrol – SBTM Color-aided Test Design Testing Pyramid Hybrid Start / End daily status Drive-by learning Lightning talks Tools: WME, SWP, PK

21. Dawn Patrol A SWAT team of hardcore testers who do exploratory testing when no one is looking.

22. Jon Bach / Quardev Laboratories, Seattle, WA 98119 Some Exploration Skills and Tactics Modeling Resourcing Questioning Recording Reporting Exploratory testing is a mindset using this skillset. Chartering Observing Manipulating Pairing Generating/Elaborating Refocusing Alternating Branching/Backtracking Conjecturing “MR.Q COMP GRABC R&R?”

23. Jon Bach / Quardev Laboratories, Seattle, WA 98119 Scrutiny What’s being askedWhat they may be thinking What was your mission?Remind me what I told you to do. Why did you do that? How did it go?Were you careful or reckless? What should I be worried about? How far did you get?Are we closer to shipping? Can you help me know our status? Need anything?Can I speed this along? Do you need more of my time? When will you be done?Will I get my bonus? I have a new task for you…

24. Jon Bach / Quardev Laboratories, Seattle, WA 98119 Session-Based Test Management: 1) Time Box 2) Reviewable Result 3) Debriefing The “Session”

25. Jon Bach / Quardev Laboratories, Seattle, WA 98119 The deliverable: a session report Charter –#AREAS CHARTER ----------------------------------------------- Analyze MapMaker’s View menu functionality and report on areas of potential risk. #AREAS OS | Windows 2000 Menu | et-jsb-010417-c.sesViewet-jsb-010417-c.ses Strategy | Function Testing Strategy | Functional Analysis Sample START ----------------------------------------------- 5/30/00 03:20 pm TESTER ----------------------------------------------- Jonathan Bach TASK BREAKDOWN ----------------------------------------------- #DURATION short #TEST DESIGN AND EXECUTION 65 #BUG INVESTIGATION AND REPORTING 25 #SESSION SETUP 20 Metrics – #DURATION – #TEST DESIGN AND EXECUTION – #SESSION SETUP – #BUG INVESTIGATION AND REPORTING – #CHARTER / OPPORTUNITY Issues – #ISSUE Bugs – #BUG Notes

26. Jon Bach / Quardev Laboratories, Seattle, WA 98119 Are you accountable? Was this mission reasonable? What did you find? How did you spend your time? Did you need some help / tools? Do you think there’s more to do here? Agenda: “PROOF” Past Results Obstacles Outlook Feelings

27. Jon Bach / Quardev Laboratories, Seattle, WA 98119 Sample project HTML status report

28. Ideas Surveys Open-Book Testing Dawn Patrol – SBTM Color-aided Test Design Testing Pyramid Hybrid Start / End daily status Drive-by learning Lightning talks Tools: WME, SWP, PK

29. Jon Bach / Quardev Laboratories, Seattle, WA 98119 Color-aided Test Design Color-aided test design is a method of test creation which aims at giving value by shaping, finding relevance and organizing tests based on over arching and broad testing concepts. By categorizing tests at a high level, test concepts designated by color, we can quickly and easily focus our aim and maximize reliable test coverage in a collaborative method to expose risks in the test plan.

30. Jon Bach / Quardev Laboratories, Seattle, WA 98119

31. Color-aided Test Design

32. Ideas Surveys Open-Book Testing Dawn Patrol – SBTM Color-aided Test Design Testing Pyramid Hybrid Start / End daily status Drive-by learning Lightning talks Tools: WME, SWP, PK

33. Jon Bach / Quardev Laboratories, Seattle, WA 98119 Testing Pyramid

34. Ideas Surveys Open-Book Testing Dawn Patrol – SBTM Color-aided Test Design Testing Pyramid Hybrid Start / End daily status Drive-by learning Lightning talks Tools: WME, SWP, PK

35. Jon Bach / Quardev Laboratories, Seattle, WA 98119 Hybrid A test document that combines exploratory and matrix-driven verification testing. Sample

36. Ideas Surveys Open-Book Testing Dawn Patrol – SBTM Color-aided Test Design Testing Pyramid Hybrid Start / End daily status Drive-by learning Lightning talks Tools: WME, SWP, PK

38. Ideas Surveys Open-Book Testing Dawn Patrol – SBTM Color-aided Test Design Testing Pyramid Hybrid Start / End daily status Drive-by learning –Lightning talks –Combo lock Tools: WME, SWP, PK

41. Tools ScrumWorksPro Windows Media Encoder Perfect Keyboard SourceForge wiki

42. Ideas ScrumWorksPro Windows Media Encoder Perfect Keyboard SourceForge wiki

43. Ideas ScrumWorksPro Windows Media Encoder Perfect Keyboard SourceForge wiki

44. Wiki

45. Recap “The skill of testing any software, any time, under any conditions, such that your work stands up to scrutiny. The closest thing in the business to a martial art of software testing.”

46. Problems Who’s on your team? How do you find time to learn? How does exploration fit? What’s your test approach? What are you doing right now? Where are we? How we doin’? How did you do that? Can we do this more smartly?

47. Bake your half-baked ideas Surveys Open-Book Testing Dawn Patrol – SBTM Color-aided Test Design Testing Pyramid Hybrid Start / End daily status Drive-by learning Lightning talks Tools: WME, SWP, PK