1. © Workroom Productions 2002 www.workroom-productions.com Slide 1 Further Adventures in Session-based Testing James Lyndsay Workroom Productions jdl@workroom-productions.com www.workroom-productions.com

2. © Workroom Productions 2002 www.workroom-productions.com Slide 2 ? What?

3. © Workroom Productions 2002 www.workroom-productions.com Slide 3 Session-Based Testing? Manages and controls unscripted testing Limited Duration Directed Exploration Recorded Activities Supports ‘Agile’ Process Allows quick metrics Has wider uses

4. © Workroom Productions 2002 www.workroom-productions.com Slide 4 Adventures? Two case studies Permanent team - 20 coders, 4 testers Inexperienced and under-motivated Internet-based product, ASP model Part-time team 2-6 coders, 0-3 testers bought in from pool of 6-8 No permanent test team Windows client / server product around known application Shared business context Rapid change Commercial constraints; budget, time-to-market

5. © Workroom Productions 2002 www.workroom-productions.com Slide 5 Further? Paper - Adventures in Session-based Testing Contains detailed methods Contains problems and (some) solutions Now a year old STARWest 2002 -Further Adventures... Methods Tools Training, Coaching

6. © Workroom Productions 2002 www.workroom-productions.com Slide 6 Strategy Stay within business constraints Deal with change rather than enforce stasis Rapidly transfer knowledge to allow independence Four goals Control the scope Control the work Measure risk and coverage Promote ongoing improvement

7. © Workroom Productions 2002 www.workroom-productions.com Slide 7 1 Methods

8. © Workroom Productions 2002 www.workroom-productions.com Slide 8 Methods: Overview Control and recording Control Scope Control Work Recording a Session Assessment and Measurement Assessing Risk Assessing Coverage Infrastructure Real-time Metrics Social techniques

9. © Workroom Productions 2002 www.workroom-productions.com Slide 9 Example: Test Points (Excel-based tool)

10. © Workroom Productions 2002 www.workroom-productions.com Slide 10 Methods: Control Scope - made of Test Points Test Point - piece of work Risk, Estimated cost Four groups New tests Regression tests Retests for fixes Investigatory tests Work done - made of Test Sessions Test Session - piece of time Actual cost, Coverage

11. © Workroom Productions 2002 www.workroom-productions.com Slide 11 Example: A Test Session

12. © Workroom Productions 2002 www.workroom-productions.com Slide 12 Methods: Control Recording a Test Session Plan: Approach, Scope, Expectations Record: Conditions, Actions, Observations Bug details Actual Cost How done are we?  Coverage Allows Review, coaching Forgetting Retrospectives

13. © Workroom Productions 2002 www.workroom-productions.com Slide 13 Methods: Assessment Risk Coverage Ask the Experts - the testers! How done are you? How much is left? Collect many subjective  few objective assessments Overall figures can go down as well as up Ongoing learning - estimates improved

14. © Workroom Productions 2002 www.workroom-productions.com Slide 14 Example: Metrics (Excel-based tool)

15. © Workroom Productions 2002 www.workroom-productions.com Slide 15 Methods: Support Real-time metrics Immediate feedback State of testing, not of system Fast, frequent, informed re-planning Soft skills/Social Techniques Experimental Approach Ongoing learning Communication Empowerment

16. © Workroom Productions 2002 www.workroom-productions.com Slide 16 2 Tools

17. © Workroom Productions 2002 www.workroom-productions.com Slide 17 Tools Freely available - under GPL Excel spreadsheet Single-user, single location Download from: Web: PHP/MySQL Multiple user Download from:

18. © Workroom Productions 2002 www.workroom-productions.com Slide 18 3 Training and coaching

19. © Workroom Productions 2002 www.workroom-productions.com Slide 19 Ongoing learning and development Training - from unskilled to skilled Skilled involvement - pair expert / novice Show - practice - challenge Careful choice of sessions Coaching - from good to great! Peer involvement, neutral assessment Challenge - innovate Discussion, metrics, personal investigation

20. © Workroom Productions 2002 www.workroom-productions.com Slide 20 Supporting the learning process Session-based testing Introduces discipline and framework Helps overlap training and testing Helps match challenge to individual Experts, managers - get involved! Sometimes compromise: Good bugs found quickly, or Better testers long-term Feedback is integral Before testing - techniques, planning, charter After testing - session log, bug log Stats and metrics - not just for management

21. © Workroom Productions 2002 www.workroom-productions.com Slide 21 Encouraging the team Involve the test team Allocate responsibility / ownership for tests Allow direct influence on clear, open metrics Relate metrics to business decisions Share successes, failures Reward discovery Discourage blind faith, boredom Step away - look at the wider picture

22. © Workroom Productions 2002 www.workroom-productions.com Slide 22 Changes: Case Study 1 Active learning process is encouraged Feedback has improved estimating skills Session review has improved test techniques Test Point ownership has improved planning and scoping skills Weekly 2-hour self-training session as a team Greater interest in testing, greater motivation Useful, visible, up-to-date metrics Increase openness, reduce surprise, blame Joint ownership of decisions, successes, failures

23. © Workroom Productions 2002 www.workroom-productions.com Slide 23 ! Conclusion

24. © Workroom Productions 2002 www.workroom-productions.com Slide 24 Conclusion Communication Sessions gave us the tools to communicate Better communication improved trust and effectiveness Ongoing improvement Sessions allow directed and disciplined approach Feedback improved techniques and estimation Empowerment increased morale and confidence Openness All figures available, all of the time, always current Encouraged trust and interest Allowed neutral feedback

25. © Workroom Productions 2002 www.workroom-productions.com Slide 25 Conclusion Test sessions provide: Control of unscripted testing Framework for ongoing improvement Overall lessons Simple measures are the best Favour effective communication and involvement over knee-jerk documentation Unobtrusive, immediate metrics allow real- time control and feedback

26. © Workroom Productions 2002 www.workroom-productions.com Slide 26 More Further details: Paper, updates, references etc. www.workroom-productions.com Contact: James Lyndsay Jdl@workroom-productions.com