SQS Group Limited Performance and Test-Driven Development: Are they compatible? Alan Gordon & David Evans Test Management Summit 2009

Test Management Summit 2009 – Performance and Test-Driven Development | Page 2 Alan does Performance Testing David does agile, test-driven development We seem to have different views on how things should be done We want your help to resolve this We want to know your experience and your views We want to know what you think of our ideas Performance Test-Driven Development The Premise

Test Management Summit 2009 – Performance and Test-Driven Development | Page 3 Agile Development basics Avoid ‘Big Design Up Front’ Develop iteratively, based on the Customer’s current priority Deliver valuable, working software at the end of every iteration Evolve functionality through User Stories User Story basics Stories represent tangible functionality for a user or other stakeholder “As a, I want, so that ” Stories are like ‘pieces of cake’ Vertical slices of usable functionality, not layers of architecture Performance Test-Driven Development Recap: Agile Software Development

Test Management Summit 2009 – Performance and Test-Driven Development | Page 4 Test-Driven Development basics Use testable acceptance criteria to elaborate each Story Specify and automate the tests before the Story is implemented The Story is 'done' when all the acceptance tests pass Re-test all completed Stories every iteration, to prevent regression TDD preferences Isolate the components under test Favour fast tests for fast feedback Use code coverage analysis to evaluate your test suite Performance Test-Driven Development Recap: Test-Driven Development

Test Management Summit 2009 – Performance and Test-Driven Development | Page 5 Performance Testing basics Test in a Live-like environment Use a dedicated Performance specialist team Test a stable application Test the whole system Use specialist tools Test end-to-end Pass / fail status may be subjective Schedule testing carefully to minimise cost Performance Test-Driven Development Recap: Performance Testing

Test Management Summit 2009 – Performance and Test-Driven Development | Page 6 Iterative development Testing in the iteration typically does not include Performance testing System under test ‘The whole’ is an evolving thing Knowledge The Customer has the final say on priority and acceptance criteria, but is probably no expert on performance Performance and other non-functional requirements Can be cumbersome to express as User Stories Environments The acceptance test environment is unlikely to be suitable for performance testing Performance Test-Driven Development Some Problems...

Test Management Summit 2009 – Performance and Test-Driven Development | Page 7 Iterative functional development, waterfall test Develop the functionality of the system iteratively Performance test at the end by a specialist team In-iteration performance testing All performance testing is included in each iteration by the agile testers Parallel non-functional testing team A separate NFT team tests the previous iteration’s output Performance Test-Driven Development Some Models...

Test Management Summit 2009 – Performance and Test-Driven Development | Page 8 What are your experiences? Have you encountered these issues? Have you encountered others? What are you views on the models? What are the pros and cons? What other models have you tried? Who should be doing the Performance testing? Testers embedded in agile team? Separate team dedicated to project? Centralised performance test service? How should the Customer prioritise performance issues in the backlog? Performance Test-Driven Development Discussion...

Test Management Summit 2009 – Performance and Test-Driven Development | Page 9 Avoid increasing ‘undone work’ every iteration Remove all faults as soon as possible after they are injected Educate the Customer about non-functional requirements Include performance requirements on Stories where relevant Describe overall NFR’s as Constraints Define quantifiable goals for non-functional qualities Gain as much useful information as quickly and cheaply as possible Refactor performance tests regularly Performance Test-Driven Development Some Thoughts...

Performance Test-Driven Development Some Good References on the topic Scott Ambler Tom & Kai Gilb Scott Barber Jamie Dobson Mike Cohn Test Management Summit 2009 – Performance and Test-Driven Development | Page 10

Thanks for your attention 7 Moorgate | London, EC2R 6AF, United Kingdom Tel.: +44 (0) | Fax: +44 (0) Internet: |