1. QWise software engineering – refactored! www.qwise.se Testing, testing A first-look at the new testing capabilities in Visual Studio 2010 Mathias Olausson Callista Knowledgebase | QWise

2. QWise software engineering – refactored! www.qwise.se About Mathias...

3. QWise software engineering – refactored! www.qwise.se About the Presentation... What’s on the menu? – Test planning and execution – Issue tracking with rich bugs – Eliminating the "works on my machine / cannot reproduce" scenario – Automated UI Testing – Virtual Lab Management

4. QWise software engineering – refactored! www.qwise.se TEST PLANNING AND EXECUTION

5. QWise software engineering – refactored! www.qwise.se Agile Testing Agile projects require different testing Incremental = re-test Iterative = test often Test documentation may be scoped out Development and test more integrated Focus on automatic testing

6. QWise software engineering – refactored! www.qwise.se Core Practices Code and Test as One Process Incremental Work Continuous Integration Controlled Test Environments

7. QWise software engineering – refactored! www.qwise.se Agile Testing Quadrants Functional Tests Exploratory Tests Unit Tests Component Tests Unit Tests Component Tests Performance Tests Load Tests Security Tests ”-ilitets” Tests Performance Tests Load Tests Security Tests ”-ilitets” Tests Business-facing Technology-facingSupporting the team Critisizing the product Automated & Manual Manual Automated

8. QWise software engineering – refactored! www.qwise.se Test Types Manual Tests UI Tests Acceptance Tests Unit Tests / Component Tests

9. QWise software engineering – refactored! www.qwise.se Testing in an Iteration sprint plan write tests for us1 write tests for us2 sprint plan build 1 build 2 build 3 implement user story ‘1’ (us1) test us1 & file bugs Implement US2 build 4 fix bugs build 6 build 7 regress impacted tests build 5 implement us2 fix bugs test us2 & file bugs verify fixes

10. QWise software engineering – refactored! www.qwise.se Microsoft Test Manager and Visual Studio Team Foundation Server Reporting Development (Teams) Testing (Teams) Web Test Load Test Coded UI Test Unit Testing Test Case Management Lab Management Microsoft Test Runner Fast Forward For Manual Testing Generalist Specialist Engineering Support Data Collectors (IntelliTrace, Test Impact, Metrics)

11. QWise software engineering – refactored! www.qwise.se The Fundamentals

12. QWise software engineering – refactored! www.qwise.se Defining and Running Tests

13. QWise software engineering – refactored! www.qwise.se ISSUE TRACKING WITH RICH BUGS

14. QWise software engineering – refactored! www.qwise.se Should We Do Bug Tracking? Pros – Tracability – Priority of work – Visibility Cons – Overhead Should we log bugs within an iteration? Source: Pixar ”A bugs’ life”

15. QWise software engineering – refactored! www.qwise.se Defect Tracking in MTM and MTR

16. QWise software engineering – refactored! www.qwise.se No-more-no-repro Automatic data collection Test settings – Configuration of diagnostic data collectors – Assigned to test plan Or per test run

17. QWise software engineering – refactored! www.qwise.se Diagnostic Data Adapters

18. QWise software engineering – refactored! www.qwise.se No-more-no-repro

19. QWise software engineering – refactored! www.qwise.se AUTOMATED UI TESTING

20. QWise software engineering – refactored! www.qwise.se Why Automated Testing? Källa: Tools for Agility, Kent Beck

21. QWise software engineering – refactored! www.qwise.se iteration 1 iteration 2 release iteration feature a feature b feature c feature d unit testing manual testing regression testing performance testing feature a Automation Strategy

22. QWise software engineering – refactored! www.qwise.se Coded UI Tests Unit test for UI High-level framework Create test from – Recording – Test result (action log) – Using API

23. QWise software engineering – refactored! www.qwise.se Automation Platform Support

24. QWise software engineering – refactored! www.qwise.se Coded UI Tests

25. QWise software engineering – refactored! www.qwise.se LAB MANAGEMENT

26. QWise software engineering – refactored! www.qwise.se Motivation Complex environments hard to set up Many configurations to test Poor utilization of hardware Hard bugs take long to reproduce Agile practices require shorter cycles

27. QWise software engineering – refactored! www.qwise.se Lab Management Aims To... Automate deployment Improve test effectiveness Speed up testing Quickly set up new environments

28. QWise software engineering – refactored! www.qwise.se Lab Management Infrastructure

29. QWise software engineering – refactored! www.qwise.se Environments An environment is a set of machines with different roles An environment can – Be deployed, saved, removed – Started, stopped – Snapsshot VM1 Lab Agent VM2 Lab Agent

30. QWise software engineering – refactored! www.qwise.se Manual Tests in Lab Testing from the outside – Easy for web applications – Limitations in data collection Testing from the inside – Run application in lab via environment viewer

31. QWise software engineering – refactored! www.qwise.se Automatic Tests in Lab Testing in VM – Test case connected to test environment – Test component comes from selected build Testing in build – Test case connected to build definition – Runs test list or test component

32. QWise software engineering – refactored! www.qwise.se Lab Management Build Workflow build app. restore env. deploy app. take env. snapshot run tests restore snapshot

33. QWise software engineering – refactored! www.qwise.se Build and Deploy

34. QWise software engineering – refactored! www.qwise.se Summary Structured test planning with MTM Integrated test execution Effective defect tracking with rich bugs Seamless integration with automated tests Powerful lab management

35. QWise software engineering – refactored! www.qwise.se Questions? mathias.olausson@callista.se http://msmvps.com/blogs/molausson