1. Mike Azocar Sr. Developer Technical Specialist Microsoft Corporation mazocar@microsoft.com

4. “we have a changed requirement, what do I need to test?” “my testers are spending too long testing the same thing” “tooling is expensive (time, licenses, people)” “developers and testers work in silos and don’t communicate/speak the same language” “when is my software ready to ship?” “the developers say the defects are useless”

5. Manager

6. 68%

7. reduce the time it takes to determine the root cause of a bug enable users to easily run acceptance tests and track their results reduce the time it takes to verify the status of a reported bug fix reduce the time it takes to replicate a bug uncovered by user actions reduce the time it takes to isolate differences between the test and production environment

8. 8

9. release planning set test strategy define done, done establish environments review configurations create plans generate data multiple iterations inception construction release release iteration regression testing release doneness testing release sign off iteration planning add stories to plan define acceptance select regression tests iteration execution author tests run tests file bugs verify fixes automate tests iteration retrospective update master plan identify product debt identify test debt select test for automation

10. GeneralistSpecialist Manual Testing Some scripting Creates scripts to set up lab, create data Strong scripting skills Some coding skills Strong coding Develops fully automated testing procedures Expert coding skills Black Box Testing White Box Testing API Testing

11. coded ui test unit testing web performance test load test test runner test case management lab management data collectors (video, action log, event log etc) team foundation server with reporting (bugs, test cases, requirements, user stories, source control, build) team foundation server with reporting (bugs, test cases, requirements, user stories, source control, build) generalistspecialist

12. 12

13. release planning set test strategy define done, done establish environments review configurations create plans generate data iteration planning add stories to plan define acceptance select regression tests iteration execution author tests run tests file bugs verify fixes automate tests iteration retrospective update master plan identify product debt identify test debt select test for automation release iteration regression testing release doneness testing release sign off inception construction release

14. Virtual Machines Hyper-V Host Automated Build- Deploy-Test Workflow Reduces Hardware Utilization Remote Diagnostic Collection Virtual Environment Setup/Tear down Built on Hyper-V Integrated with TFS 2010 Enabled by SCVMM Integrated management Multi-hypervisor management P2V and V2V Template deployment Optimize resource usage Managed by Microsoft Test Manager

15. Library Share 1Stored VM 1 Stored VM 2 Stored VM 3 Build Controller Test Controller Physical Environment / ESX Build Agent Test Agent Machine 1 Machine 2 Machine 3 Lab Management Build Management Test Case Management Project Management Source Control Work Item Tracking Reporting TFS 2010 Visual Studio 2010Microsoft Test Manager Hyper-V Host 1 Build Agent Lab Agent Test Agent Virtual Machine 1 Virtual Machine 2 Virtual Machine 3

16. Windows Vista SCVMM VM Template Library Sysprepped “gold images” Windows Server 2003 Windows Server 2008 …etc. Stored in SCVMM Library Windows 7 Windows Server 2008 Test Agent Build Agent Lab Agent Windows Server 2003

17. Microsoft Confidential 17

18. Build progress overtime is tracked Recent build status is reported to the team

19. Are we getting enough coverage? How many lines of code are we churning?

20. Workflow run for Deployment and Lab Restore snapshots create new ones after build Test against Lab Environments Automatically

21. Failing Tests are Flagged for further review Amount of Code Covered by Test is show Amount of code checked in per build

22. Microsoft Confidential 22

27. 27