2. DEV233 Team Foundation: Using Metrics to Manage and Troubleshoot Your Projects Sam Guckenheimer Group Product Planner Microsoft Corporation

3. Managing 20 th Century Projects Functionality Time Resources Quality “The Iron Triangle” (err… tetrahedron) Image copyright Tetra Pak

4. 21 st Century Mantra Do more with less! But if your only variables are: FunctionalityResourcesTimeQuality …then how are you going to do that?

5. An Older Truth Happy families are all alike; every unhappy family is unhappy in its own way. Tolstoy, Anna Karenina Все счастливые семьи похожи друг на друга, каждая несчастливая семья несчастлива по-своему.

6. Two Process Paradigms Traditionally, we decompose tasks and measure completion This is the cost accounting tradition Alternatively, we account for customer value incrementally delivered This is the Lean Manufacturing and Theory of Constraints model

7. Two Process Paradigms Work-downValue-up Planning and change process Get planning and design right up front Change happens, embrace it Primary measurementTask completionOnly deliverables that the customer count Definition of qualityConformance to specificationValue to the customer Acceptance of varianceTasks can be identified and estimated deterministically Variance is part of all process flows Intermediate work products Documents, models, and other artifacts are necessary Just enough to minimize the uncertainty Troubleshooting approach Time, resource, functionality and quality Find and remove the bottlenecks Approach to TrustMonitor and measure; incent performance relative to plan Pride of workmanship and teamwork

8. Prescriptive vs. Descriptive Metrics Robert D. Austin, Measuring and Managing Performance in Organizations Metrics lose their meaning when made prescriptive

9. Standard Reporting with Team Foundation

10. How Far Can We Get in The Available Time? Work planned Work completed

11. How Effectively Is Our (Outsourced) Team Delivering? Test rates (pass, inconclusive, fail) shown in bars Against code coverage, … code churn, … and active bugs

12. What Requirements Haven’t Been Tested? Tracks progression of requirements’ states from Untested to Passed by successive Build

13. Are We Getting Better?

14. Custom Reporting with Team Foundation

15. Signs of Unhappiness Underestimating Uneven Task Decomposition Scope Creep Inadequate Bug Allotment Resource Leaks Development Practices Too Loose Build Failures Inadequate Unit Testing Tests Passing; Solution Doesn’t Work Solution Stuck in Testing Too Little Testing

16. Underestimating Steady rates of progress, but slope too shallow Slow progress leading to cuts in planned work, but not enough cuts

17. Uneven Task Decomposition High variance in task completion/day indicates inconsistency

18. Scope Creep “Dark matter” emerging during iteration Planned work is squeezed out

19. Inadequate Bug Allotment New work not planned at iteration start

20. Resource Leaks Sudden drop in velocity indicates other drain on resources

21. Development Practices Too Loose Growing “Fault Feedback Ratio” – Bugs requiring multiple handling

22. Build Failures

23. Inadequate Unit Testing Rising Code Churn Falling Code Coverage Fewer Passing and More Inconclusive Tests

24. Tests Passing; Solution Doesn’t Work Flat Big Rate Flat & Low Code Coverage

25. Tests Passing; Solution Doesn’t Work Find > Fix leads to growing bug backlog above

26. Tests Passing; Solution Doesn’t Work Tests aren’t catching or regressing bugs

27. Solution Stuck in Testing Bulge in resolved → Insufficient resources or inadequate quality from dev

28. Too Little Testing Testing Rates Stuck Low

29. Drilldown with Microsoft Office Excel

30. Friction-Free Metrics Enable Transparency

31. Visual Studio Team System Visual Studio Team Suite MSF Process and Guidance Visual Studio Team Foundation Server Visual Studio Industry Partners Software Architects Software Developers Software Testers Database Professionals Visual Studio Team Explorer Application Modeling Infrastructure and Deployment Modeling Code Analysis Performance Tuning Security Analysis Database Deployment Database Change Management Database Testing Performance Testing Manual Testing Test Case Management Visual Studio Professional Edition Change Management Work Item Tracking Reporting Project Site Integration Services Project Management Load Test Agent Visio and UML Modeling Class Modeling Unit Testing Code Coverage New!

32. Naked Self-Promotion For More Information…

33. Resources Technical Chats and Webcasts http://www.microsoft.com/communities/chats/default.mspx http://www.microsoft.com/usa/webcasts/default.asp Microsoft Learning and Certification http://www.microsoft.com/learning/default.mspx MSDN & TechNet http://microsoft.com/msdn http://microsoft.com/technet Virtual Labs http://www.microsoft.com/technet/traincert/virtuallab/rms.mspx Newsgroups http://communities2.microsoft.com/ communities/newsgroups/en-us/default.aspx Technical Community Sites http://www.microsoft.com/communities/default.mspx User Groups http://www.microsoft.com/communities/usergroups/default.mspx

34. Fill out a session evaluation on CommNet and Win an XBOX 360!

35. © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.