Presentation on theme: "DEV233 Team Foundation: Using Metrics to Manage and Troubleshoot Your Projects Sam Guckenheimer Group Product Planner Microsoft Corporation."— Presentation transcript:
DEV233 Team Foundation: Using Metrics to Manage and Troubleshoot Your Projects Sam Guckenheimer Group Product Planner Microsoft Corporation
Managing 20 th Century Projects Functionality Time Resources Quality “The Iron Triangle” (err… tetrahedron) Image copyright Tetra Pak
21 st Century Mantra Do more with less! But if your only variables are: FunctionalityResourcesTimeQuality …then how are you going to do that?
An Older Truth Happy families are all alike; every unhappy family is unhappy in its own way. Tolstoy, Anna Karenina Все счастливые семьи похожи друг на друга, каждая несчастливая семья несчастлива по-своему.
Two Process Paradigms Traditionally, we decompose tasks and measure completion This is the cost accounting tradition Alternatively, we account for customer value incrementally delivered This is the Lean Manufacturing and Theory of Constraints model
Two Process Paradigms Work-downValue-up Planning and change process Get planning and design right up front Change happens, embrace it Primary measurementTask completionOnly deliverables that the customer count Definition of qualityConformance to specificationValue to the customer Acceptance of varianceTasks can be identified and estimated deterministically Variance is part of all process flows Intermediate work products Documents, models, and other artifacts are necessary Just enough to minimize the uncertainty Troubleshooting approach Time, resource, functionality and quality Find and remove the bottlenecks Approach to TrustMonitor and measure; incent performance relative to plan Pride of workmanship and teamwork
Prescriptive vs. Descriptive Metrics Robert D. Austin, Measuring and Managing Performance in Organizations Metrics lose their meaning when made prescriptive
Standard Reporting with Team Foundation
How Far Can We Get in The Available Time? Work planned Work completed
How Effectively Is Our (Outsourced) Team Delivering? Test rates (pass, inconclusive, fail) shown in bars Against code coverage, … code churn, … and active bugs
What Requirements Haven’t Been Tested? Tracks progression of requirements’ states from Untested to Passed by successive Build
Are We Getting Better?
Custom Reporting with Team Foundation
Signs of Unhappiness Underestimating Uneven Task Decomposition Scope Creep Inadequate Bug Allotment Resource Leaks Development Practices Too Loose Build Failures Inadequate Unit Testing Tests Passing; Solution Doesn’t Work Solution Stuck in Testing Too Little Testing
Underestimating Steady rates of progress, but slope too shallow Slow progress leading to cuts in planned work, but not enough cuts
Uneven Task Decomposition High variance in task completion/day indicates inconsistency
Scope Creep “Dark matter” emerging during iteration Planned work is squeezed out
Inadequate Bug Allotment New work not planned at iteration start
Resource Leaks Sudden drop in velocity indicates other drain on resources
Development Practices Too Loose Growing “Fault Feedback Ratio” – Bugs requiring multiple handling
Inadequate Unit Testing Rising Code Churn Falling Code Coverage Fewer Passing and More Inconclusive Tests
Tests Passing; Solution Doesn’t Work Flat Big Rate Flat & Low Code Coverage
Tests Passing; Solution Doesn’t Work Find > Fix leads to growing bug backlog above
Tests Passing; Solution Doesn’t Work Tests aren’t catching or regressing bugs
Solution Stuck in Testing Bulge in resolved → Insufficient resources or inadequate quality from dev
Too Little Testing Testing Rates Stuck Low
Drilldown with Microsoft Office Excel
Friction-Free Metrics Enable Transparency
Visual Studio Team System Visual Studio Team Suite MSF Process and Guidance Visual Studio Team Foundation Server Visual Studio Industry Partners Software Architects Software Developers Software Testers Database Professionals Visual Studio Team Explorer Application Modeling Infrastructure and Deployment Modeling Code Analysis Performance Tuning Security Analysis Database Deployment Database Change Management Database Testing Performance Testing Manual Testing Test Case Management Visual Studio Professional Edition Change Management Work Item Tracking Reporting Project Site Integration Services Project Management Load Test Agent Visio and UML Modeling Class Modeling Unit Testing Code Coverage New!
Naked Self-Promotion For More Information…
Resources Technical Chats and Webcasts Microsoft Learning and Certification MSDN & TechNet Virtual Labs Newsgroups communities/newsgroups/en-us/default.aspx Technical Community Sites User Groups
Fill out a session evaluation on CommNet and Win an XBOX 360!