COMP3001 Technology Management & Professional Issues: Project Management Metrics appropriate for iterative projects Lecture 6 Graham Collins, UCL

Development Metrics (measurements)  Measurements have traditionally included lines of code (LOC) and there are several models based on this including the hierarchy of cost and effort models COCOMO (COnstructive COst MOdel) developed by Boehm  More recently other metrics have been used such as function points, which give a better indication of size and complexity  Recent development work (Bittner and Spence) indicates that a move to working software is perhaps one of the best measures of progress. What is iterative development? Part 3: The management perspective 15 May 2005 www-128.ibm.com/developerworks/rational/library/may05/bittner-spence/index.html

Iterative projects  From the project managers perspective iterative projects can be viewed as a series of self contained projects with the application of all the disciplines of software development (requirement, analysis, design, implementation, and testing) to produce a release of the project  With the Unified Process initial iterations may establish design including architecture. With agile methods emphasis is placed on releases of working software  Bittner and Spence outline this at its simplest as series of stages.

Earned Value compared to Agile Process Planning Based on predictive planning Reliable estimates effort, cost and completion date End-to-end value tracking Adaptive planning. Iteration to iteration tracking Predication of the next iterations effort It is possible to schedule most of the activities Adaptation to unpredictable events is problematic. Changes may require the planned to be revised or baselined Near the beginning, it is not always possible to schedule. Time based iterations allow initial estimate of duration which can be revised through the adaptive driven build-feedback cycle Estimates based on past performanceEstimates are based on progress being made (velocity) Change rates often lowUnpredictable change the norm Small variations in early measurements of cost and time at the start the project give wide variation in forward predications Unknown team development rates Some organisations do not chart progress for an initial period Progress is tracked immediately Earned value well establishedPrioritization of the value of user stories No earned value approaches in methods Earned Value Agile Development

CMMI Comparative Advantages Grants explicit freedom to select the order of improvement that best meets the organization’s business objectives and mitigates the organisation’s areas of risk Enables organisations to have a predefined path Enables increased visibility of the capability achieved in each individual process area Focuses on a set of processes that provide an organization with a specific capability that is characterized by each maturity level Provides a capability-level rating that is used primarily for improvement in an organisation and is rarely communicated externally Provides a maturity-level rating that is often used in internal management communication, statements external to the organization, and during acquisitions as a means to qualify bidders Allows improvements of different processes to be performed at different rates Summarizes process-improvement results in a simple form – a single maturity-level number Reflects a newer approach that does not yet have the data to demonstrate its ties to return on investment Builds on a relatively long history of use that includes case studies and data that demonstrate proved return on investment Continuous Representation Staged Representation

Agile Manifesto Individuals and interactionsover processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more Several agile projects have achieved CMMI level 3, example David Anderson, Stretching Agile to fit CMMI Level 3, Agile Conference 2005

The Agile Principles Our highest priority is to satisfy the customer through early and continuous delivery of valuable software Agile processes promote sustainable development Welcome changing requirements, even late in development. Agile processes harness change for the customer competitive advantage The sponsors, developer, and users should be able to maintain a constant pace indefinitely Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter time scale Continuous attention to technical excellence and good design enhances agility Business people and developers must work together daily throughout the project Simplicity - the art of maximizing the amount of work done – is essential Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. The best architectures, requirements, and designs emerge from self-organizing teams The most efficient and effective method of conveying information to and within a development team is face-to-face conversation At regular intervals the team reflects on how to become more effective, then tunes and adjusts its behaviour accordingly Working software is the primary measure of progress

Iterative Development (Bittner-Spence) 1. Agree with the team the objectives for the iteration, including evaluation criteria, timescales, and constraints 2. Agree on a plan for how the team will achieve the objectives 3. Execute the plan 4. Assess the achievements of the team against the initial set of objectives and evaluation criteria 5. Assess the impact of the iteration’s results on the project as a whole 6. Start the next iteration. What is iterative development? Part 3: The management perspective 15 May 2005 www-128.ibm.com/developerworks/rational/library/may05/bittner-spence/index.html

Fundamental shift in measurement Progress ( % complete measured in scenarios 100% 0% Iteration codedtestedTested & Passed

Developer Perspective  Developers are less interested in the business value, benefits realization and return on investment  They work on a small number of requirements or change requests from their list of outstanding work  They anticipate a decreasing number of requirements and change requests as the product is developed  Outstanding requirements and change requests is termed the product backlog  The developer will therefore be aware of progress via work completed, product backlog and new work allocated.

User Satisfaction driving Development User satisfaction Iteration Release User satisfaction Release planning DevelopmentIncrementIteration planning Iteration Planning (Goal identification, story selection, tasks, estimation, team commitment)

To develop members’ capabilities; to build and exchange knowledge Passion, commitment, and identification with the group’s expertise To accomplish a specified task The project’s milestones and goals Adapted from: Communities of Practice: The organizational Frontier, Etienne C. Wenger and William M. Snyder, Harvard Business Review p Jan-Feb 2000 What is the purpose?What holds it together? Community of practice Project team Project teams need to adopt some attributes

Rate of work - velocity

Individuals and Moving Range (XmR) Charts

Control Limits for XmR Charts k sequential measurements provide k-1 =r (two-point) moving range values ith moving range = mRi = │Xi+1 – X i │where integer i is 1 ≤ i ≤ k - 1 ___ i=r Individuals average moving range =mR = 1 ∑ mR i r i=1 _ ___ _ ___ Upper Natural Process Limit =UNPL x = X + 3mR = X mR d2 _ i=k Centerline = CL x = X = 1 ∑ Xi (average of individual values) k i=1 _ ___ _ ___ Lower Natural Process Limit =LNPL x = X - 3mR = X mR d2 ___ Centerline or average moving range = CL R = mR ___ ___ Upper Control Limit for moving range =UCL R = D4mR = 3.268mR ___ Sigma for individual values = sigmax (σ) = mR d2 When n=2 d2 =1.128and D4 =3.268 (from Dispersion and Bias factor tables)

‘Under Control’ Velocity measures of work rate are useful in that estimates of the next iteration can be planned in a rolling process The use of σ variation is supportive in this aim Automated colour coding (Red Amber Green) can be used to show condition requirements

‘Burn-down’

With the appropriate metrics we can improve

Use of Multipliers Iterations Completed Low MultiplierHigh Multiplier or more Multipliers for estimating velocity based on number of iterations completed from Cohn 2006

Charts and Metrics  Velocity and Burn-down  Cumulative acceptance tests Inventory Failing Passing  Cumulative Issue Charts Backlog - Active issues (which show inventory line) Resolved issues Closed issues  Earned Value EV progress charts Performance via cpi and spi Cpi and spi combined with control charts

Earned Value  EV can be applied to estimates of agile projects - this is complex if more stories are added as the work progresses  EV may need to be shown to senior managers - who are used to EV figures, or comparison to other projects where EV figures have been tracked  EV estimates can be accurate - story points tend to remain static in an iteration when the process is understood by managers and developers. When additional stories are added, stories with lower business priority level may be dropped to compensate and keep the work load (story points) similar.

Business Value More importantly business value (or contribution) should be considered and evaluated Often units of measure such as story points can be valued as 0.5 or 1.0 units The key issue in agile project management is to continually assess with the client the most important work that should be done. Story number ‘Business Value’ Story Points ‘Points earned’ Planned (developer hours) Actual (logged hours) EV (earned value) total

Further reading  Agile Estimating and Planning, Mike Cohn, Prentice Hall (Pearson Education) 2006 ISBN:  Managing Agile Projects, Sanjiv Augustine, Prentice Hall (Pearson Education) 2005 ISBN: