5Objectives of software measurement “You can not control what you cannot measure.” – Tom DeMarco“Not everything that counts can be counted. Not everything that is counted counts.” – Albert Einstein
6Software Metrics Numerical data related to software development Strongly support software project management activitiesCan be directly observable quantities or can be derived from one
7A simplified measurement information model productsDecisions / ActionsMeasurementsInformationneedsInformationNeeds,Objectives,ControlAttributesProcessWork ProductsResultsRef: Ebert and Dumke 2007
8How the software measurements are used? Understand and communicateSpecify and achieve objectivesIdentify and resolve problemsDecide and Improve
9Measurement Standard How to do How to do better ISO/IEC 12207 Software Life Cycle ProcessesISO/IEC 15288System Life Cycle processesSWEBOKSoftware Engineering Body of KnowledgePMBOKProject Management Body of KnowledgeCMMICapability Maturity Model IntegrationISO 15504Software Process Capability DeterminationISO 9001Quality Management SystemISO/IEC 9126Software Product QualityTL 9000, AS 9100, etc.Objectives adaptationsHow to measure what you are doingISO/IEC 15939:2002Software Measurement Process
10Ground rules for a Metrics Metrics must beUnderstandable to be usefulEconomicalField testedHighly leveragedTimelyMust give proper incentives for process improvementEvenly spaced throughout all phases of developmentUseful at multiple levels
11Measurements for Senior Management Easy and reliable visibility of business performanceForecasts and indicators where action is neededDrill-down into underlying information and commitmentsFlexible resource refocus
12Measurements for Project Management Immediate project reviewsStatus and forecasts for quality, schedule, and budgetFollow-up action pointsReport based on consistent raw data
13Project management supporting metrics Planning - Metrics serve as a basis of cost estimating, training planning, resource planning, scheduling, and budgeting.Organizing - Size and schedule metrics influence a project's organization.Controlling - Metrics are used to status and track software development activities for compliance to plans.Improving - Metrics are used as a tool for process improvement and to identify where improvement efforts should be concentrated and measure the effects of process improvement efforts.
14Measurements for Engineers Immediate access to team planning and progressGet visibility into own performance and how it can be improvedIndicators that show weak spots in deliverablesFocus energy on software development
15The E4-Measurement Process Objectives, needsDecisions, re-direction, updated plansBusiness ProcessEnvironment, resources1. Establish2. Extract3. Evaluate4. ExecuteRef: Ebert and Dumke 2007
16Aggregation of information EnterpriseDivisionProduct Line/ DepartmentProjectsCash flow, Shareholder value, Operations costCost reduction, Sakes, margins, Customer ServiceSales, cost reduction, innovative products, level of customizationCycle time, quality, cost, productivity, customer satisfaction, resources, skills
17SMART goals Specific - precise Measurable - tangible Accountable – in line with individual responsibilitiesRealistic - achievableTimely – suitable for the current needs
18What do you want to measure? ProcessesSoftware-related activitiesProductsArtifacts, deliverables, documentsResourcesThe items which are inputs to the process
28Requirements Churn/ Requirements Creep/ Requirements Volatility number of changes to system requirements in each phase/week/increment
29Code Churn Software change history Large / recent changes Total added, modified and deleted LOCNumber of times that a binary was editedNumber of consecutive edits
30Code Complexity Gathered from code itself Multiple complexity values Cyclomatic complexityFan-In / Fan-Out of functionsLines of CodeWeighted methods per classDepth of InheritanceCoupling between objectsNumber of subclassesTotal global variables
31Code coverage Degree to which the source code is tested Statement coverageHas each node in the program been executed?Branch coverageHas each control structure been evaluated both to true and false?
33Package Level Converage JUnit Code Coverage Tool instruments byte code with extra code to measure which statements are and are not reached.Line Level CoverageA Code Coverage ReportPackage Level Converage
34Defect reporting metric Can be categorized byStatusRemaining / Resolved / FoundDefect SourcesRequirements / Design / DevelopmentDefect foundPeer review / unit testing / sanity checkTimeDefect arrival rate / Defect age
49Measurements for progress vs predictions Project PhaseFor MeasurementsFor PredictionsProject ManagementEffort and Budget TrackingRequirements StatusTask StatusTop 10 risksCost to completeSchedule evolutionQuality ManagementCode StabilityOpen defectsReview status and follow upResidual defectsReliabilityCustomer satisfactionRequirements ManagementAnalysis statusSpecification progressRequirements volatility / completenessConstructionStatus of documentsChange requestsReview statusDesign progress of reqmTime to completeTestTest progress (defects, coverage, efficiency, stability- Residual defects- reliabilityTransition, deploymentField performance (failure, corrections)maintenance effortMaintenance effortRef: Ebert and Dumke, 2007
50Recommended booksPractical Software Measurement: Objective Information for Decision Makers by John McGarry, David Card, Cheryl Jones and Beth Layman (Oct 27, 2001)Software Measurement: Establish - Extract - Evaluate – Execute by Christof Ebert, Reiner Dumke (2010)
51ReferencesFenton NE, Software Metrics: A Rigorous Approach, Chapman and Hall, 1991.Christof Ebert, Reiner Dumke, Software measurement: establish, extract, evaluate, execute, Springer 2007Richard W. Selby, Northrop Grumman Space Technology, ICSP '09 Title: "Synthesis, Analysis, and Modeling of Large-Scale Mission-Critical Embedded Software Systems“Measuring Agility, Peter Behrens[Nikora 91] Nikora, Allen P. Error Discovery Rate by Severity Category and Time to Repair Software Failures for Three JPL Flight Projects. Software Product Assurance Section, Jet Propulsion Laboratory, 4800 Oak Grove Drive, Pasadena, CA , November 5, 1991.[Nikora 91] Nikora, Allen P. Error Discovery Rate by Severity Category and Time to Repair Software Failures for Three JPL Flight Projects. Software Product Assurance Section, Jet Propulsion Laboratory, 4800 Oak Grove Drive, Pasadena, CA , November 5, 1991.