1. R&D SDM 1 Metrics How to measure and assess software engineering? 2009 Theo Schouten

2. Contents What are “software metrics” Why is measurement important? Software Quality Qualitative measures Quantitative measures Final remarks Book chapter 15

3. Metrics What is a metric? – “A quantitative measure of the degree to which a system, component or process pocesses a given attribute” (IEEE Software Engineering Standards 1993) : Software Quality Different from – Measure (size of a system, component e.d) – Measurement (act of determining a measure) Metrics – Qualitative Metrics – Quantitative Metrics

4. Why important, difficult Why is measurement difficult? –No “exact” measure (‘measure the unmeasurable’, subjective factors) –Dependent on technical environment –Dependent on organizational environment –Dependent on application and ‘fitness for use’ Why is measurement important to characterize to evaluate to predict to improve

5. Quality definition Requirements are the foundation to measure software quality Standards define the development criteria for software engineering Software quality should conform to explicit and implicit requirements (degree of) conformance to: explicitly stated functional and performance requirements explicitly documented developments standards implicit characteristics that are expected of all professional developed software.

6. Software quality attributes Product Revision Product Transition Maintainability Flexibility Testability Portability Reusability Interoperability Product Operation Software Quality Factors (McCall, 1977): Correctness Usability Efficiency Reliability Integrity

7. Software Quality: Qualitative Measures McCall: Metrics that affect cq influence software quality factors: – Software Quality Factors are the dependent, metrics are the independent variable – Metrics: audibility, accuracy, communication commonality, completeness, consistency, data commonality, error tolerance, execution efficiency, expandability, generality, hardware independence, instrumentation, modularity, operability, security, self- documentation, simplicity, software system independence, traceability, training. – Software quality factor = c 1 m 1 + c 2 m 2 + … + c n m n – C n is a regression coefficient based on empirical data – Software quality factor gives an indication of the quality of the software

8. McCall Matrix

9. FURPS, Quality Factors Developed at Hewlett-Packard (Grady, Caswell, 1987) Functionality: – Feature set and capability of the system – Generality of the functions - Security of the overall system Usability: – Human factors (aesthetics, consistency and documentation) Reliability: – Frequency and severity of failure – Accuracy of output – MTTF - Failure recovery and predictability Performance: – Speed, response time, resource consumption, throughput and efficiency Supportability: – Extensibility, Maintainability, Configurability, Etc.

10. ISO 9126 Quality Factors 6 key quality attributes, each with several sub-attributes Functionality Reliability Usability Efficiency Maintainability Portability Also often not direct measurable, but gives ideas for indirect measures and checklists. defect : The nonfulfilment of intended usage requirements nonconformity : The nonfulfilment of specified requirements superseded by the new project SQuaRE, ISO 25000:200

11. Quantitative Metrics Desired attributes of Metrics (Ejiogu, 1991) – Simple and computable – Empirical and intuitively persuasive – Consistent and objective – Consistent in the use of units and dimensions – Independent of programming language, so directed at models (analysis, design, test, etc.) or structure of program – Effective mechanism for quality feedback Type of Metrics: – Size oriented Focused on the size of the software (LinesOfCode, Errors, Defects, size of documentation, etc.) independent of programming language? – Function oriented Focused on the realization of a function of a system

12. Product Metrics Landscape See chapter 15.2.6 Metrics for the analysis model Metrics for the design model Metrics for the source code Metrics for testing

13. Function Oriented Metrics Function Point Analysis (FPA) a method for the measurement of the final functions of an information system from the perspective of an end user – method to determine size of a system on basis of a functional specification – method is independent of programming language and operational environment – its empirical parameters are not. – Usable for estimate cost or effort to design, code and test the software predict number of components or Lines of Code predict the number of errors encountered during testing determining a ‘productivity measure’ after delivery

14. Function Point Analysis Basis is the Function Punt Index (FPI) of a system to build:. Amount of work = Function Point Index * Resource factor Resource factor depends on: – development environment – experience of project team with environment and tools – size of the team Benchmarks and measurements of previous projects are used to estimate the resource factor.

15. Function Point Index How do you calculate the FPI of a system to be build? Three steps are used: Determine Function Point Index Determine System Attributes Determine ‘Value Adjustment”

16. FPA: System Attributes Count the number of each ‘system attribute’ : 1.User (Human or other system) External Inputs (EI) 2.User External Outputs (EO) 3.User External Inquiries (EQ) 4.Internal Logical Master Files (MF) 5.Interfaces to other systems (IF) System Environment Other Systems Transactions External User Transactions Interface MFs EI EO EQ IF

17. FPA: System Atributes weighting Determine per system attribute how difficult it is: – Low – Medium – High Use the following matrix to determine the weighting factor: Calculate the weighted sum of the system attributes: the ‘Unadjusted Function Point Index’ (UFPI) System AttributeLowMediumHigh User External Input346 User External Output457 User External Inquiry346 User Logical Master File71015 Interfaces5710

18. FPA: Value Adjustment The UFPI needs to be adapted to the environment in which the system has to operate. The ‘degree of influence’ is determined for the 14 ‘values adjustment ‘ factors:. – Data Communications – Distributed Processing – Performance Objectives – Tight Configuration – Transaction Volume – On-line Data Entry – End User Efficiency – Logical File Updates – Complex Processing – Design for Re-usability – Conversion and Installation Ease – Multiple Site Implementation – Ease of Change and Use Value between 0 and 5

19. FPA: final function point index 1.Total sum of the ‘degree of influence’ (DI) (0-70 2.Determine Value Adjustment : VA= 0.65+ (0.01*DI) (0.65-1.35) 3.Function Point Index = VA*UF Historical data can then be used, e.g. 1 FP -> 60 lines of code in object oriented language 12 FP's are produced in 1 person-month 3 errors per FP during analysis, etc.

20. Example Alarm Gebruiker’s Interactie Module Functie: Gebruiker Password Sensor opvragen Panic button Zone opvragen Aan/Uit Systeem Configuratie Data Sensors Gebruiker Externe Monitoring Test Sensor Zone instellen Berichten Sensor Status Aan/Uit Alarm AlertPassword, sensors etc.

21. Unadjusted Function Point Index Determine the number of system attributes: 1.User External Inputs (EI) 2.User External Outputs (EO) 3.User External Inquiries (IQ) 4.User Logical Master Files (MF) 5.Interfaces to other systems (IF) System AttributeNumberLowMediumHighTotal User External Input 33469 User External Output 24578 User External Inquiry 23466 User Logical Master File 1710157 Interfaces 4571020 Total 50

22. End remarks Metrics are usable for a relative view on a system, not an absolute view. Metrics have qualitative and quantitative aspects. Realize you try to measure the unmeasurable Use metrics to determine the functionality of a system on basis of the wishes of the end user Use metrics for: – To characterize – To evaluate – To predict – To improve