1. University of Sunderland CIFM03Lecture 4 1 Software Measurement and Reliability CIFM03 Lecture 4

2. University of Sunderland CIFM03Lecture 4 2 Where does Software Measurement fit in? Look at software measurement from the perspective of software quality assurance….. SQA REVIEWS TESTING MEASURE- MENT STANDARDS& PROCEDURES

3. University of Sunderland CIFM03Lecture 4 3 Software Measurement is... “the objective assignment of numbers or symbols to attributes of a software process or product in such a way as to describe them according to clearly defined rules...” see ‘Software Metrics, A Rigorous & Practical Approach’, Fenton & Pfleeger

4. University of Sunderland CIFM03Lecture 4 4 Why should we measure?  what are our goals?  what information is required?  what can we collect?  How do we act upon the data?

5. University of Sunderland CIFM03Lecture 4 5 DON’T! Don’t define what you won’t collect.. Don’t collect what you won’t analyse.. Don’t analyse what you won’t use..

6. University of Sunderland CIFM03Lecture 4 6 Goal / Question / Metric Goal - achieve a 10-fold improvement in post-release defects. Question - how many existing faults? How effective are inspection techniques? How effective are tests? Metric - no. & type of faults, no. of defects found, % paths tested, complexity of code.

7. University of Sunderland CIFM03Lecture 4 7 Hard & Soft Data HARD - objective figures - no.of people on a project, duration of tasks, size of documentation, etc. SOFT - subjective - level of experience, balance of teams, etc.

8. University of Sunderland CIFM03Lecture 4 8 Obstacles to Effective Measurement  no agreement over standard measures  patchy tool support  initial set-up expense

9. University of Sunderland CIFM03Lecture 4 9 Basic Measurement Theory  Nominal - names / categories  Ordinal - ‘cold, warm, hot’ ‘small, med, large’  Interval - temperature ‘20 o C, 40 o C’ - scale  Ratio - absolute temperature ‘20 o K, 40 o K’, time  Absolute - counting no. of occurrences

10. University of Sunderland CIFM03Lecture 4 10 Analysis.. Nominal - equivalence Ordinal - as above + greater than Interval - as above + ratio of any intervals Ratio - as above + ratio of any 2 scale values Absolute - as above + geometric mean, coeff. of variation

11. University of Sunderland CIFM03Lecture 4 11 Function Points & Cyclomatic Complexity Function Points: Measure of functionality for ‘size’ - no. of external inputs - no. of external outputs - no. of enquiries - no. of files - no. external interfaces Cyclomatic Complexity: Module complexity based on a ‘graph’ of the module - nodes - predicate nodes - paths

12. University of Sunderland CIFM03Lecture 4 12 Measurement and the CMM

13. University of Sunderland CIFM03Lecture 4 13 Software Reliability is… The probability that the software will; operate as required (i.e., without fail), for a specified time, in a specified environment.

14. University of Sunderland CIFM03Lecture 4 14 Software Reliability - features failures in software are design faults, reliability during test changes continually (new problems are found as old ones are fixed / new code is never perfect) phenomenon of software reliability growth environment is important (platform/inputs) - new envt. may require s/w retest

15. University of Sunderland CIFM03Lecture 4 15 Reliability - General (i) Of all software quality characteristics, reliability is consistently identified as most important. Reliability, for both hardware and software, is a probability - SYSTEM reliability can be derived from combining the hardware and software reliabilities. Reliability is directly related to the number of failures. [ p(Reliable) =1 - p(Fail) ]

16. University of Sunderland CIFM03Lecture 4 16 Reliability - General (ii) Reliability measures are much more useful than fault measures - from the user perspective. Measures of faults/defects are developer oriented. Change of environment (operating platform, operational data profile) changes reliability.

17. University of Sunderland CIFM03Lecture 4 17 Fault, Error, Failure (defined by BS5760 part8: 1998) Fault a defect in a software component which, when triggered, causes one or many errors Error a discrepancy between its actual state and the ‘correct state’ which, if it propagates, means that the component ceases to perform as required Failure the component ceases to perform as required

18. University of Sunderland CIFM03Lecture 4 18 Fault, Error, Failure FAULTS ERRORS FAILURE ENVIRONMENT OPERATOR INPUT OR REVEALING MECHANISM AND LEADS TO ZERO OR MANY POTENTIALLY LEADS TO ZERO OR MANY MISTAKES (PERSON MAKES) CAN BE ATTRIBUTED TO ONE OR MANY (This slide from BS5760 Part 8: 1998)

19. University of Sunderland CIFM03Lecture 4 19 Reliability Measures When we talk of reliability measures the irony is that we invariably talk about failure measures. There are four general ways of measuring failures against time;  time of failure,  interval between failures,  cumulative failures experienced up to a given time,  failures experienced in a time interval.

20. University of Sunderland CIFM03Lecture 4 20 Availability & Maintainability Availability –is the probability that the software is operating at a given time. It is equal to the mean time to failure divided by the mean time between failures. Maintainability –is the mean time to repair the software and restore to working order.

21. University of Sunderland CIFM03Lecture 4 21 Uses of S/w Reliability Measurement System Reliability evaluation Monitoring operational performance –e.g., decisions on when to go for enhancement Evaluation of test status / test schedule Management of new features and design changes