Software Measurement Theoretical issues

Measurement Theory – Sources Measurement theory in mathematics Metrology (physical science & Engineering) – Definitions of : units – quantities – scales – etalon - …. Representational Theory – 1st comprehensive framework – S.S. Stevens (1946) – Social Science (Psychology) – In Wikipedia ".. measurement is the process of obtaining the magnitude of a quantity, such as length or mass, relative to a unit of measurement, such as a meter or a kilogram. "The term can also be used to refer to the result obtained after performing the process"

Measurement Theory Representational Theory – Measurement = Characterizing a given attribute of a given entity by a numerical value (symbolic) -Measurement = mapping Albert Gabriel Paula Attribute : tallness

Measurement Theory The intelligence barrier (cfr. Kritz) reasoning meaningful relations – operations ….

Measurement Theory Representational Theory – Measurement = Characterizing a given attribute of a given entity by a numerical value (symbolic) -Measurement = mapping In Software : no consensus about how to characterize (define) software attributes Example : the term "complexity" – Size – Coupling – Connectivity (fan-in / fan-out) – Understandability – Lack of Cohesion – ….

Measurement Theory Representational Theory – Measurement = Characterizing a given attribute of a given entity by a numerical value (symbolic) -Measurement = mapping In Software : ambiguity & misuse about the concerned software entity (the product) Ex. "maintainability" is an attribute of – code – design – code + design – …. Ex. "size" is an attribute of – code – requirement –

Measurement Theory Representational Theory – Measurement = Characterizing a given attribute of a given entity by a numerical value (symbolic) -Measurement = mapping How to measure = F "a precise characterization of what we are measuring" – Ex To measure the mass of an sphere, measure the time it takes the object to fall from some height? its diameter? its external area? its volume? a calculation on basis of those – Ex To measure the reliability of a light bulb, take its color ? the length of the wire? the number of watts? the number of times it is switched on? (during ??? ) the average lifetime? (of how many bulbs ??? how to choose them)

Measurement Theory measurement should capture one’s own intuitive understanding on an attribute of a set of entities Albert Gabriel Paula Mapping property 1: Order preservation Albert "taller" Paula   (Albert) >  (Paula)

Measurement Theory measurement should capture one’s own intuitive understanding on an attribute of a set of entities Albert Gabriel Paula Mapping property 2 : Operations preservation : if Gabriel "on the shoulder of" Albert  Albert   Gabriel then  (Gabriel "on the shoulder of" Albert ) =  (Albert)   (Gabriel) ex (x)  (y) is defined as 0,87 (  (x) +  (y)) ?

Measurement Theory measurement should capture one’s own intuitive understanding on an attribute of a set of entities Question : which are our understanding ? – Which order ? Which operations ? Ex Sound characterization – Timbre: same timbre / different timbers (identity) – Loudness: a sound louder than another sound (order) – Pitch: same pitch / different pitches (identity) a sound higher than another sound (order) a “distance” from a reference note (distance) ??

Measurement Theory Mathematically A measure involves – Empirical Relational System = a triple E : a set of entities R1, R2, …, Rn: a collection of relations among entities o1, o2, …, om: a collection of binary operations between entities R1, R2, …, Rn and o1, o2, …, om determine meaningful (consensual knowledge on the domain) !!! (no numbers) – Numerical Relational System = a triple V : a set of values S1, S2, …, Sn: a collection of relations among values, – such as : >, >= op1, op2, …, opm: a collection of binary operations between values – such as : + !!! no real entities

Measurement Theory Mathematically Example 1: – program size  number of LOC – concatenation  + Empirical Relational System = a triple – E : a set of program pieces p1, p2,…. – R1 : px "is longer than" py (observable on all pairs) – O1 : program concatenation (cfr. technically…) Numerical Relational System = a triple – V : non negative real numbers – S1 : > – op1 : +

Measurement Theory Mathematically Example 1: – program size  number of LOC – concatenation  + What about representational condition ?? – The relations (the order) For all px, py – px "is longer than" py  size(px) > size(py) – The operation For all px, py – size (px "concatenation" py) = size(px) + size(py)

Measurement Theory Mathematically Example : – program complexity  graph complexity (McCabe) – Composition (via call)  + – Empirical Relational System = a triple E : a set of procedures pr1, pr2,…. R1 : prx "is more complex than" pry O1 : procedure composition via "call" – Numerical Relational System = a triple V : non negative real numbers S1 : > op1 : +

Measurement Theory Mathematically Example : – program complexity  graph complexity (McCabe) – Composition (via call)  + What about representational condition ?? – The relations (the order) For all px, py – prx "is more complex than" py  complexity(x) > complexity (py) – The operation For all prx, pry – complexity (prx "call" pry) = complexity(prx) + complexity(pry) ?

Measurement Theory Scale & Validity A scale = – Empirical Relational System – Numerical Relational System – Measure (mapping) satisfying the "Representation Condition" A scale is not (necessary) unique – distance : KM, M, MM, … – Temperature : degrees celsius, kelvin, fahrenheit … – Electric Resistance : ohm – Electric Current : ampere – Pressure : pascal, bar, A family of scales – Scales + admissible transformations – Admissible transformations = preserving "knowledge", (i.e., statements like "x > y", "x op y = z", …) (i.e., "meaningful statements")

Measurement Theory Scale & Validity meaningful statements in real world ? – Dupont is taller than Tintin – Yesterday’s temperature was 5 times as high as today’s ? – Program P1 is three times as long as program P2 ? – Program P1 is three times as complex as program P2 ? – Failure F1 is more critical than failure F2 ? – Failure F1 is five times as critical as failure F2 ? – The average "note" of my students is ? – The average "grade" of my students is "distinction" – The UML design D1 (with the pattern P) is better then the design D2 – The UML modeling language is more expressive than the ERA – Question : Which "knowledge" is consensually admitted ? weaker view /researcher view – which "knowledge" is potentially a good candidate for a consensus – How to reach such a consensus

Measurement Theory Scale & Validity 5 classical "scale types" – nominal least informative one / largest set of admissible transformations no order classification : ex. (yes/no) (blue/green/red)...) – ordinal order only (no +/-) (no distance) ex. Students grade : A, B, C, D, E Statistics : no mean (debate in social science)  median – interval order with +/- ; with distance (no ratio) ex. Celsius temperature Statistics : mean correlation etc – ratio order with +/- with ratio with zero, … ex. Distance in Km/m/cm/…. Statistics : (geometric) mean correlation etc – absolute most informative one / smallest set of admissible transformations Absolute : Ratio + uniqueness Ex.

Measurement Theory Scales : nominal scale Examples – Types of (vehicles /energy sources / …) gender / birthplaces … – SE: pgming languages / methodology /… Mainly a classification (rather than an order) Meaningful assertions in the empirical world – Entity e belong / do not belong to class C Meaningful operations in the numerical world – Mainly a classification (no order) – Any collections of symbols (x,y,z) (yes / no/ no answer) (C / Java / Fortran)…. (0,1,2)…. But without operations nor order Admissible Transformations from scale to scale – one-to-one

Measurement Theory Scales : nominal scale empirical world: – P : set of programs p1, p2,… (or modules) – order :  – operation "same lgge as"" numerical world: – X: {java, C,C#,Fortran} – order :  – operation : = Example of operations – p1 "same lgge than" p23 – not (p1 "same lgge than" p12) Example of statistics – C (30%), FORTRAN(10%), Java (35%), C# (25%) – Frequencies ok ;here p(C) =.3 ; p(FORTRAN) =.1 ; p(Java) =.35 ; p(C#) =.25 – Mode (the most likely value(s) ) ok ; here Java

Measurement Theory Scales : ordinal scale Examples – Ranking in general : Michelin guide stars, wine, students … – SE: maturity (CMMi), criticality, security levels (some norms) ; (some def) complexity ; A simple order Meaningful assertions in the empirical world – Entity e1 is "as …. as" e2 – Entity e2 is "more …. than" e5 Meaningful operations in the numerical world – Order : – Any collections of symbols with defined order (a,b,c,d,e) (0,1,2)…. With order but with no distance Admissible Transformations from scale to scale – Admissible transformations : any monotonically increasing t: V→V (m(e1) > m(e2) ⇔ t(m(e1)) > t(m(e2)) ) – two entities are at the same level before the transformation if and only if they are at the same level after the transformation – one entity is at a higher level before the transformation if and only if it is at a higher level after the transformation – Km to Cm / Celsius to Fahernheit /

Measurement Theory Scales : ordinal scale empirical world: – P : set of programs p1,p2,… (or modules) – order : more maintainable – operation  numerical world: – X: {easy, medium, difficult, too difficult} – order : > – operation :  Example of operations – p1 "as maintainable as" p23 – not (p1 "more maintainable than" p12) Example of statistics – median ok – average : problems !!

Measurement Theory Scales : interval scale Examples – calendar – Celsius temperature – SE: milestones A simple order + relative distance between values Meaningful assertions in the empirical world – Entity e1 is "as …. as" e2 – Entity e2 is "more …. than" e5 – From e1 to e2 is equal/greater/twice….. than from e2 to e3 Meaningful operations in the numerical world – Order & distance – Ratio on distances – A collections of symbols with order and distance integer – real (0,1,2)…. With order but and with distance (no zero) Admissible Transformations from scale to scale – Admissible transformations : linear transformations ( m ’ = am + b ) – (m(e1) – m(e2)) / (m(e3) – m(e4)) = (m ’ (e1) – m ’ (e2)) / (m ’ (e3) – m ’ (e4)) = (am(e1) + b – am(e2) - b) / (am(e3) + b – am(e4) - b) – !!!! a subset of the admissible transformations of ordinal scale

Measurement Theory Scales : ratio scale Examples – length – weight – SE: size – complexity (some definition) A simple order + distance from a zero Meaningful assertions in the empirical world – … + – distance from entity e1 to origin is equal/greater/twice….. than distance from e2 to origin Meaningful operations in the numerical world – Order & distance – Ratio on distances from zero – Numbers (Reals…) Meaningful statistics: – all those for interval scales, plus (descriptive) geometric mean Admissible Transformations from scale to scale – m ’ = am (with a > 0) – we can still change the unit of measurement as we like – we cannot change the zero

Measurement Theory Scales : absolute scale Examples – probabilities and any direct counting – SE: size in LOC – number of faults - Meaningful assertions in the empirical world – // ratio scales Meaningful operations in the numerical world – Order & distance & zero – Ratio on distances from zero – Numbers Meaningful statistics: – // ratio scales Admissible Transformations from scale to scale – m ’ = m (no possible transformation) –.