Measurements, Meaningfulness and Scale Types Simon French

A few thoughts to begin with Kate (aged 6): “You cannot have a ‘5’, can you, Dad? You have to have 5 things.” Would you average the numbers on a few car number plates? REF provides a perfect way of measuring my research ability I am overweight …

Numbers have mystique … … especially to scientists! –“…when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind.” Lord Kelvin Ferguson Report of the BAAS 1932: –“…any law purporting to express a quantitative relation between sensation intensity and stimulus intensity is not merely false but is in fact meaningless unless and until a meaning can be given to the concept of addition as applied to sensation.” Essentially if you cannot put a number to it which you can add, subtract, kick around with arithmetic, it isn’t science

Statistics and Numbers Well, they are the same thing, aren’t they? Without numbers, where would statistics be? So lets unpack the meaning of numbers in ‘measurement’ and statistics

Representation by Numbers The ‘World’ Objects in the World Possible relations between objects: Just recognisable as separate objects To the right of Taste nicer Hotter then Rounder than Heavier than ….. Measurement : Attaching numbers to represent the objects in such a way that numerical relations between the numbers mimic the relations between the objects But once we have numbers we can do lots of arithmetical things with them. Is it meaningful to do so?

UGA of Transarcadia UniversityABCDE Rosenblock5%65%5% 20% Altenburg30%5%25%35%5% Award teaching funds according to mean matriculation grade Change grades to numbers: A → 5, B→4, C→3, D→2, E→1. Rosenblock mean grade: 3.3. Altenburg mean grade: 3.2 Instead associate grades with minimum mark for grade: A → 90, B→75, C→65, D→50, E→30 Rosenblock mean grade: 65. Altenburg mean grade: 66 The ranking of means of ranking measurements is meaningless It depends on the ‘spurious’ choice of numbers to represent the ranking

Reliability, Validity and Meaningfulness In statistical analysis need to consider Validity –Are we measuring what we think we are Reliability –If we measure it again and again, do we get the same measurement Meaningfulness –When can we use the numbers arithmetically in ways that reflect the world out there?

Scale types Scale TypeExampleMeaningfulUnique up to … Nominal Bank a/c nos. Anonymised items … No arithmetical operations are meaningful Any 1-1 transformation Ordinal Perceived loudness Simple preference … greater/less than but little else. Certainly not means Any monotonic transformation Interval Preference with trade-offs … Most operations but nothing that relies on the origin. Ordering of means OK Any change of scale and origin Ratio Money Weight …. Any operations that do not depend on scale. Ordering of means OK Any change of scale; origin is fixed Absolute Probability ….. Most operations that preserve scale. Ordering of means OK unique

Issues for statistics Many statistical methods require data measured on interval or ratio scales –in theory at least –pragmatically they may work on ordinal data … if you do not believe p values, significance levels, etc. There are methods for ordinal data –e.g. many nonparametric methods.

Naude et al (2010) MBA rankings Naude, P, Henneberg, S and Jiang, Z (2010) ‘Varying roots to the top: identifying different strategies in the MBA marketplace’ Journal of the Operational Research Society 61,

Context MBAs are judged according to –Various criteria Top criteria: ‘Career progress’, ‘diversity’ and ‘idea generation’ –Accreditation Data –FT rankings of top 100 MBAs, each assessed against a number of criteria, some ordinal, some interval. –So it would be dubious to use any analysis that assumes that all criteria are measured on interval or ratio scales

Initial analysis Table 1, p1195 –Simple grouping to look at high level behaviour –Uses means so ‘assumes’ everything on interval or ratio scales –Check things make some sort of rough sense Check with ANOVA roughly …. –Again assumes interval or ratio scales Then with discriminant analysis –Plot in three dimensions and find planes that separate the MBAs into clumps –Again assume interval or ratio scales But only getting a ‘feel’ for the data –Seeing what makes ‘sense;

Cluster Analysis (Table 2, p 1196) Takes a measure of distance between two MBAs (e.g. Euclidean) Find clusters of MBAs that are close together But checked with other distance measures –Little/no effect on clusters

Multidimensional scaling (Figure 1, p1198) Each MBA represented by 8 sub-criteria of career progress. Try and plot in two dimensions so that 2-d distance between MBA reflects distance between them in full 8- dimensions Again the plots do not vary much if different distance measures used

Factor Analysis (p1198) Diversity is measured by 9 different sub- criteria Can we reduce the dimensions Again factor analysis is geometric (rotating axes to find the one that has the most variation) so assumes interval scales But develops two components (Table 5)

Cluster Analysis (p1200) This time on ‘idea generation’

Naude et al’s Analysis Exploratory Multiple runs with different ‘spurious’ choices –Sensitivity analysis Use p-values and significance levels as indications only Looked for common themes and qualitative understanding Their analyses ‘triangulate’

Theory of Measurement Stevens: Scale types Krantz, Luce, Suppes and Tversky: Theory of Measurement Vols 1, 2 and 3. –Very mathematical F.S. Roberts (1979) Measurement Theory. Vol 7 in the Encyclopaedia of Mathematics Cambridge University Press.