MEASUREMENT

Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant

Measurement Selecting measurable phenomena Developing a set of mapping rules Applying the mapping rule to each phenomenon

What is to be measured? CONCEPTS/CONSTRUCTS/VARIABLES Operational Definitions Operational Definitions - Dimensions - Elements Scales Scales

CONCEPT A GENERALIZED IDEA ABOUT A CLASS OF OBJECTS, ATTRIBUTES, OCCURRENCES, OR PROCESSES. (e.g., Technology, Dynamism, Adoption, Learning)

CONSTRUCT An image or idea specifically invented for a given research and/or theory-building purpose An image or idea specifically invented for a given research and/or theory-building purpose Higher level concepts for specialised scientific explanatory purposes that are not directly observable and for thinking about and communicating abstractions Higher level concepts for specialised scientific explanatory purposes that are not directly observable and for thinking about and communicating abstractions Concepts and constructs are used at theoretical levels

VARIABLE Used at empirical level Used at empirical level Accept numeral or values for the purpose of testing and measurement Accept numeral or values for the purpose of testing and measurement

OPERATIONAL DEFINITION A definition for a construct stated in terms of specific criteria for testing or measurement A definition for a construct stated in terms of specific criteria for testing or measurement Specifies what the researcher must do to measure the concept/construct under investigation Specifies what the researcher must do to measure the concept/construct under investigation

SCALE - Definition SERIES OF ITEMS SERIES OF ITEMS ARRANGED ACCORDING TO VALUE ARRANGED ACCORDING TO VALUE FOR THE PURPOSE OF QUANTIFICATION FOR THE PURPOSE OF QUANTIFICATION A CONTINUOUS SPECTRUM A CONTINUOUS SPECTRUM

SCALE PROPERTIES UNIQUELY CLASSIFIES UNIQUELY CLASSIFIES PRESERVES ORDER PRESERVES ORDER DISTANCE/EQUAL INTERVALS DISTANCE/EQUAL INTERVALS NATURAL ORIGIN (ZERO) NATURAL ORIGIN (ZERO)

Types of Scales Ordinal Interval Ratio Nominal

NOMINAL SCALE

Nominal Scales Mutually exclusive and collectively exhaustive categories Mutually exclusive and collectively exhaustive categories Exhibits the classification characteristic only Exhibits the classification characteristic only

NOMINAL SCALE PROPERTIES UNIQUELY CLASSIFIES Male/Female Male/Female Academic/Admin Academic/Admin Asian/European Asian/European Strategy types Strategy types Adopters/Non Adopters Adopters/Non Adopters

Levels of Measurement Ordinal Interval Ratio Nominal Classification

ORDINAL SCALE

Ordinal Scales Characteristics of nominal scale plus an indication of order Implies statement of greater than and less than

ORDINAL SCALE PROPERTIES UNIQUELY CLASSIFIES PRESERVES ORDER Win, Place & Show Win, Place & Show Podium/Grid positions Podium/Grid positions Ranking Ranking

Levels of Measurement Ordinal Interval Ratio Nominal Classification Order Classification

INTERVAL SCALE

Interval Scales Characteristics of nominal and ordinal scales plus the concept of equality of interval. Characteristics of nominal and ordinal scales plus the concept of equality of interval. Equal distance exists between numbers Equal distance exists between numbers

INTERVAL SCALE PROPERTIES UNIQUELY CLASSIFIES PRESERVES ORDER EQUAL INTERVALS Consumer Price Index (Base 100) Consumer Price Index (Base 100) Fahrenheit Temperature Fahrenheit Temperature Assessment of attitude, beliefs, intention Assessment of attitude, beliefs, intention

Levels of Measurement Ordinal interval Ratio Nominal Classification Order Classification Order Classification Distance

RATIO SCALE

Ratio Scales Characteristics of previous scales plus an absolute zero point Characteristics of previous scales plus an absolute zero point Examples Examples  Weight  Height  Number of children

RATIO SCALE PROPERTIES UNIQUELY CLASSIFIES PRESERVES ORDER EQUAL INTERVALS NATURAL ZERO Weight and distance Weight and distance Age, years of service Age, years of service

Levels of Measurement Ordinal Interval Ratio Nominal Classification Order Classification Order Classification Distance Natural Origin Order Classification Distance

Goodness of Measures Accurately measuring what actually the concept one sets out to measure Accurately measuring what actually the concept one sets out to measure Should be easy and efficient to use Should be easy and efficient to use

Evaluating Measurement Tools Criteria Validity Practicality Reliability

Validity The ability of a scale to measure what was intended to be measured.

Validity Determinants Content Construct Criterion

Increasing Content Validity Content Literature Search Expert Interviews Group Interviews Question Database

……Goodness of Measures Validity Validity  Content Validity  Includes Face Validity  How well the dimensions and elements of a concept have been delineated  Professional agreement that a scale logically appears to accurately measure what it is intended to measure

Validity Determinants Content Construct

Increasing Construct Validity New measure of trust Known measure of trust Empathy Credibility

……Goodness of Measures  Construct Validity – when empirical evidence generated by a measure is consistent with the theoretical logic of the concept  Convergent Validity – the measure should “converge” with other similar measures  Discriminant Validity – when the measure has low correlation with measures of dissimilar concept

Judging Criterion Validity Relevance Freedom from bias Reliability Availability Criterion

Criterion-Related Validity  Concurrent – When the scale discriminates individuals who are known to be different; A type of criterion validity whereby a new measure correlates with a criterion measure taken at the same time.  Predictive Validity – Ability to differentiate among individuals with reference to a future criterion

VALIDITY

Reliability The degree to which measures are free from random error and therefore yield consistent results.

Reliability Estimates Stability Internal Consistency Equivalence

Reliability Estimates Stability Internal Consistency Equivalence

 Stability of Measures  Test-Retest Reliability – the administering of the same scale or measure to the same respondents at two separate points in time  Parallel-Form Reliability - when responses on two comparable sets of measures tapping the same construct are highly correlated … Reliability

Reliability Estimates Stability Internal Consistency Equivalence

Variations at one point in time among observers and samples of items Variations at one point in time among observers and samples of items

Reliability Estimates Stability Internal Consistency Equivalence

… Reliability  Internal Consistency Measures  Interitem Consistency Reliability - where items that are independent measures of the same concept correlate with one another.  Split-Half Reliability – reflects the correlations between two halves of an instrument

RELIABILITY

Understanding Validity and Reliability

Practicality EconomyInterpretability Convenience

SENSITIVITY A measurement instrument’s ability to accurately measure variability in stimuli or responses. A measurement instrument’s ability to accurately measure variability in stimuli or responses.

Rating Scales Used to elicit responses with regard to the object, event, or person studies: Dichotomous scale Category scale Likert scale Numerical scale Semantic differential scale

….. Rating Scales Itemised rating scale Fixed or constant sum rating scale Stapel scale Graphic rating scale Consensus scale

Ranking Scales To tap preferences between two or more objects or items Paired Comparison Paired Comparison Forced Choice Forced Choice Comparative Comparative