MEASUREMENT: SCALE DEVELOPMENT Lu Ann Aday, Ph.D. The University of Texas School of Public Health

Summarizing Study Variables Typically, several questions are asked in order to fully measure a concept (e.g., health, knowledge) Result in a more valid and reliable data

Summarizing Study Variables Dependent on level of measurement (nominal, ordinal, interval) Typology Index Scales

TYPOLOGY Definition: Cross-classification of answers to different questions to produce a new variable Level of Measurement: Nominal Computation: Dichotomize relevant study variables and then combine categories into a new variable

TYPOLOGY: Example ORIGINAL VARIABLES Environmental receptivity (ER) Positive Negative Organizational capacity (OC) Strong Weak NEW VARIABLE Program Potential (PP) Great potential ER=positive; OC=strong Could be a winner ER=negative; OC=strong Needs to grow into it ER= positive; OC=weak All odds are against it ER=negative; OC=weak

INDEX Definition: Accumulation of scores assigned to answers to questions Level of Measurement: Ordinal/Interval Computation: Assign scores to answers and add up scores

INDEX: Example ORIGINAL VARIABLES HIV/AIDS Risk Knowledge Series of true/false statements about factors that increase risk of HIV/AIDS NEW VARIABLE HIV/AIDS Risk Knowledge Index Assign score of 1 for correct answer to statement Assign score of 0 for incorrect answer to statement Add up scores for all statements

SCALE DEVELOPMENT Determine clearly what it is you want to measure Generate an item pool Determine the format for measurement Have item pool reviewed by experts and consider inclusion of validation items (content validity) Administer items to development sample Evaluate the items (item analysis) Optimize scale length (internal consistency reliability analysis)

LIKERT SCALE Definition: summarizes answers to questions presumed to tap the same underlying, unobservable construct Level of Measurement: Ordinal Computation: Assign scores to answers and add up scores (summative scale)

LIKERT SCALE: Examples The Medical Outcomes Study 36-Item Short- Form Health Survey (MOS SF-36) Physical Mental Cultural Beliefs related to Hypertension

Item Response Theory IRT methods differentiate error more finely IRT focuses primarily on individual items and their characteristics Central concept= item characteristics curve (ICC) Items are designed to tap different degrees or levels of the attribute

Item Response Theory Models: One parameter model/Rasch model -difficulty Two parameter model -difficulty, discrimination Three parameter model -difficulty, discrimination, guessing

SURVEY ERRORS: Defining and Clarifying the Survey Variables Systematic Errors: low or poor validity Variable Errors: low or poor reliability Solutions to errors Monitor and evaluate systematic departures in the content of a survey question from the meaning of the concept itself (content validity), the accuracy of answers based on comparisons with another data source (criterion validity), and/or the strength of hypothesized relationships of the concept being measured with other measures or concepts (construct validity). Monitor and evaluate random variation in answers to a survey question due to when it is asked (test-retest reliability), who asked it (inter-rater reliability), and/or as one of a number of questions asked to construct a summary scale (internal consistency reliability).

REFERENCES  DeVellis, Robert F. (2003). Scale Development: Theory and Applications. Second Edition. Thousand Oaks, CA: Sage.