Presentation is loading. Please wait.

Presentation is loading. Please wait.

Measurement and Scaling

Similar presentations

Presentation on theme: "Measurement and Scaling"— Presentation transcript:

1 Measurement and Scaling
Farzin Madjidi, Ed.D. Pepperdine University Graduate School of Education and Psychology

2 Variables Independent Dependent Extraneous
Precedes, influences or predicts results Dependent Affected by or predicted by the Independent Variable Extraneous Affected by the D.V., but not controlled or measured. Causes error

3 Variables Confounding Intervening
An extraneous variable that varies systematically (has a relationship) with the I.V. Intervening Unobservable trait that influences behavior (e.g., effect of new intervention on self-esteem may be affected by the motivation level of subjects)

4 Variables Control Organismic
Used to eliminate the effect of extraneous variables Organismic Aka, measured, or assigned Characteristics of the subjects that cannot be manipulated

5 Levels of Measurements
Four levels of Measurements Nominal Measures categories Ordinal Categories + rank and order Interval Equal distance between any two consecutive measures Ratio Intervals + meaningful zeros

6 Categories of Scales Categorical (ratings) Comparative (ranking)
Score without comparison - 1 to 5 scales Comparative (ranking) Score by comparing - Smartest Preference Subjective - which do you prefer Non-preference Objective - which solution is less costly

7 Categories of Scales Unidimensional Multi-dimensional
Involves only one aspect of the measurement Measurement by one construct Multi-dimensional Involves several aspects of a measurement Uses several dimensions to measure a single construct

8 Types of Scales Likert/Summated Rating Scales
Semantic Differential Scales Magnitude Scaling Thruston Scales Guttman Scales

9 Likert Scales A very popular rating scale
Measures the feelings/degree of agreement of the respondents Ideally, 4 to 7 points Examples of 5-point surveys Agreement SD D ND/NA A SA Satisfaction SD D ND/NS S SS Quality VP P Average G VG

10 Summative Ratings A number of items collectively measure one construct (Job Satisfaction) A number of items collectively measure a dimension of a construct and a collection of dimensions will measure the construct (Self-esteem)

11 Summative Likert Scales
Must contain multiple items Each individual item must measure something that has an underlying, quantitative measurement continuum There can be no right/wrong answers as opposed to multiple-choice questions Items must be statements to which the respondent assigns a rating Cannot be used to measure knowledge or ability, but familiarity

12 Semantic Differential Scales
Uses a set of scale anchored by their extreme responses using words of opposite meaning. Example: Dark ___ ___ ___ ___ ___ Light Short ___ ___ ___ ___ ___ Tall Evil ___ ___ ___ ___ ___ Good Four to seven categories are ideal

13 Magnitude Scaling Attempts to measure constructs along a numerical, ratio level scale Respondent is given an item with a pre-assigned numerical value attached to it to establish a “norm” The respondent is asked to rate other items with numerical values as a proportion of the “norm” Very powerful if reliability is established

14 Thurston Scales Thurston Scales Items are formed
Panel of experts assigns values from 1 to 11 to each item Mean or median scores are calculated for each item Select statements evenly spread across the scale

15 Thurston Scales Example:
Please check the item that best describes your level of willingness to try new tasks I seldom feel willing to take on new tasks (1.7) I will occasionally try new tasks (3.6) I look forward to new tasks (6.9) I am excited to try new tasks (9.8)

16 Guttman Scales Also known as Scalograms
Both the respondents and items are ranked Cutting points are determined (Goodenough-Edwards technique) Coefficient of Reproducibility (CReg) - a measure of goodness of fit between the observed and predicted ideal response patterns Keep items with CReg of 0.90 or higher

17 Scale Construction Define Constructs
Conceptual/theoretical basis from the literature Are their sub-scales (dimensions) to the scale Multiple item sub-scales Principle of Parsimony Simplest explanation among a number of equally valid explanations must be used

18 Item Construction Agreement items Write declarative statements
Death penalty should be abolished I like to listen to classical music Frequency items (how often) I like to read Evaluation items How well did your team play How well does the police serve your community

19 Item Writing Mutually exclusive and collectively exhaustive items
Use positively and negatively phrased questions Avoid colloquialism, expressions and jargon Avoid the use of negatives to reverse the wording of an item Don’t use: I am not satisfied with my job Use: I hate my job! Be brief, focused, and clear Use simple, unbiased questions

20 Sources of Error Social desirability Response sets Acquiescence
Giving politically correct answers Response sets All yes, or all no responses Acquiescence Telling you what you want to hear Personal bias Wants to send a message

21 Sources of Error Response order Item order
Recency - Respondent stops reading once s/he gets to the response s/he likes Primacy - Remember better the initial choices Fatigue Item order Answers to later items may be affected by earlier items (simple, factual items first) Respondent may not know how to answer earlier questions

22 Assessing Instruments
Three issues to consider Validity: Does the instrument measure what its supposed to measure Reliability: Does it consistently repeat the same measurement Practicality: Is this a practical instrument

23 Types of Validity Face validity Content validity
Does the instrument, on its face, appear to measure what it is supposed to measure Content validity Degree to which the content of the items adequately represent the universe of all relevant items under study Generally arrived at through a panel of experts

24 Types of Validity Criterion related
Degree to which the predictor is adequate in capturing the relevant aspects of criterion Uses Correlation analysis Concurrent validity Criterion data is available at the same time as predictor score- requires high correlation between the two Predictive validity Criterion is measured after the passage of time Retrospective look at the validity of the measurement Known-groups

25 Types of Validity Construct Validity
Measures what accounts for the variance Attempts to identify the underlying constructs Techniques used: Correlation of proposed test with other existing tests Factor analysis Multi-trait-multimethod analysis Convergent validity - Calls for high correlation between the different measures of the same construct Discriminant validity - Calls for low correlation between sub-scales within a construct

26 Types of Reliability Stability
Test-retest: Same test is administered twice to the same subjects over a short interval (3 weeks to 6 months) Look for high correlation between the test and retest Situational factors must be minimized

27 Types of Reliability Equivalence
Degree to which alternative forms of the same measure produce same or similar results Give parallel forms of the same test to the same group with a short delay to avoid fatigue Look for high correlation between the scores of the two forms of the test Inter-rater reliability

28 Types of Reliability Internal Consistency
Degree to which instrument items are homogeneous and reflect the same underlying constructs Split-half testing where the test is split into two halves that contain the same types of questions Uses Cronbach’s alpha to determine internal consistency. Only one administration of the test is required Kuder-Richardson (KR20) for items with right and wrong answers

29 Practicality Is the survey economical Convenience
Cost of producing and administering the survey Time requirement Common sense! Convenience Adequacy of instructions Easy to administer Can the measurement be interpreted by others Scoring keys Evidence of validity and reliability Established norms

Download ppt "Measurement and Scaling"

Similar presentations

Ads by Google