2 Scaling, Reliability and Validity Chapter 9Scaling, Reliability and Validity
3 Chapter ObjectivesKnow how and when to use the different forms of rating scales and ranking scalesExplain stability and consistency, and how they are establishedExplain the different forms of validityDiscuss what ‘goodness’ of measures means, and why it is necessary to establish it in research
4 Rating and Ranking Scales Rating Scaleshave several response categories and are used to elicit responses with regard to the object, event or person studied.Ranking Scalesmake comparisons between or among objects, events or persons, and elicit the preferred choices and ranking among them.
16 Paired ComparisonUsed when, among a small number of objects, respondents are asked to choose between two objects at a time.The paired choices for n objects will be ((n) (n-1)/2).
17 Forced ChoiceRank your preferences among the following magazines, 1 being your preferred choice and 5 being your least preferred:Australian Financial Review __Business Review Weekly __Playboy __The Economist __Time __
18 Comparative ScaleIn a volatile financial environment, compared with shares, how useful is it to invest in government bonds?More useful About the same Less useful
19 Goodness of Measures Reliability measures Validity measures How stable and consistent is the measuring instrument?Validity measuresAre we measuring the right thing?
22 Reliability Stability Internal consistency refers to the ability of a measure to maintain stability over time, despite uncontrollable testing conditions or the state of the respondents themselvesInternal consistencyindicates how well the items ‘hang together as a set’ and can independently measure the same concept, so respondents attach the same overall meaning to each of the items
23 Stability of Measures Test-retest reliability the reliability coefficient obtained with a repetition of the same measure on a second occasionParallel-form reliabilitythe correlation obtained from responses on two comparable sets of measures (changed for wording & question order) tapping the same construct
24 Internal Consistency of Measures Inter-item consistency reliabilitytest of the consistency of respondents’ answers to all the items in a measureusually tested by Cronbach’s coefficient alphaSplit-half reliabilityreflects the correlations between two halves of an instrument