1. Reliability & Validity
“dependability”
is the indicator consistent?
same result every time?
Does not necessary measure what you think it measures (ex. May consistently measure something other than the concept)
Validity
measurement validity - how well the conceptual and operational definitions mesh with each other
does measurement tool measure what we think ?

2. Relationship between Measurement Reliability and Validity
reliability necessary for validity but does not guarantee it (“necessary but not sufficient”)
measure can be reliable but invalid
Source: Neuman (2000: 171)

3. Types of Reliability stability over time representative
across different subgroups of a population
(ex. of problems: young people may exaggerate their ages, older people may reduce theirs)
equivalence
Comparable results from multiple indicators
intercoder reliability

4. Improving Reliability
clearly conceptualize constructs
increase level of measurement
use pretests, pilot studies
e.g. use multiple indicators :
Measurement Using
Multiple Indicators
Neuman (2000: 167)
Dependent
Variable Measure
Independent
Empirical
Association? a2 a3 a1 b1 b2 A B
Specific Indicators

5. Selected Types of Validity: Face Validity
judgement by the scientific community that indicator measures the concept
Construct/concept
Measure ?
Scientific Community

6. Content Validity
measure represents all the aspects of conceptual definition of construct.
judgement concerning how adequately a test samples behavior representative of the universe of behavior the test was designed to sample.
Love
Beliefs
& Values

7. Criterion Validity
The validity of an indicator is verified by comparing it with another measure of the same construct in which a researcher has confidence.
Predictive and concurrent validity
Future Behavior
Predictive
Construct
Measure
Old Measure
Concurrent
Similar Results ?

8. Construct Validity
A type of measurement validity that uses multiple indicators
convergent and discriminate

9. Other Dimensions of Validity
Internal Validity
no error of logic internal to research design
External Validity
results can be generalized
Statistical validity
correct statistical methodology chosen ?
assumptions fully met

10. More terms & concepts for assessing validity (usually in experimental research)
Threats to Internal Validity
Selection bias
History effects (something in context changes)
Maturation
Testing effect
Instrumentation
Mortality
Statistical regression
Contamination
Compensatory behaviour
Experimenter expectanc
Threats to External Validity
Realism
Reactivity

11. Another Aspect of Measurement Design: Levels of Measurement
1-Nominal (ex. Mother tongue)
different categories (names, labels, images)
not ranked
2-Ordinal (county fair prizewinners ranked by first, second & third prize)
different categories
rank-ordered
attributes indicate relatively more or less of that variable
distance between the attributes of a variable is imprecise

12. Levels of Measurement (cont’d)
3- Interval Measures (age measured by 5 year age groups)
different categories
ranked in order
Can tell amount of difference between categories
Usually no true zero
4- Ratio Measures (age measured by date of birth)
amount of difference between categories
also possible to state proportion (have a true zero)
Relations between levels --can collapse from higher into lower, not vice versa

13. The Research Process
Babbie (1995: 101)