1. 5. Reliability and Validity
What are random error and systematic error, and how do they
influence measurement?
What is reliability? Why must a measure be reliable?
How are test-retest and equivalent-forms reliability measured?
How are split-half reliability and coefficient alpha used to
assess the internal consistency of a measured variable?
What is interrater reliability?

2. What are face validity and content validity?
How are convergent and discriminant validity used to assess
the construct validity of a measured variable?
What is criterion validity?
What methods can be used to increase the reliability and validity
of a self-report measure?
How are reliability and construct validity similar? How are
they different?

3. Random and Systematic Error
Random Error
1) fluctuations in the person’s current mood.
2) misreading or misunderstanding the questions
3) measurement of the individuals on different days or in different
places.
These error may cancel out as you collect many samples
Systematic Error
Sources of error including the style of measurement, tendency
toward self-promotion, cooperative reporting, and other conceptual
variables are being measured.

4. So, we have to reduce these errors to prove scientific findings
How well do our measured variables “capture” the conceptual
variables?
Reliability
The extent to which the variables
are free from random error, usually
determined by measuring the
variables more than once. * * * * * *
CVs *
Construct Validity *
The extent to which a measured
variable actually measures the
conceptual variables that is design
to assess the extent to which it is
known to reflect the conceptual
variable other measured variables * * * * *
CVs * * * *

5. Test-Retest Reliability
The extent to which scores on the same
measured variable correlate with each
other on two different measurements
given at two different time.
Questionnaire 9/20
___ I feel I do not have much proud of.
___ On the whole, I am satisfied with myself
___ I certainly feel useless at times
___ At times I think I am no good at all
___ I have a number of good qualities
___ I am able to do things as well as others
Questionnaire 9/27
___ I feel I do not have much proud of.
___ On the whole, I am satisfied with myself
___ I certainly feel useless at times
___ At times I think I am no good at all
___ I have a number of good qualities
___ I am able to do things as well as others 4 4 3 4 2 1 1 1 4 4 3 4
Retesting Effect

6. Equivalent-Forms Reliability
The extent to which two equivalent
variables given at different time
correlate each other.
Example. GRE, SAT, GMAT, TOEFL
22 X 45 =
85 X (23-11) =
72-14 X 12 X (7-1) =
32 X 45 =
85 X (41-11) =
72-14 X 25 X (6-1) =

7. Reliability as Internal Consistency
The extent to which the scores on
the items correlate with each other
and thus are all measuring the true
score rather than reflecting random
error.
How Do You Measure
Internal Consistency?
Questionnaire 9/20
___ I feel I do not have much proud of.
___ On the whole, I am satisfied with myself
___ I certainly feel useless at times
___ At times I think I am no good at all
___ I have a number of good qualities
___ I am able to do things as well as others
Split-half Reliability
Coefficient Alpha

8. Interrater Reliability
The extent to which the scores
counted by coders correlate
each other.
How Do You Measure
Interrater Reliability?
Aggression Code
Coder 1 Coder 2
Hit boy A ______ ______
Hit boy B ______ ______
Hit girl A ______ ______
Hit girl B ______ ______
Cohen’s Kappa 1 3 3 2 1

9. t Questionnaire 1 Item 1 Item 2 Item 3 Questionnaire 1 Item 1 Item 2
Test-Retest Reliability
Reliability as
Internal Consistency
Questionnaire 2
Item 1
Item 2
Item 3
Equivalent-Forms
Reliability
Interrater Reliability

10. Validity Construct Validity The extent to which a measured variable
actually measures the conceptual variable
(that is, the construct) that it is designed
to assess.
Criterion Validity
The extent to which a self-report measure
correlates with a behavioral measured
variables.

11. X Face Validity The extent to which the measured
Construct Validity
Face Validity
The extent to which the measured
variable appears to be an adequate
measure of the conceptual variables
I don’t like Japanese
Strongly Disagree Strongly Agree
Discrimination
towards Japanese
Measured
Variable
Conceptual
Variable X

12. The degree to which the measured variable appears to have adequately
Construct Validity
Content Validity
The degree to which the measured
variable appears to have adequately
sampled from the potential domain
of question that might relate to
the conceptual variable of interest.
Sympathy
Verbal Aptitude
Intelligence
Math Aptitude

13. Interdependence Scale The extent to which a measured variable
Construct Validity
Convergent Validity
Interdependence Scale
The extent to which a measured variable
is found to be related to other measured
variables designed to measure the same
conceptual variable.
Collectivism Scale
Discriminant Validity
The extent to which a measured variable
is found to be unrelated to other measured
variables designed to measure the different
conceptual variables.
Independence Scale
Interdependence Scale

14. The extent to which the scores can predict the participants’
Criterion Validity
Predictive Validity
The extent to which the scores
can predict the participants’
future performance.
Example. GRE, SAT...
Concurrent Validity
The extent to which the self-report
measure correlate with the behavioral
measure that is assessed at the same
time.

15. How Do You Improve the Reliability and Validity of
Your Measured Variables?
1. Conduct a pilot test, trying out a questionnaire or other
research instruments on a small group.
2. Use multiple measures.
3. Ensure variability that there is in your measures.
4. Write good items.
5. Get your respondents to take your questions
seriously.
6. Make your items nonreactive.
7. Be certain to consider face and content validity by choosing
reasonable terms and that cover a broad range of issues
reflecting the conceptual variables..
8. Use existing measures.

16. time Items-Scales Face Validity Predictive Validity
Conceptual
Variables
Future
behaviors
Face Validity
Predictive Validity
Other
Domain of the
CVs
Measured
Variables
(Self-Report)
Measured
Variables
(Behavioral)
Concurrent Validity
Content Validity
Similar Items-Scales
Items-Scales
Other Items-Scales
Convergent Validity
Discriminant Validity