1. Reliability and Validity
Chapter 4
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

2. Reliability-Having confidence in the consistency of the test results.
Validity-Having confidence that the
test is measuring what it is supposed to
measure.
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

3. Correlation- A statistical method of observing
the degree of relationship between two sets of
data or two sets of variables.
Correlation coefficient -The numerical
representation of the strength and direction of the
relationship between two sets of variables or data
(0 to 1.0; can be positive or negative).
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

4. General Principles of Research
Correlational Studies
A positive correlation (+) means that as one variable increases, so does the other.
A zero or near zero correlation means that the variables have no relationship

6. The direction of the line plotted on the scatter plot provides
a clue about the relationship. If the relationship is positive,
the direction of the line looks like this:
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

8. General Principles of Research
Correlational Studies
The value of the correlation coefficient can range from –1.00 to
The higher the absolute value, the stronger the relationship is, regardless of the direction.
A negative correlation (-) means that as one variable increases, the other decreases.

10. If the data represent a negative correlation, the direction
of the line in the scatter plot looks like this:
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

11. Concept Check
The greater the score on a depression inventory, the lower the score on a memory test
Negative

12. Concept Check Which relationship is stronger? +. 85 or -.90 or 1.2
- .90
+. 85 or -.90 or 1.2
.25 or or + .63
+.63

13. Figure 2.9 A strong correlation between depression and impaired sleep does not tell us whether depression interferes with sleep, poor sleep leads to depression, or whether another problem leads to both depression and sleep problems.

14. Positive Correlation
* In investigating how data are related, it is important to
determine if the two sets of data represent positive,
negative, or no correlation.
*In a positive correlation, when a student scores high on
the first variable or test, the student will also score high
on the second measure.
*The data below illustrate a positive correlation:
Student
Student
Student
Student
Student
Student
Student
Student
Set 2
Set 1
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

15. Negative Correlation In a negative correlation, when a student
scores high on one variable or test, the
student will score low on the other variable
or test.
The data below illustrate a negative
correlation:
Student
Student
Student
Student
Student
Student
Student
Student
Set 1
Set 2
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

16. Height and Temperament of 20 Men
Correlation
Height and Temperament of 20 Men 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 80 63 61 79 74 69 62 75 77 60 64 76 71 66 73 70 68 90 42 81 39 48 72 57 30 84
Subject
Height in
Inches
Temperament

17. Scatterplot of Height and Temperament
Correlation 95 90 85 80 75 70 65 60 55 50 45 40 35 30 25
Temperament
scores
Height in inches
Scatterplot of Height and Temperament

18. Pearson correlation coefficient
r = the Pearson coefficient
r measures the amount that the two variables (X and Y) vary together (i.e., covary) taking into account how much they vary apart
Pearson’s r is the most common correlation coefficient; there are others.

19. Computing the Pearson correlation coefficient
To put it another way: Or

20. Sum of Products of Deviations
Measuring X and Y individually (the denominator):
compute the sums of squares for each variable
Measuring X and Y together: Sum of Products
Definitional formula
Computational formula
n is the number of (X, Y) pairs

21. Correlation Coefficent:
the equation for Pearson’s r:
expanded form:

22. Example
What is the correlation between study time and test score:

23. Calculating values to find the SS and SP:

24. Calculating SS and SP

25. Calculating r

26. Correlation Coefficient Interpretation
Range
Strength of
Relationship
Very Low
Low
Moderate
High Moderate
Very High

27. Methods of Studying Reliability
Test-retest reliability- A study that employs the readministration
of a single instrument to check for consistency across time.
Equivalent forms reliability-Consistency of a test using like
forms that measure the same skill, domain, or trait; also known
as alternate forms reliability.
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

28. Methods of Studying Reliability, Continued
Internal consistency-Methods to study the reliability
across the items of the test. Examples include split-half
reliability, K-R 20, and coefficient alpha.
Split-half reliability-A method of measuring internal
consistency by studying the reliability across items by
comparing the data of the two halves of the test.
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

29. Methods of Studying Reliability, Continued
Interrater Reliability- The consistency of a test to
measure a skill, trait, or domain across examiners.
This type of reliability is most important when
responses are subjective or open-ended.
Reliability coefficients may vary across age and
grade levels of a specific instrument.
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

30. Methods of Studying Reliability, Continued
Kuder-Richardson 20- (K-R 20)- A formula used to check
consistency across items of an instrument that has items
scored as 1 or 0 or right/wrong.
Coefficient Alpha- A formula used to check the consistency
across items of instruments with responses of varying credit.
For example, items may be scored as 0,1,2,or 3 points.
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

31. Standard Error of Measurement
Each test score is made of two parts: true score and error.
A student’s true score may only be estimated. The standard
error of measurement is a method used to estimate the
amount of error of a test. It represents the typical amount of
error of any obtained score. The standard error of
measurement is used to estimate a range of scores within
which the student’s true scores exists.
Obtained Score = True Score + Error
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

32. Standard Error of Measurement, Continued
The standard error of measurement is calculated using the
following formula:
SEM = SD
1-.r
Where SEM = the standard error of measurement
SD = the standard deviation of the norm group of scores
obtained during the development of the instrument
r = the reliability coefficient
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

33. Example of Calculating SEM
For a specific test, the standard deviation is 4. The
reliability coefficient is .89. The SEM would be: 4
1- .89
.11
This represents the
amount of
error on this test
instrument. 4
4 x
.33 = 1.32
SEM = 1.32
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

34. The range of possible scores is
Applying the SEM
The SEM was A student’s obtained score was 89. To determine the range of possible true scores, add and subtract the SEM from the obtained score of 89.
= 90.32
= 87.68
The range of possible scores is
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

35. Selecting the Best Test Instruments
When considering which tests will be the most
reliable, it is important to select a test that has
the highest reliability coefficient and the
smallest standard of error.
This will mean that the results obtained are
more likely to be more consistent with the
student’s true ability.
The obtained score will contain less error.
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

36. Test Validity Validity indicates the degree of quality of the test
instrument. This is a measure of how accurately
the test measures what it is designed to measure.
Some of the methods that indicate validity of a
test include criterion-related validity, content
validity, construct validity.
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

37. Criterion-related Validity
There are two ways to study criterion-related validity.
Concurrent-related validity- When a test is compared
with another measure at the same time.
Predictive validity-When a test is compared with a
measure in the future. For example, when college
entrance exams are compared with student performance in
College (GPAs).
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

38. Content Validity In order for a test to have good content validity, it
must have items that are representative of the
domain or skill being assessed.
During the development of the test, items are
selected after careful study of the items and the
domain they represent.
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

39. Content Validity In order for a test to have good content validity, it
must have items that are representative of the
domain or skill being assessed.
During the development of the test, items are
selected after careful study of the items and the
domain they represent.
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

40. Construct Validity Construct validity means that the instrument has
the ability to assess the psychological constructs
it was meant to measure.
A construct is a psychological trait or
characteristic such as creativity or mathematical
ability.
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

41. Studying Construct Validity
Construct validity can be studied by
Investigating:
Developmental changes-If an ability is
expected to change across time and that
ability is the construct of the test, testing
across time will show changes in
the performance on that test as the ability
responds to developmental change.
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

42. Construct Validity Continued
Correlation with other tests-If the construct has been
measured successfully by other instruments, a correlation
with those other instruments is evidence of construct
validity.
Factor analysis-Using a statistical method known as factor
analysis, evidence can be gathered for construct validity.
Factor analysis allows the investigator to see if items that
are thought to measure a construct are answered in the
same manner.
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

43. Construct Validity Continued
Internal Consistency-This measure also provides
evidence of construct validity.
Convergent or discriminant validity-These
Measures indicate that the performance is
consistent with like measures (convergent) or
the performance is unlike measures thought to
measure different constructs than the one being
assessed by the specific instrument.
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e

44. Construct Validity Continued
Experimental interventions-When the
construct being assessed can be influenced
by an intervention or treatment, assessment
before and after the intervention provides
evidence of construct validity.
Copyright ©2006 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved.
Terry Overton Assessing Learners with Special Needs, 5e