1. Reliability
When a Measurement Procedure yields consistent scores when the phenomenon being measured is not changing.
Degree to which scores are free of “measurement error”
Consistency of measurement

2. In broad sense, test reliability indicates the extent to which individual differences
in test scores are attributable to ‘true’ differences in the characteristics under
consideration.
Reliability is concerned with the degree of consistency or agreement between
two independently derived sets of scores.
Technically reliability is the ration of the variance of the observed score on
the shorter test and the variance of the long- test true score.

3. TYPES OF RELIABILITY (OR) METHODS OF RELIABILITY MEASUREMENT
Test – Restest Reliability
Alternate – Form Reliability or
Parallel Forms Method
3. Split – Half Method or Odd – Even Split – Half Method
4. Kuder – Richardson Reliability

4. Test-Retest Stability
Measure the same thing over and over to see if it always gives you the same result
Does not work as well with paper and pencil surveys 4

5. Types of Reliability Test-retest Inter-rater reliability
Intra-rater reliability
Statistical measures 5

6. Inter- and Intra-Rater Reliability
Inter- and intra-rater reliability equivalence parallel
Inter-rater: Two different raters rate the same thing to see if getting similar results
Intra-rater: Give the same survey to the same person a week apart to see if getting the same results
Reliability coeff. ® is the correlation between the scores obtained by the same persons on two administrations of the test. 6
Measurement 6

7. Different aspects of reliability Inter-observer, Inter-item
Inter-observer reliability
Repeated measures by different observers on the same subject
Especially important in coding open-ended questions
Inter-item reliability
Do the items in a composite measure correlate highly
Cronbach’s alpha

8. Types of Reliability Internal consistency
correlations amongst multiple items in a factor
e.g., Cronbach’s Alpha (a)
Test-retest reliability
correlation between time 1 & time 2
e.g., Product-moment correlation (r)
Internal consistency (single administration) and Test-reliability (multiple administrations) are based on classical test theory. A more advanced/refined reliability technique is based Item Response Theory, which involves examining the extent to which each item discriminates between individuals with high/low overall scores.
For more info see:

9. Reliability Interpretation
<.6 = not reliable
.6 = OK
.7 = reasonably reliable
.8 = good, strong reliability
.9 = excellent, very reliable
>.9 = potentially overly reliable or redundant measurement – this is subjective and whether a scale is overly reliable depends also on the nature what is being measured
Rule of thumb - reliability coefficients should be over .70, up to approx. .90

10. Alternate – Form Reliability or Parallel Forms Method
Same set of persons can thus be tested with one form on the first occasion and with another equivalent form on the other.
Correlation between the scores obtained on the two forms represent the reliability coefficient

11. Split – Half Method or Odd – Even Split – Half Method
Test is divided into two equal parts. The total score for both the halves is calculated.
Provides a measure of consistency with regard to content sampling

12. Kuder – Richardson Method
KR20 = r = N ( S2 - Σ pq )
N – S2
Where KR20 = the reliability estimate (r)
N = the number of items on the test
S2 = the variance of the total test score
P = the proportion of people getting each item correct
(this is found separately for each item)
q = the proportion of people getting each item incorrect .
For each item, q equals 1 – p.
Σ pq = the sum of the products of p times q for each item on
the test.

13. Problem : 1
Compute Internal Consistency Reliability for the 6 – item test, for which the following data
has been collected on 10 respondents. Each right answer on the item is scored 1 and
Each wrong answer is scored 0
Respondent
Item Score
Total Score 1 2 3 4 5 6 7 8 9 10

14. No of respondents passing the item
Item Score
Total Score
(x – x)
(x – x) 2 1 2 3 4 5 6
0.6
0.36
- 0.4
0.16
-0.4
- 2.4
5.76
- 1.4
1.96 7 8 9 10
1.6
2.56
No of respondents passing the item
Σx = 44
X = Σx n
44 =4.4
Σ(x – x)2
= 12.4
varaince
=Σ(x – x)2
12.4=1.24

15. No of respondents failing item2 4 3 1 P
0.8
0.6
0.7
0.9 q
0.2
0.4
0.3
0.1 Pq
0.16
0.24
0.21
0.09
= 1.1
rtt = N [ S2 – Σpq ]
N – S2
= [ 1.24 – 1.1 ]
6 –
= 1.2 * 0.113
. ˙ . Rtt = 0.14

16. Problem : 2
Respondent
Item Score
Total Score 1 2 3 4 5 6 7 8 9 10

17. No of respondent passing the item
Item Score
Total Score
(x – x)
(x – x)2 1 2 3 4 5 6
1.6
2.56
0.6
0.36
- 0.4
0.16
- 2.4
5.76 7 8 9 10
No of respondent passing the item
Σx = 44
X = Σx n
= 44 =4.4
Σ(x – x)2
= 18.4
varaince
=Σ(x – x)2
=18.4=1.84

18. No of respondents failing item3 2 1 4 P
0.7
0.8
0.9
0.6 q
0.3
0.2
0.1
0.4 Pq
0.21
0.16
0.09
0.24
rtt = N [ S2 – Σpq ]
N – S2
= [ 1.84 – 1.12 ]
6 –
= 1.2 *
. ˙ . rtt = 0.47

19. Problem : 3
Respondent
Item Score
Total Score 1 2 3 4 5 6 7 8 9 10
rtt =

20. ODD – EVEN SPLIT- HALF RELIABILITY
Problem 4
Compute the ODD-EVEN “SPLIT-HALF” reliability for the 6 item test for which the
following data has been collected for 10 respondents
Respondent
No.
Item Score 1 2 3 4 5 6 7 8 9 10

21. Solution Respondent No. Item Score Total Odd Even 1 2 3 4 5 6 T (T)2 O25 9 36 16 7 8 10 34
132 38 18

22. Variance
Var ( Total ) = ΣT2 – ( ΣT )2
N N
= 132 – ( 34 )2 = 1.64
Var ( Odd ) = ΣO2 – ( ΣO )2
N N
= 38 – ( 16 )2 = 1.24
Var ( Even ) = ΣE2 – ( ΣE )2
= 38 – ( 18 )2 = 0.56

23. ∞ = 2 [ Var.Total – ( Var. Odd + Var. Even ) ] Var.Total
Cronbach’s Alpha Co efficient
∞ = 2 [ Var.Total – ( Var. Odd + Var. Even ) ]
Var.Total
= 2 [ 1.64 – ( ) ]
1.64
∞ = ( -) 0.2
. ˙ . Here, there is low negative reliability.
Spearman's Browns Formula
. ˙ . rtt = 2r12 = 2( ) = = ( - ) 0.50
1 + r –

24. Problem 5
Respondent No.
Item Score 1 2 3 4 5 6 7 8 9 10

25. Solution Respondent No. Item Score Total Odd Even 1 2 3 4 5 6 7 8 T64 16 49 9 25 36 10 56
364 29 97 27 89

26. Variance
Var ( Total ) = ΣT2 – ( ΣT )2
N N
= 364 – ( 56 )2 = 5.04
Var ( Odd ) = ΣO2 – ( ΣO )2
N N
= 97 – ( 29 )2 = 1.29
Var ( Even ) = ΣE2 – ( ΣE )2
= 89 – ( 27 )2 = 1.61

27. ∞ = 2 [ Var.Total – ( Var. Odd + Var. Even ) ] Var.Total
Cronbach’s Alpha Co efficient
∞ = 2 [ Var.Total – ( Var. Odd + Var. Even ) ]
Var.Total
= 2 [ 5.04 – ( ) ]
5.04
∞ = 0.85
. ˙ . High Positive reliability.
Spearman's Browns Formula
. ˙ . rtt = 2r12 = 2 * = = = 0.92
1 + r

28. KR21 ( If KR20 & odd. Even cannot be used or no
KR21 ( If KR20 & odd. Even cannot be used or no. are not in ‘0’ & ‘1’ Form)
Problem 6
Respondent
Item Score 1 2 3 4 5 6 7 8 9 10

29. Solution 18 19 17 14 24 21 187 60.1 Respondent Item Score
Total Score x
(x – x)
(x – x) 2 1 2 3 4 5 6 18
- 0.7
0.49 19
0.3
0.09 17
- 1.7
2.89 14
- 4.7
22.09 7 8 24
5.3
28.09 9 21
2.3
5.29 10
Total (y) 36 27 33 30 28
187
60.1
( y – y )
4.83
- 4.16
1.83
- 1.16
- 3.16
( y – y ) 2
23.36
17.36
3.36
1.36
10.03

30. ( x = Σx ) x = 187 =18.7 n 10 ( y = Σ y ) y = 187 = 31.16 N 6 n = 10
Var (y) = Σ (y – y) 2 = = 9.81
Var (x) = Σ (x – x) 2 = = 6.01 n
KR21 = N [ Var x – Var y ]
N – Var x
= 6 [ 6.01 – 9.81 ]
= ( - ) 0.76
. ˙ . Negative high reliability

31. Since the reliability is poor one has to expand the test i. e
Since the reliability is poor one has to expand the test i.e. add new items in the test. Existing test contains =6 items
K=6 , r= -.76
To increase internal consistency to 0.5 one should add more no. of items to the test
Kd= rd/1-rd
K r/1-r

32. Problem 7
Respondent
Item Score 1 2 3 4 5 6 7 8 9 10