2 Reliability of Consistency of What? Observers or ratersTests over timeDifferent versions of the same testA test at one point in time
3 Inter-Rater or Inter-Observer Reliability Object orphenomenon
4 Inter-Rater or Inter-Observer Reliability Object orphenomenonObserver 1
5 Inter-Rater or Inter-Observer Reliability Object orphenomenonObserver 1Observer 2
6 Inter-Rater or Inter-Observer Reliability Object orphenomenon?=Observer 1Observer 2
7 Inter-Rater or Inter-Observer Reliability Are different observers consistent?Can establish this outside of your study in a pilot study.Can look at percent of agreement (especially with category ratings).Can use correlation (with continuous ratings).
10 Test-Retest Reliability Stability over timeTest=TestTime 1Time 2
11 Test-Retest Reliability Measure instrument at two times for multiple persons.Compute correlation between the two measures.Assumes there is no change in the underlying trait between time 1 and time 2.
13 Parallel-Forms Reliability Form A=Form BTime 1Time 2
14 Parallel-Forms Reliability Stability across formsForm A=Form BTime 1Time 2
15 Parallel-Forms Reliability Administer both forms to the same people.Get correlation between the two forms.Usually done in educational contexts where you need alternative forms because of the frequency of retesting and where you can sample from lots of equivalent questions.
16 Internal Consistency Reliability Average inter-item correlation
17 Internal Consistency Reliability Average Inter-Item CorrelationTest
38 Like the average of all possible split-half correlations Internal Consistency ReliabilityCronbach’s alpha ()Item 1.87item 1item 3item 4item 2item 5item 6.87item 1item 3item 4item 2item 5item 6.87item 1item 3item 4item 2item 5item 6Item 2Item 3TestSH1 .87SH2 .85SH3 .91SH4 .83SH5 .86...SHn .85Item 4Like the average of all possible split-half correlationsItem 5Item 6 = .85