Presentation is loading. Please wait.

Presentation is loading. Please wait.

Types of Reliability. Reliability of Consistency of What? l Observers or raters l Tests over time l Different versions of the same test l A test at one.

Similar presentations


Presentation on theme: "Types of Reliability. Reliability of Consistency of What? l Observers or raters l Tests over time l Different versions of the same test l A test at one."— Presentation transcript:

1 Types of Reliability

2 Reliability of Consistency of What? l Observers or raters l Tests over time l Different versions of the same test l A test at one point in time

3 Inter-Rater or Inter-Observer Reliability Object or phenomenon

4 Inter-Rater or Inter-Observer Reliability Observer 1 Object or phenomenon

5 Inter-Rater or Inter-Observer Reliability Observer 1 Observer 2 Object or phenomenon

6 Inter-Rater or Inter-Observer Reliability Observer 1 Observer 2 Object or phenomenon = ?

7 Inter-Rater or Inter-Observer Reliability l Are different observers consistent? l Can establish this outside of your study in a pilot study. l Can look at percent of agreement (especially with category ratings). l Can use correlation (with continuous ratings).

8 Test-Retest Reliability Time 1 Time 2

9 Test-Retest Reliability TestTest Time 1 Time 2 =

10 Test-Retest Reliability TestTest Time 1 Time 2 = Stability over time

11 Test-Retest Reliability l Measure instrument at two times for multiple persons. l Compute correlation between the two measures. l Assumes there is no change in the underlying trait between time 1 and time 2.

12 Parallel-Forms Reliability Time 1 Time 2

13 Parallel-Forms Reliability Form B Time 1 Time 2 Form A =

14 Parallel-Forms Reliability Form B Time 1 Time 2 Stability across forms Form A =

15 Parallel-Forms Reliability l Administer both forms to the same people. l Get correlation between the two forms. l Usually done in educational contexts where you need alternative forms because of the frequency of retesting and where you can sample from lots of equivalent questions.

16 Internal Consistency Reliability Average inter-item correlation

17 Internal Consistency Reliability Test Average Inter-Item Correlation

18 Internal Consistency Reliability Test Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Average Inter-Item correlation

19 Internal Consistency Reliability Test Item 1 Item 2 Item 3 Item 4 Item 5 Item I1I2I3I4I5I6I1I2I3I4I5I6I1I2I3I4I5I6I1I2I3I4I5I6 I1I1I2I2I3I3I4I4I5I5I6I6I1I1I2I2I3I3I4I4I5I5I6I6 Average inter-item correlation

20 Internal Consistency Reliability Test Item 1 Item 2 Item 3 Item 4 Item 5 Item I1I2I3I4I5I6I1I2I3I4I5I6I1I2I3I4I5I6I1I2I3I4I5I6 I1I1I2I2I3I3I4I4I5I5I6I6I1I1I2I2I3I3I4I4I5I5I6I6 Average inter-item correlation.90

21 Average item-total correlation Internal Consistency Reliability

22 Test Average item-total correlation Internal Consistency Reliability

23 Average item-total correlation Internal Consistency Reliability Test Item 1 Item 2 Item 3 Item 4 Item 5 Item 6

24 I1I2I3I4I5I6I1I2I3I4I5I6I1I2I3I4I5I6I1I2I3I4I5I6 I 1 I 2 I 3 I 4 I 5 I 6 Total Average item-total correlation Internal Consistency Reliability Test Item 1 Item 2 Item 3 Item 4 Item 5 Item 6

25 I1I2I3I4I5I6I1I2I3I4I5I6I1I2I3I4I5I6I1I2I3I4I5I6 I 1 I 2 I 3 I 4 I 5 I 6 Total Average item-total correlation.85 Internal Consistency Reliability Test Item 1 Item 2 Item 3 Item 4 Item 5 Item 6

26 Split-half correlations Internal Consistency Reliability

27 Test Split-half correlations Internal Consistency Reliability

28 Split-half correlations Internal Consistency Reliability Test Item 1 Item 2 Item 3 Item 4 Item 5 Item 6

29 Split-half correlations Item 1 Item 3 Item 4 Internal Consistency Reliability Test Item 1 Item 2 Item 3 Item 4 Item 5 Item 6

30 Split-half correlations Item 2 Item 5 Item 6 Internal Consistency Reliability Item 1 Item 3 Item 4 Test Item 1 Item 2 Item 3 Item 4 Item 5 Item 6

31 Split-half correlations.87 Internal Consistency Reliability Item 2 Item 5 Item 6 Item 1 Item 3 Item 4 Test Item 1 Item 2 Item 3 Item 4 Item 5 Item 6

32 Cronbach’s alpha (  ) Internal Consistency Reliability

33 Test Cronbach’s alpha (  ) Internal Consistency Reliability

34 Cronbach’s alpha (  ) Internal Consistency Reliability Test Item 1 Item 2 Item 3 Item 4 Item 5 Item 6

35 Cronbach’s alpha (  ).87 item 1 item 3 item 4 item 2 item 5 item 6.87 item 1 item 3 item 4 item 2 item 5 item 6.87 item 1 item 3 item 4 item 2 item 5 item 6 Internal Consistency Reliability Test Item 1 Item 2 Item 3 Item 4 Item 5 Item 6

36 SH 1.87 SH 2.85 SH 3.91 SH 4.83 SH SH n.85 Internal Consistency Reliability Cronbach’s alpha (  ).87 item 1 item 3 item 4 item 2 item 5 item 6.87 item 1 item 3 item 4 item 2 item 5 item 6.87 item 1 item 3 item 4 item 2 item 5 item 6 Test Item 1 Item 2 Item 3 Item 4 Item 5 Item 6

37 SH 1.87 SH 2.85 SH 3.91 SH 4.83 SH SH n.85  =.85 Internal Consistency Reliability Cronbach’s alpha (  ).87 item 1 item 3 item 4 item 2 item 5 item 6.87 item 1 item 3 item 4 item 2 item 5 item 6.87 item 1 item 3 item 4 item 2 item 5 item 6 Test Item 1 Item 2 Item 3 Item 4 Item 5 Item 6

38 SH 1.87 SH 2.85 SH 3.91 SH 4.83 SH SH n.85  =.85 Like the average of all possible split-half correlations Internal Consistency Reliability Cronbach’s alpha (  ).87 item 1 item 3 item 4 item 2 item 5 item 6.87 item 1 item 3 item 4 item 2 item 5 item 6.87 item 1 item 3 item 4 item 2 item 5 item 6 Test Item 1 Item 2 Item 3 Item 4 Item 5 Item 6

39 Internal Consistency Reliability - Summary l Average inter-item correlation l Average item-total correlation l Split-half reliability Cronbach’s alpha (  ) Cronbach’s alpha (  )


Download ppt "Types of Reliability. Reliability of Consistency of What? l Observers or raters l Tests over time l Different versions of the same test l A test at one."

Similar presentations


Ads by Google