Presentation is loading. Please wait.

Presentation is loading. Please wait.

PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Chapter 5 Reliability and Validity.

Similar presentations


Presentation on theme: "PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Chapter 5 Reliability and Validity."— Presentation transcript:

1 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Chapter 5 Reliability and Validity

2 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Overview Measuring Variables l Choosing a Behavior to Measure l Overview of Types of Measurement Errors –Bias –Random error l Reliability l Validity Manipulating Variables l Validity –Threats to – Establishing l Types of manipulations

3 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Two Types of Measurement Error l Bias l Random error

4 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Three “Places” Measurement Error Can Occur l Observer/Scorer l Participant l Person administering the measure

5 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Two Types of Observer Error l Observer bias (Scorer bias) l Random observer error

6 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Minimizing Observer Errors l Why it is more important to reduce observer bias than random error l Techniques for reducing observer bias*

7 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Techniques for Reducing Observer Bias l Eliminating human observer errors by eliminating the human observer l Limiting human observer errors by limiting the human observer’s role l Reducing observer bias by making observers “blind” l Conclusions about reducing observer bias

8 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Reducing Random Observer Error l Most of the techniques that reduce observer bias reduce random observer error

9 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Errors in Administering the Measure l Types –Experimenter (researcher) bias –Random error l Solutions –Blind technique to reduce bias –Standardization to reduce both bias and random error

10 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Errors Due to the Participant l Bias due to the participant (Subject bias) l Random error due to the participant

11 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Subject (Participant) Bias l Obeying demand characteristics l Social desirability bias

12 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Conclusions about Reducing Subject Biases l Blind techniques can reduce demand characteristics l Making participants anonymous can reduce social desirability bias

13 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Summary of Types of Measurement Error l Try to reduce all forms of measurement error l Really focus on reducing bias

14 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Reliability: The (Relative) Absence of Random Error l The importance of being reliable: Reliability as a prerequisite to validity l Using test-retest reliability to assess overall reliability: To what degree is a measure “random error free”?

15 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Identifying (and Then Dealing with) the Main Source of a Measure’s Reliability Problems l Are observers to blame for low test-retest reliability?: Assessing observer reliability l Non-observer sources of random error l Using internal consistency measures to estimate random error due to participants

16 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Internal Consistency: Test Questions Should Agree with Each Other l Random error due to participants may cause low internal consistency

17 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Two Solutions to Problems Caused by Random Participant Error l Add questions to let random participant error balance out l Ask better questions to reduce random participant error

18 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Measuring Internal Consistency l Average inter-item correlations as indexes of internal consistency l Split-half coefficients as indexes of internal consistency l Additional indexes of internal consistency l Conclusions about internal consistency’s relationship to reliability

19 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Conclusions About Reliability l Reliability is a prerequisite for validity l If test-retest reliability is low, try to find out where reliability problem is and fix it. l Reliability does not guarantee validity

20 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Beyond Reliability: Establishing Construct Validity l Content Validity l Internal Consistency l Convergent Validity: Getting evidence that you are measuring the right construct l Discriminant Validity: Showing that you are not measuring the wrong construct

21 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Manipulating Variables l Common threats to a manipulation’s validity l Evidence used to argue for a manipulation’s construct validity l Tradeoffs among three common types of manipulations l Conclusions

22 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Common Threats to a Manipulation’s Validity l Random error l Experimenter bias l Subject biases

23 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Evidence Used to Argue for a Manipulation’s Construct Validity l Consistency with theory l Manipulation checks

24 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Tradeoffs Among Three Common Types of Manipulations l Instructional manipulations l Environmental manipulations l Manipulations involving stooges

25 PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Concluding Remarks Operational definitions should l Be consistent with dictionary/theory definitions l Be standardized to reduce bias and random error l Have evidence to support their validity


Download ppt "PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Chapter 5 Reliability and Validity."

Similar presentations


Ads by Google