Presentation is loading. Please wait.

Presentation is loading. Please wait.

Chapter 6 Just the Truth: An Introduction to Understanding Reliability and Validity Part II  igma Freud & Descriptive Statistics.

Similar presentations


Presentation on theme: "Chapter 6 Just the Truth: An Introduction to Understanding Reliability and Validity Part II  igma Freud & Descriptive Statistics."— Presentation transcript:

1 Chapter 6 Just the Truth: An Introduction to Understanding Reliability and Validity Part II  igma Freud & Descriptive Statistics

2 What you will learn in Chapter 6 What reliability and validity are and why they are important Basic measurement scales Computing and interpreting reliability coefficients Computing and interpreting validity coefficients What reliability and validity are and why they are important Basic measurement scales Computing and interpreting reliability coefficients Computing and interpreting validity coefficients

3 Why Measurement? You need to know that the data you are collecting represents what it is you want to know about. How do you know that the instrument you are using to collect data works every time (reliability) and measures what it is supposed to (validity)? You need to know that the data you are collecting represents what it is you want to know about. How do you know that the instrument you are using to collect data works every time (reliability) and measures what it is supposed to (validity)?

4 Scales of Measurement Measurement is the assignment of values to outcomes following a set of rules There are four types of measurement scales Nominal Ordinal Interval Ratio Measurement is the assignment of values to outcomes following a set of rules There are four types of measurement scales Nominal Ordinal Interval Ratio

5 Nominal Level of Measurement Characteristics of an outcome that fits one and only one category Mutually exclusive categories such as male or female, Caucasian or African-American, etc. Categories cannot be ordered meaningfully Least precise level of measurement Characteristics of an outcome that fits one and only one category Mutually exclusive categories such as male or female, Caucasian or African-American, etc. Categories cannot be ordered meaningfully Least precise level of measurement

6 Ordinal Level of Measurement Characteristics being measured are ordered Rankings such as #1, #2, #3 You know that a higher rank is better, but not by how much Characteristics being measured are ordered Rankings such as #1, #2, #3 You know that a higher rank is better, but not by how much

7 Interval Level of Measurement Test or tool is based on an underlying continuum that allows you to talk about how much higher one score is than another Intervals along the scale are equal to one another Test or tool is based on an underlying continuum that allows you to talk about how much higher one score is than another Intervals along the scale are equal to one another

8 Ratio Level of Measurement Characterized by the presence of absolute zero on the scale An absence of any of the trait being measured Characterized by the presence of absolute zero on the scale An absence of any of the trait being measured

9 Things to Remember Any outcome can be assigned one of four scales of measurement Scales of measures have an order The “higher” up the scale of measurement, the more precise the data More precise scales contain all of the qualities of the scales below it Any outcome can be assigned one of four scales of measurement Scales of measures have an order The “higher” up the scale of measurement, the more precise the data More precise scales contain all of the qualities of the scales below it

10 Classical Test Theory: Os = Ts + E Observed score the actual score on a test True score theoretical reflection of the actual amount of a trait or characteristic an individual possesses Error score part of the score that is random Reliability = True Score / True Score + Error Observed score the actual score on a test True score theoretical reflection of the actual amount of a trait or characteristic an individual possesses Error score part of the score that is random Reliability = True Score / True Score + Error

11 Types of Reliability Test-Retest Measure of Stability Parallel Forms Measure of Equivalence Internal Consistency Measure of Consistency Cronbach’s Alpha (coefficient alpha) Inter-Rater Measure of Agreement Test-Retest Measure of Stability Parallel Forms Measure of Equivalence Internal Consistency Measure of Consistency Cronbach’s Alpha (coefficient alpha) Inter-Rater Measure of Agreement

12 Using the Computer SPSS and Cronbach’s Alpha

13 How Big is Big? Reliability coefficients should be positive 0.0 to 1.0 General Rules of Thumb… Test-Retest = Inter-Rater = 85% agreement Internal Consistency =.70 – 1.0 High Reliability DOES NOT mean quality!! Reliability coefficients should be positive 0.0 to 1.0 General Rules of Thumb… Test-Retest = Inter-Rater = 85% agreement Internal Consistency =.70 – 1.0 High Reliability DOES NOT mean quality!!

14 Establishing Reliability Make sure instructions are standardized across all settings Increase number of items or observations Delete unclear items Moderate easiness or difficulty of tests Minimize the effect of external events Make sure instructions are standardized across all settings Increase number of items or observations Delete unclear items Moderate easiness or difficulty of tests Minimize the effect of external events

15 What is the Truth? Validity The extent to which inferences made from a test are… Appropriate Meaningful Useful (American Psychological Association & the National Council on Measurement) Does the test measure what it is supposed to measure? Validity The extent to which inferences made from a test are… Appropriate Meaningful Useful (American Psychological Association & the National Council on Measurement) Does the test measure what it is supposed to measure?

16 Types of Validity Traditionally speaking there are three types of validity evidence: Content Validity Criterion Validity Predictive Criterion validity Concurrent Criterion validity Construct Validity Traditionally speaking there are three types of validity evidence: Content Validity Criterion Validity Predictive Criterion validity Concurrent Criterion validity Construct Validity

17 Content Validity Property of a test such that the test items sample the universe of items for which the test is designed. How to Establish… Content Expert Do items represent all possible items? How well do the number of items reflect what was taught? Property of a test such that the test items sample the universe of items for which the test is designed. How to Establish… Content Expert Do items represent all possible items? How well do the number of items reflect what was taught?

18 Criterion Validity Assesses whether a test reflects a set of abilities in a current (concurrent) or future (predictive) setting as measured by some other test. Concurrent Validity How well does my test correlate with the outcomes of a similar test right now? Predictive Validity How well does my test predict performance on a similar measure in the future? Assesses whether a test reflects a set of abilities in a current (concurrent) or future (predictive) setting as measured by some other test. Concurrent Validity How well does my test correlate with the outcomes of a similar test right now? Predictive Validity How well does my test predict performance on a similar measure in the future?

19 Construct Validity Most interesting…most difficult source of validity to establish Construct = group of interrelated variables such as... Aggression Hope Intelligence Want your construct to correlate with related behaviors and not correlate with behaviors that are not related. Most interesting…most difficult source of validity to establish Construct = group of interrelated variables such as... Aggression Hope Intelligence Want your construct to correlate with related behaviors and not correlate with behaviors that are not related.

20 All About Validity

21 Validity & Reliability The “Kissing Cousins” A test can be reliable but not valid A test cannot be valid unless it is reliable because… “A test cannot do what it is supposed to do (validity) until it does what it is supposed to do consistently (reliability).” The “Kissing Cousins” A test can be reliable but not valid A test cannot be valid unless it is reliable because… “A test cannot do what it is supposed to do (validity) until it does what it is supposed to do consistently (reliability).”

22 Chapter 7 Hypotheticals and You: Testing Your Questions Part III Taking Chances for Fun and Profit

23 What you will learn in Chapter 7 The difference between samples and populations The importance of… The null hypothesis The research hypotheses How to judge a good hypothesis The difference between samples and populations The importance of… The null hypothesis The research hypotheses How to judge a good hypothesis

24 What is a hypothesis? An “educated guess” Their role is to reflect the general problem statement or question that is driving the research Translates the problem or research question into a form that can be tested. An “educated guess” Their role is to reflect the general problem statement or question that is driving the research Translates the problem or research question into a form that can be tested.

25 Samples and Populations Population The large group to which you would like to generalize your findings Sample The smaller, representative group of the population that is used to do the research Sampling error – a measure of how well a sample represents the population Population The large group to which you would like to generalize your findings Sample The smaller, representative group of the population that is used to do the research Sampling error – a measure of how well a sample represents the population

26 The Null Hypothesis Statements that contain two or more things that are equal (unrelated) to one another H 0 :  1 =  2 The starting point and is accepted as true without knowing more information Benchmark to compare actual outcomes Statements that contain two or more things that are equal (unrelated) to one another H 0 :  1 =  2 The starting point and is accepted as true without knowing more information Benchmark to compare actual outcomes

27 The Research Hypothesis Statement that there is a relationship between two variables Two Types… Nondirectional -- H 1 : X 1 ≠ X 2 Reflects a difference; direction is not specified Two-tailed test Directional -- H 1 : X 1 > X 2 Reflects a difference; direction is specified One-tailed test Statement that there is a relationship between two variables Two Types… Nondirectional -- H 1 : X 1 ≠ X 2 Reflects a difference; direction is not specified Two-tailed test Directional -- H 1 : X 1 > X 2 Reflects a difference; direction is specified One-tailed test

28 Null & Research Hypotheses

29 Differences Between Null and Research Hypotheses NullResearch No relationship between variables Relationship between variables Refers to the populationRefers to the sample Indirectly testedDirectly tested Written using Greek symbolsWritten using Roman symbols Implied hypothesisExplicit hypothesis

30 What Makes a Good Hypothesis? Stated in a declarative form rather than a question Defines an expected relationship between variables Reflects the theory or literature on which they are based Brief and to the point Testable – includes variables that can be measured Stated in a declarative form rather than a question Defines an expected relationship between variables Reflects the theory or literature on which they are based Brief and to the point Testable – includes variables that can be measured

31 Glossary Terms to Know Hypothesis Null Hypothesis Research Hypothesis Directional & Non-directional hypotheses One-tailed & Two-tailed test Population Sample Sampling error Hypothesis Null Hypothesis Research Hypothesis Directional & Non-directional hypotheses One-tailed & Two-tailed test Population Sample Sampling error


Download ppt "Chapter 6 Just the Truth: An Introduction to Understanding Reliability and Validity Part II  igma Freud & Descriptive Statistics."

Similar presentations


Ads by Google