Presentation on theme: "Part II Sigma Freud & Descriptive Statistics"— Presentation transcript:
1 Part II Sigma Freud & Descriptive Statistics Chapter 6 Just the Truth:An Introduction to Understanding Reliability and Validity
2 Why Measurement? What is measurement? You need to know that the data you are collecting represents what it is you want to know about.How do you know that the instrument you are using to collect data works every time (reliability) and measures what it is supposed to (validity)?
3 Scales of MeasurementMeasurement is the assignment of values to outcomes following a set of rulesThere are four types of measurement scalesNominalOrdinalIntervalRatio
4 Nominal Level of Measurement Characteristics of an outcome that fits one and only one categoryMutually exclusive categories such asMale or FemaleDemocrat, Republican, or IndependentCategories cannot be ordered meaningfullyLeast precise level of measurement
5 Ordinal Level of Measurement Characteristics being measured are orderedRankings such as #1, #2, #3You know that a higher rank is better, but not by how much
6 Interval Level of Measurement Test or tool is based on an underlying continuum that allows you to talk about how much higher one score is than anotherIntervals along the scale are equal to one anotherExample: “Rate your restaurant experience on a scale of 1-7 with 1 = unsatisfactory and 7 = excellent”
7 Ratio Level of Measurement Characterized by the presence of absolute zero on the scaleAn absence of any of the trait being measuredExamples:How many kids do you have? (can have 0)Scores on a test (0 is possible!)
8 Things to RememberAny outcome can be assigned one of four scales of measurementScales of measures have an orderThe “higher” up the scale of measurement, the more precise (and useful) the data areUse the scale most appropriate for the research task at hand
9 Classical Test Theory: Os = Ts + E Observed scorethe actual score on a test, scale or measureTrue scoretheoretical reflection of the actual amount of a trait or characteristic an individual possessesError scorepart of the score that is random, or the difference between the Observed and True scoresReliability = True Score / (True Score + Error)
10 Types of Reliability Test-Retest Parallel Forms Internal Consistency Measure of StabilityParallel FormsMeasure of EquivalenceInternal ConsistencyMeasure of ConsistencyCronbach’s Alpha (coefficient alpha)Inter-RaterMeasure of Agreement
12 How Big is Good Enough? Reliability coefficients should be positive 0.0 to 1.0General Rules of Thumb…Test-Retest =Inter-Rater = 85% agreement or betterInternal Consistency alpha = .70 – 1.0High Reliability alone DOES NOT mean you are testing or measuring the right thing!!
13 Establishing Reliability Make sure instructions are standardized across all settingsIncrease number of items or observationsDelete unclear itemsModerate easiness or difficulty of tests (“middle-of-the-road” strategy)Minimize the effect of external events
14 What is the Truth? Validity The extent to which inferences made from a test are…AppropriateMeaningfulUseful(American Psychological Association & the National Council on Measurement)Does the test measure what it is supposed to measure?
15 Types of Validity Three types of validity: Content Validity Criterion ValidityPredictive Criterion validityConcurrent Criterion validityConstruct Validity
16 Content ValidityProperty of a test such that the test items sample the universe of items for which the test is designed.How to Establish…Content ExpertDo items represent all possible items?How well do the number of items reflect what was taught?
17 Criterion ValidityAssesses whether a test reflects a set of abilities in a current (concurrent) or future (predictive) setting as measured by some other test.Concurrent ValidityHow well does my test correlate with the outcomes of a similar test right now?Predictive ValidityHow well does my test predict performance on a similar measure in the future?
18 Construct Validity Most difficult source of validity to establish Construct = group of interrelated variables such as...AggressionHopeIntelligence (Verbal, Quantitative, Emotional)Want your construct to correlate with related behaviors and not correlate with behaviors that are not related.
20 Validity & Reliability The “Kissing Cousins”A test can be reliable but not validA test cannot be valid unless it is reliable because…“A test cannot do what it is supposed to do (validity) until it does what it is supposed to do consistently (reliability).”