Presentation on theme: "Part II Sigma Freud & Descriptive Statistics"— Presentation transcript:
1 Part II Sigma Freud & Descriptive Statistics Chapter 6 Just the Truth:An Introduction to Understanding Reliability and Validity
2 What you will learn in Chapter 6 What reliability and validity are and why they are importantBasic measurement scalesComputing and interpreting reliability coefficientsComputing and interpreting validity coefficients
3 Why Measurement?You need to know that the data you are collecting represents what it is you want to know about.How do you know that the instrument you are using to collect data works every time (reliability) and measures what it is supposed to (validity)?
4 Scales of MeasurementMeasurement is the assignment of values to outcomes following a set of rulesThere are four types of measurement scalesNominalOrdinalIntervalRatio
5 Nominal Level of Measurement Characteristics of an outcome that fits one and only one categoryMutually exclusive categories such as male or female, Caucasian or African-American, etc.Categories cannot be ordered meaningfullyLeast precise level of measurement
6 Ordinal Level of Measurement Characteristics being measured are orderedRankings such as #1, #2, #3You know that a higher rank is better, but not by how much
7 Interval Level of Measurement Test or tool is based on an underlying continuum that allows you to talk about how much higher one score is than anotherIntervals along the scale are equal to one another
8 Ratio Level of Measurement Characterized by the presence of absolute zero on the scaleAn absence of any of the trait being measured
9 Things to RememberAny outcome can be assigned one of four scales of measurementScales of measures have an orderThe “higher” up the scale of measurement, the more precise the dataMore precise scales contain all of the qualities of the scales below it
10 Classical Test Theory: Os = Ts + E Observed scorethe actual score on a testTrue scoretheoretical reflection of the actual amount of a trait or characteristic an individual possessesError scorepart of the score that is randomReliability = True Score / True Score + Error
11 Types of Reliability Test-Retest Parallel Forms Internal Consistency Measure of StabilityParallel FormsMeasure of EquivalenceInternal ConsistencyMeasure of ConsistencyCronbach’s Alpha (coefficient alpha)Inter-RaterMeasure of Agreement
13 How Big is Big? Reliability coefficients should be positive 0.0 to 1.0General Rules of Thumb…Test-Retest =Inter-Rater = 85% agreementInternal Consistency = .70 – 1.0High Reliability DOES NOT mean quality!!
14 Establishing Reliability Make sure instructions are standardized across all settingsIncrease number of items or observationsDelete unclear itemsModerate easiness or difficulty of testsMinimize the effect of external events
15 What is the Truth? Validity The extent to which inferences made from a test are…AppropriateMeaningfulUseful(American Psychological Association & the National Council on Measurement)Does the test measure what it is supposed to measure?
16 Types of ValidityTraditionally speaking there are three types of validity evidence:Content ValidityCriterion ValidityPredictive Criterion validityConcurrent Criterion validityConstruct Validity
17 Content ValidityProperty of a test such that the test items sample the universe of items for which the test is designed.How to Establish…Content ExpertDo items represent all possible items?How well do the number of items reflect what was taught?
18 Criterion ValidityAssesses whether a test reflects a set of abilities in a current (concurrent) or future (predictive) setting as measured by some other test.Concurrent ValidityHow well does my test correlate with the outcomes of a similar test right now?Predictive ValidityHow well does my test predict performance on a similar measure in the future?
19 Construct ValidityMost interesting…most difficult source of validity to establishConstruct = group of interrelated variables such as...AggressionHopeIntelligenceWant your construct to correlate with related behaviors and not correlate with behaviors that are not related.
21 Validity & Reliability The “Kissing Cousins”A test can be reliable but not validA test cannot be valid unless it is reliable because…“A test cannot do what it is supposed to do (validity) until it does what it is supposed to do consistently (reliability).”
22 Part III Taking Chances for Fun and Profit Chapter 7 Hypotheticals and You: Testing Your Questions
23 What you will learn in Chapter 7 The difference between samples and populationsThe importance of…The null hypothesisThe research hypothesesHow to judge a good hypothesis
24 What is a hypothesis? An “educated guess” Their role is to reflect the general problem statement or question that is driving the researchTranslates the problem or research question into a form that can be tested.
25 Samples and Populations The large group to which you would like to generalize your findingsSampleThe smaller, representative group of the population that is used to do the researchSampling error – a measure of how well a sample represents the population
26 The Null HypothesisStatements that contain two or more things that are equal (unrelated) to one anotherH0 : m1 = m2The starting point and is accepted as true without knowing more informationBenchmark to compare actual outcomes
27 The Research Hypothesis Statement that there is a relationship between two variablesTwo Types…Nondirectional -- H1 : X1 ≠ X2Reflects a difference; direction is not specifiedTwo-tailed testDirectional -- H1 : X1 > X2Reflects a difference; direction is specifiedOne-tailed test
29 Differences Between Null and Research Hypotheses No relationship between variablesRelationship between variablesRefers to the populationRefers to the sampleIndirectly testedDirectly testedWritten using Greek symbolsWritten using Roman symbolsImplied hypothesisExplicit hypothesis
30 What Makes a Good Hypothesis? Stated in a declarative form rather than a questionDefines an expected relationship between variablesReflects the theory or literature on which they are basedBrief and to the pointTestable – includes variables that can be measured