Stats 95 Experimental Design –Experimental Design & Lady Tasting Tea –Type I and Type II Errors –Null Hypothesis an Research Hypothesis

Lady Tasting Tea How would you design the experiment? What task would you give her? What would be the Independent Variable? Dependent variable? Control condition How many could she guess right by chance? What if she can taste the difference, but she makes mistakes? Do you know for certain she can? Do you know for certain she cannot?

Hypotheses H 0 Null Hypothesis: there is nothing going on, Straw Man, the probability of guessing Tea in Milk is equal to guessing Milk in Tea H 1 Research Hypothesis: something is going on, probability of correct identification is not equal to guessing between Milk in Tea and Tea in Milk.

N S+N Hits (response “yes” on signal trial) Criterion Internal response Probability density Say “yes”Say “no”

N S+N Correct rejects (response “no” on no-signal trial) Criterion Internal response Probability density Say “yes”Say “no”

N S+N Misses (response “no” on signal trial) Criterion Internal response Probability density Say “yes”Say “no”

N S+N False Alarms (response “yes” on no-signal trial) Criterion Internal response Probability density Say “yes”Say “no”

“What Cold Possibly Go Worng?”: Type I and Type II Errors Reality Perception YES (Signal + Noise) NO (Noise) YES (Signal + Noise) Hit False Alarm (Type I) False Positive NO (Noise) Miss (Type II) False Negative Correct Rejection

“What Cold Possibly Go Worng?”: Type I and Type II Errors REALITY of PREGNANCY TEST RESULTS YESNO “PREGNANT” (Reject Null) HIT (Pregnant & “+” on test) FALSE ALARM (Type I) Also called False Positive (Not Pregnant & “+”) “NOT PREGNANT” (Fail to reject the Null) Miss (Type II) Also called False Negative (Pregnant & “-”) Correct Rejection (Not Pregnant & “-”)

The End

Statistics in Correlations & Experiments Correlations measure Relationship –Strength and direction of relatioship Experiments measure the Differences –Statistical significance of the difference

Correlation: Measuring Relationship Sir Francis Galton (Uncle to Darwin –Development of behavioral statistics –Father of Eugenics –Science of fingerprints as unique –Retrospective IQ of 200 –Drove himself mad just to prove you could do it –Invented the pocket

The Science of Explanation Measuring correlation –more-more/less-less –more-less/less-more Correlation coefficient –measure of direction & strength –r = 1 –r = -1 –r = 0

14 Correlation What does correlation coefficient mean?

The Science of Explanation Experiment — 2 critical features (1) Manipulation –independent variable – dependent variable—measured –Control Group Condition (or Variable) –Experimental Group Condition (or Variable) (2) Randomization - controls for a 3 rd variable (you know exists but are not interested in) –versus self-selection

Dependent Variables Without Demand Charcteristics DVs that aren’t subject to biased responses Examples: – Is a painting in a museum popular? There will be increased wear on the carpet near it. – Did a dental flossing lecture work? Students will have cleaner teeth the next day. – Did a safer sex intervention for commercial sex workers work? There will be more condoms discarded in the park they work in.

Variation in IV Causes Variation in DV 1.Cause → Effect: whenever IV occurs, outcome DV should result. Safe sex intervention  Condoms in Park 2. Cause absent → Effect absent No SS intervention  no condoms 3. Cause variation → Effect variation More or better interventions  more condoms in park

Experimental & Control Groups Experimental Condition: Cause is valid –E.g., drug, alcohol Control Condition: cause is invalid –Placebo, juice Essence of experiment is to control conditions beforehand

21 The Science of Observation Validity—able to draw accurate inferences –construct validity: e.g., describing what intelligence is and is not, “construct” refers to the “theory” –predictive validity: over time you find X predicts Y Reliability—same result each time? -Test/Re-Test -Parallel - Inter-Item

Statistical Significance A finding is statistically significant if the data differ from what we would expect from chance alone, if there were, in fact, no actual difference. They may not be significant in the sense of big, important differences, but they occurred with a probability below the critical cutoff value, usually a z-score or p <.05 Reject or Fail to Reject the NULL Hypothesis

Graphing Frequency Discrete: HistogramContinuous: Frequency Polygon

Stem-and-Leaf: Exam 1 & 3 Selection of ranges & bins like Histogram, but usually simpler. These plots represent the scores on an exam given to two different sections for the same course.