1. The Logic of Statistical Significance & Making Statistical Decisions
Textbook Chapter 11 & more

2. Statistical Significance & Probability
Last Day– using Normal Distribution, standard deviation & z-scores to calculate probabilities that what we observe in the sample falls outside a certain range due to sampling error

3. Elements in logic of statistical significance (p. 273)
Assumptions regarding independence of two variable
Assumptions regarding representativeness of samples drawn using probability sampling techniques
Observed distribution of sample in terms of two variable
Example: Study of a population of 256 people, half male & half female about confort using Internet to make purchases

4. Expected outcomes if Null hypothesis holds (no relationship between sex & comfort)

5. Illustration of a representative sample for previous case

6. Illustration of an unrepresentative sample for previous case (if null hypothesis holds)

7. But what if there IS a relationship between sex & comfort in the population?

8. Logical foundations of tests of statistical significance
How to explain discrepancy between assumptions about
independence of variables in a observed distribution of sample elements
Observed distribution of sample elements
1-sample unrepresentative
2-variables NOT independent

9. Type I (alpha) & Type II (beta) Errors
Situation in the “real world”
What the researcher says
No relationship
Causal relationship
No error
Type II error
Type I error

10. Type 1 & Type II Errors (text p. 278)

11. Use of Type I & II Errors Establishing error risk levels
Conventional .05 level for rejecting null hypothesis
But exceptions (familywise error)
Avoiding Type II errors
Using interval or ratio-levels of measurement (parametric statistics)
Directional hypotheses (one-tailed tests)
Increase sample size (depends on effect size)

12. Effect Size
Degree to which your variables are interdependent in the population
Could be large or small
Variables can be related but with smaller level of significance (.001 as opposed to .05)
Not the same as the strength of the interdependence (small differences can be important)

13. First step: Decide on Error Risk & Sample size
Some researchers have proposed ideal sample sizes for different expectations about effect sizes (Cohen)

14. Second Step: Selecting appropriate statistical test
depends on research methodology
Parametric statistics assume
Probabilistic sampling techniques
Independence of observations (selection one element won’t influence likelihood of selecting another)
Normal distribution of population
Comparison groups reflect population with same variance
Dependent variables measured at interval or ratios level
non-parametric statistics
Usually presume independence but not normal distribution of variables
Nominal or ordinal levels of measurement

15. Measures of Association and Difference
Depends on research questions & operationalization
Measures of association (ex. Correlation coefficients like Pearson’s r)
Measures of difference (ex. Median split procedure)

16. Steps 3 & 4: Computing Test Statistic & Using Tables of Critical Values
Numerical guides to sampling distribution of statistic
Indicate likelihood of observations being due to sampling error

17. Deciding to reject or not reject null hypothesis
If statistical value based on sample observations is at least as large as critical value—null hypothesis can be rejected
5% chance that the null hypothesis might be correct (type 1 error)
If computed value smaller than critical value of table then null hypothesis cannot be rejected (but risk of Type II error)

18. Examples of Five Common Statistical Measures in Textbook
Non-parametric measure of difference (chi-square)
Parametric test of difference between 2 groups (t-test)
Parametric test of difference betwee 3 or more groups that vary on one independent variable (ANOVA)
Factorial ANOVA
Parametric measure of association (Pearson correlation)

19. Example: Chi-Square Goodness of Fit (One-Variable)
Independence (Two-Variable)

20. Chi-Square Test
Evaluates whether observed frequencies for a qualitative variable (or variables) are adequately described by hypothesized or expected frequencies.
Qualitative (or categorical) data is a set of observations where any single observation is a word or code that represents a class or category.

21. Goodness of Fit One-Way Chi-Square
Asks whether the relative frequencies observed in the categories of a sample frequency distribution are in agreement with the relative frequencies hypothesized to be true in the population.

22. Goodness of Fit

23. Goodness of Fit Observed Frequency
The obtained frequency for each category.

24. Goodness of Fit State the research hypothesis.
Is the rat’s behavior random?
State the statistical hypotheses.

25. Goodness of Fit: Expected Outcomes (with Null Hypothesis)
.25
.25
.25
.25
If picked by chance.

26. Goodness of Fit Expected Frequency
The hypothesized frequency for each distribution, given the null hypothesis is true.
Expected proportion multiplied by number of observations.

27. Goodness of Fit Set the decision rule. Degrees of Freedom
Number of Categories -1

28. Goodness of Fit
Set the decision rule.

29. Goodness of Fit Calculate the test statistic.
Subtract the expected frequency from the observed frequency in each cell.
Square this difference for each cell.
Divide each squared difference by the expected frequency of that cell.
Add together the results for all the cells in the table.

30. Goodness of Fit
Calculate the test statistic.

31. Goodness of Fit Decide if your result is significant.
Reject H0, 9.25>7.81
Interpret your results.
The rat’s behavior was not random.

32. Goodness of Fit: An Example
You may have heard, “Stay with your first answer on a multiple-choice test.” Is changing answers more likely to be helpful or harmful? To examine this, Best (1979) studied the responses of 261 students in an introductory psychology course. He recorded the number of right-to-wrong, wrong-to-right, and wrong-to-wrong answer changes for each student. More wrong-to-right changes than right-to-wrong changes were made by 195 of the students, who were thus “helped” by changing answers; 27 students made more right-to-wrong changes than wrong-to-right changes and thus hurt themselves. Using a .05 level of significance, test the hypothesis that the proportions of right-to-wrong and wrong-to-right changes are equal.

33. Goodness of Fit State the research hypothesis.
Should you stay with your first answer on a multiple-choice test?
State the statistical hypotheses.

34. Goodness of Fit Observed Frequency
The obtained frequency for each category.
right-to-wrong wrong-to-right
Observed
Expected Frequency
The hypothesized frequency for each distribution, given the null hypothesis is true.
Expected proportion multiplied by number of observations.
Expected .5*222 = *222 = 111
Note: Total number of observations, not 100.

35. Goodness of Fit
Set the decision rule.

36. Goodness of Fit Calculate the test statistic.
right-to-wrong wrong-to-right
Observed
Expected

37. Goodness of Fit Decide if your result is significant.
Reject H0, >3.84
Interpret your results.
The proportion of right-to-wrong changes and wrong-to-right changes is not equal.

38. Chi-Square Cannot be negative because all discrepancies are squared.
Will be zero only in the unusual event that each observed frequency exactly equals the corresponding expected frequency.
Other things being equal, the larger the discrepancy between the expected frequencies and their corresponding observed frequencies, the larger the observed value of chi-square.
It is not the size of the discrepancy alone that accounts for a contribution to the value of chi-square, but the size of the discrepancy relative to the magnitude of the expected frequency.
The value of chi-square depends on the number of discrepancies involved in its calculation.

39. Chi-Square Test for Independence (Two-Way Chi-Square)
Asks whether observed frequencies reflect the independence of two qualitative variables.
Compares the actual observed frequencies of some phenomenon (in our sample) with the frequencies we would expect if there were no relationship at all between the two variables in the larger (sampled) population.
Two variables are independent if knowledge of the value of one variable provides no information about the value of another variable.

40. Chi-Square Test for Independence: Sex & Auto Choice

41. Chi-Square Test for Independence
Recent studies have found that most teens are knowledgeable about AIDS, yet many continue to practice high-risk sexual behaviors. King and Anderson (1993) asked young people the following question: “If you could have sexual relations with any and all partners of your choosing, as often as you wished, for the next 2 (or 10) years, but at the end of that time period you would die of AIDS, would you make this choice?” A five-point Likert scale was used to assess the subjects’ responses. For the following data, the responses “probably no,” “unsure,” “probably yes”, and “definitely yes” were pooled into the category “other.” Using the .05 level of significance, test for independence.
Definitely No Other
Males
Females

42. Chi-Square Test for Independence
State the research hypothesis.
Is willingness to participate in unprotected sex independent of gender?
State the statistical hypothesis.

43. Chi-Square Test for Independence
To find expected values:
Find column, row, and overall totals.
Definitely No Other Total
Males
Females
Total

44. Chi-Square Test for Independence
To find expected values:
Definitely No Other Total
Males 451 (475.75)
Females
Total

45. Chi-Square Test for Independence
To find expected values:
Definitely No Other Total
Males 451 (475.75)
Females 509 (484.25)
Total

46. Chi-Square Test for Independence
To find expected values:
Definitely No Other Total
Males 451 (475.75) 165 (140.25) 616
Females 509 (484.25)
Total

47. Chi-Square Test for Independence
To find expected values:
Definitely No Other Total
Males 451 (475.75) 165 (140.25) 616
Females 509 (484.25) 118 (142.75) 627
Total

48. Chi-Square Test for Independence
Set the decision rule.
Degrees of Freedom
(number of columns - 1) (number of rows -1)
(c-1)(r-1)
Definitely No Other
Males
Females

49. Chi-Square Test for Independence
Set the decision rule.

50. Chi-Square Test for Independence
Calculate the test statistic.
Definitely No Other Total
Males 451 (475.75) 165 (140.25) 616
Females 509 (484.25) 118 (142.75) 627
Total

51. Chi-Square Test for Independence
Decide if your result is significant.
Reject H0, 11.21>3.84
Interpret your results.
Willingness to engage in unprotected sex and gender are not independent.
Definitely No Other Total
Males 451 (475.75) 165 (140.25) 616
Females 509 (484.25) 118 (142.75) 627
Total

52. An Example: Chi-Square Test for Independence
In one large factory, 100 employees were judged to be highly successful and another 100 marginally successful. All workers were asked, “Which do you find more important to you personally, the money you are able to take home or the satisfaction you feel from doing the job?” In the first group, 49% found the money more important, but in the second group 53% responded that way. Test the null hypothesis that job performance and job motivation are independent using the .01 level of significance.

53. An Example: Chi-Square Test for Independence
State the research hypothesis.
Are job performance and job motivation independent?
State the statistical hypotheses.

54. An Example:
Set the decision rule.

55. An Example: Chi-Square Test for Independence
Calculate the test statistic.
High Success Marginal Success Total
Money 49 (51) 53 (51) 102
Satisfaction 51 (49) 47 (49) 98
Total

56. An Example: Chi-Square Test for Independence
Calculate the test statistic.
High Success Marginal Success Total
Money 49 (51) 53 (51) 102
Satisfaction 51 (49) 47 (49) 98
Total

57. An Example: Chi-Square Test for Independence
Decide if your result is significant.
Retain H0, 0.32<6.64
Interpret your results.
Job performance and job motivation are independent.
High Success Marginal Success Total
Money 49 (51) 53 (51) 102
Satisfaction 51 (49) 47 (49) 98
Total