1. HUDM4122 Probability and Statistical Inference April 1, 2015

2. Continuing from last class

3. You Try It 49 students use another curriculum and take pre and post tests The students average a gain of 3 points The students get a standard deviation of 14 Do the students learn from this curriculum? Use a two-tailed Z test to find out

4. 49 students use another curriculum and take pre and post tests The students average a gain of 3 points The students get a standard deviation of 14 Do the students learn from this curriculum?

5. 1.5> 1.96 It is not statistically significant 49 students use another curriculum and take pre and post tests The students average a gain of 3 points The students get a standard deviation of 14 Do the students learn from this curriculum?

6. Questions? Comments?

7. P-value As you’ve probably noticed, most papers don’t just report whether a result is statistically significant, they report a p-value as well

8. P-value The p-value is the smallest value of  For which the test is still statistically significant Or in other words, it’s the probability that you could have seen the result you got, if the null hypothesis was true

9. To compute that probability Compute a Z for your data Take the values –Z and +Z Find the area to the left of the smaller Z on the Z distribution Find the area to the right of the bigger Z on the Z distribution Add those together That’s your p

10. Example Z = -1.96 So take -1.96 and +1.96 Area to the left of Z=-1.96 is 0.025 – See your probability table Area to the right of Z=+1.96 is 1-0.975= 0.025 – See your probability table 0.025+0.025=0.05 So for Z=-1.96, p =0.05

11. You try it Z = 1.53

12. You try it Z = -1.03

13. How you report it “The difference between the curricula was not statistically significant, Z=1.50,p=0.13” “The difference between the curricula was statistically significant, Z=5,p<0.001”

14. Reporting Customarily – p=actual value for p>=0.05 – p<0.05 for 0.01<p<0.05 – p<0.01 for 0.001<p<0.01 – p<0.001 for p<0.001

15. MBB Say This is non-standard; don’t do this You can sometime say “marginally significant” for 0.05<p<0.10; depends on the journal

16. Comments? Questions?

17. Comparing a sample to a single value Let’s say you want to compare a sample to a single value that is not zero Let’s review the example from last time And then you will do one

18. Example from last time A TC professor is studying the grades on an exam taken by 49 students The students get an average (sampled) grade of 72 The students get a standard deviation of 7 Are the students doing statistically significantly better than the C cut-off line of 70?

19. A TC professor is studying the grades on an exam taken by 49 students The students get an average (sampled) grade of 72 The students get a standard deviation of 7 Are the students doing statistically significantly better than the C cut-off line of 70?

20. 2 > 1.96, so it is statistically significant A TC professor is studying the grades on an exam taken by 49 students The students get an average (sampled) grade of 72 The students get a standard deviation of 7 Are the students doing statistically significantly better than the C cut-off line of 70?

21. For Z=2, p = 0.023 + (1-0.977)=0.046 A TC professor is studying the grades on an exam taken by 49 students The students get an average (sampled) grade of 72 The students get a standard deviation of 7 Are the students doing statistically significantly better than the C cut-off line of 70?

22. You try it A fisherman is examining the size of the fish he catches to decide if it’s worth fishing in these here waters If the average catch size is 36” or under, those jerks in Albany will confiscate his catch He catches 64 fish in his first net The fish have an average size of 37” The fish have a standard deviation of 40” Should he fish in these here waters? What’s the p value?

23. A fisherman is examining the size of the fish he catches to decide if it’s worth fishing in these here waters If the average catch size is 36” or under, those jerks in Albany will confiscate his catch He catches 64 fish in his first net The fish have an average size of 37” The fish have a standard deviation of 40” Should he fish in these here waters? What’s the p value?

24. A fisherman is examining the size of the fish he catches to decide if it’s worth fishing in these here waters If the average catch size is 36” or under, those jerks in Albany will confiscate his catch He catches 64 fish in his first net The fish have an average size of 37” The fish have a standard deviation of 40” Should he fish in these here waters? What’s the p value?

25. p = 0.84 A fisherman is examining the size of the fish he catches to decide if it’s worth fishing in these here waters If the average catch size is 36” or under, those jerks in Albany will confiscate his catch He catches 64 fish in his first net The fish have an average size of 37” The fish have a standard deviation of 40” Should he fish in these here waters? What’s the p value?

26. p = 0.84 So he shouldn’t fish in these here waters A fisherman is examining the size of the fish he catches to decide if it’s worth fishing in these here waters If the average catch size is 36” or under, those jerks in Albany will confiscate his catch He catches 64 fish in his first net The fish have an average size of 37” The fish have a standard deviation of 40” Should he fish in these here waters? What’s the p value?

27. Comments? Questions?

28. Two-group Z-test Combines our previously studied analysis to estimate the confidence interval of the difference between two groups And the process for computing statistical significance rather than confidence intervals

29. Two-sample Z-test A statistical test involving the Z distribution Which, yes, means that your samples should each have N>30

30. The test H 0 : The difference between sample means is no different than 0 H a : The difference between sample means is different than 0 Calculate a Z value for the difference between sample means

31. Significance Criterion

32. For example You’re comparing the difference between Reasoning Mind and Reasoning Lime Reasoning Mind: average grade = 72, standard deviation = 6, sample size = 36 Reasoning Lime: average grade = 60, standard deviation = 30, sample size = 36

33. Hypotheses Null hypothesis: There is no difference between Reasoning Mind and Reasoning Lime Alternative hypothesis: There is a difference between Reasoning Mind and Reasoning Lime

34. You’re comparing the difference between Reasoning Mind and Reasoning Lime Reasoning Mind: average grade = 72, standard deviation = 6, sample size = 36 Reasoning Lime: average grade = 60, standard deviation = 30, sample size = 36

35. You’re comparing the difference between Reasoning Mind and Reasoning Lime Reasoning Mind: average grade = 72, standard deviation = 6, sample size = 36 Reasoning Lime: average grade = 60, standard deviation = 30, sample size = 36

36. You try it Our friend the fisherman is fishing in two rivers and wants to know if the fish are bigger in one river than another Salmon River: average size = 42”, standard deviation = 20”, sample size = 100 Hudson River: average grade = 49”, standard deviation = 30”, sample size = 100

37. Questions? Comments?

38. Types of Errors

39. Statistician Terminology Data Miner Terminology

40. “Type I Error” False Positive Rejecting the Null Hypothesis when the Null Hypothesis is true Saying the result is statistically significant when there’s nothing there 

41. “Type II Error” False Negative Accepting the Null Hypothesis when the Null Hypothesis is false Saying the result is not statistically significant when there’s actually something there 

42. In the traditional statistical significance paradigm You control  You are unable to control 

43. Type I or Type II error? Reasoning Mind is better than Reasoning Lime, but your stat test got p=0.13 Dreambox is not better than Bob’s Discount Math Curriculum, but your stat test got p=0.03 Columbia University is better than Columbia College of Hollywood CA, but your stat test got p=0.17

44. Questions? Comments?

45. Upcoming Classes 4/15 Statistical power – HW8 due 4/20 Independent-samples t-test – HW9 due 4/22 Paired-samples t-test 4/23 Special session on SPSS