1. Handout Three: Review of t-Tests Partitioning of Variance, F-Statistic, & F-Distributions EPSE 592 Experimental Designs and Analysis in Educational Research Instructor: Dr. Amery Wu Handout Three: Review of t-Tests Partitioning of Variance, F-Statistic, & F-Distributions EPSE 592 Experimental Designs and Analysis in Educational Research Instructor: Dr. Amery Wu 1

2. About Analysis of Variance Designs Measurement of the data: quantitative Type of statistical inference: descriptive or inferential Type of Modeling: Within each of the four cells, the statistical model can be summative/descriptive or explanatory/predictive. Analysis of Variance Design Measurement of Data QuantitativeCategorical Type of Inference Descriptive Summative/Descriptive Explanatory/Predictive Inferential Summative/Descriptive Explanatory/Predictive 2

3. 3 Review of “Describing and Explaining Quantitative Data”Brushing up Your SPSS Goals of Today’s Class

4. 4

5. Where We have been & Where We Are Today  So far, this course has reviewed descriptive statistics (model) for the sample data.  Today, we will review the two sets of machinery that will transit our discussion from the sample statistics to inferences about the population parameters.  we will use the one-sample t-test as an example for our review of inferential statistics. Measurement of Data ContinuousCategorical Type of the Inference Descriptive AB Inferential CD 5

6. Sampling Errors  The sampling error is the difference between sample statistic and the population parameter.  Sampling errors are caused by sporadic and unpredictable sources as a result of sampling, i.e., sample to sample variation.  For example, among the randomly sampled participants, one got fired at work, another won the lottery,... just before being surveyed to rate their happiness. 6

7. Problems with Inferring the Population ??? Modeling the unknown population is analogous to putting the jigsaw puzzles together without knowing its real image Sampling distribution Hypothesis testing M= 65.32 μ= 0, 50, 65, or 70? 7

8. 8 Two Statistical Machinery for Inferential Statistics

9.  Inferential statistics takes into account the sampling errors when trying to infer from the statistic of one single sample to the population parameter.  Two essential but esoteric machinery for making inferences about the population based on one singe sample: 1.Sampling Distribution 2.Hypothesis Testing Inferential Statistics 9

10. Sampling Distributions  Consider a very large normal population, assume we repeatedly take 1000 samples of a size of 100 from the population and calculate the sample mean, our “statistic of interest”, for each sample.  Each sample of 100 will yield a somewhat different sample means from the rest. The distribution of the 1000 means is the "sampling distribution of the sample mean”.  A sampling distribution is the (probability) distribution of a statistic of interest under repeated sampling from the population. 10

11. 11  A sampling distribution is a theoretical distribution, against which the sample statistic be tested.  A sampling distribution is a theoretical distribution that takes into account the sampling errors by considering the size of the sample (i.e., the degrees of freedom).  The shape of the sampling distribution depends on the population distribution, the statistic of interest, and the degree of freedom. Sampling Distributions

12.  According to the central limit theorem, even if the population is not normal, the sampling distribution of the sample mean will still be approximately normal provided the sample size is sufficiently large.  Also, the central limit theorem states that given a population distribution with a mean of μ and a SD of σ, as the sample size increases, the mean of “the sampling distribution of the mean” approaches μ, and the SD approaches σ/  N.  Namely, the sampling distribution of the mean, with a sufficient sample size, will distribute normally with a mean of μ and standard deviation of σ/  N.  The standard deviation, σ/  N, of the sampling distribution has a special name - the standard error of the mean. Sampling Distributions of the sample Mean 12

13. Lab Activity: Online Sampling Distribution Simulation Go to Home page of Rice Virtual Lab in Statistics. Choose “Simulation and Demonstrations” on the menu. Choose “Sampling Distribution” and enter. Or directly go to http://onlinestatbook.com/stat_sim/sampling_dist /index.html Read and follow the instructions. 13

14. The Family of t-sampling (Theoretical) Distributions 14

15.  In this class, we used the mean to demonstrate the notion of sampling distribution. Statistics other than the mean have sampling distributions too. Students often equate "sampling distribution" with the “sampling distribution of the mean”. That is an unfortunate mistake.  For instance, the sampling distribution of the median is the distribution that would result if the median instead of the mean were computed for each re-sampling.  It is crucial to understand the notion and mechanism of sampling distribution since almost all inferential statistics (you will learn) are based on reference to appropriate sampling distributions. Side Notes for Sampling Distribution 15

16. Hypothesis Testing (HT)  HT is a statistical mechanism for making inference from the sample statistic to the population parameter by taking into account the “random error” (sample to sample error).  In other words, one uses the data to provide evidence for their hypothesis about the population via the “sampling distribution”. 16

17. Step One: Specify the Hypotheses  Specify the null hypothesis (H 0 ) and the alternative hypothesis (H 1 ). Typically, the null hypothesis is the statement a researcher would like to “reject”, and the alternative hypothesis is the statement a researcher would like to “retain”.  Take the kid’s self-reported injection pain (kidrate) for  example, if a researcher hypothesizes that the pain score would be different from 0, the H 0 : μ kidrate = 0 H 1 : μ kidrate = 0  Note that the hypotheses are specified about the Population parameter NOT the sample statistic. Generally, The Greek letters are used to denote population parameters. 17

18. Step Two: Specify the Significance Level ( α )  The significance level (α) is the criterion chosen for rejecting the H 0. Simply put, the significance level is the maximum Type-1 error (false positive) a researcher is willing to tolerate.  If the p-value (explained in the following two slides) is Less than α, then the p-value i s said to be statistically significant, and the H 0 is rejected..  Typically, the α level is set to be 0.05 or 0.01. The lower the α level, the more the data must diverge from the null hypothesis (i.e., a smaller p-value) to be significant.  Therefore, an α of 0.01 is more conservative than that of 0.05 in rejecting the H 0. 18

19.  The third step is to calculate the “statistic of interest”, which is a sample estimate for the population parameter specified in H o  In our example, the mean of kidrate M kidrate is the statistic of interest, which we base to infer the population mean μ kidrate. Step Three: Calculate the Statistic of Interest 19

20.  Next, one needs to know what is the theoretical distribution of the statistic of interest” This is when the mechanism of sampling distribution comes to play and helps identify the appropriate theoretical distribution for the statistic of interest.  In our example, our “statistic of interest” is the mean. We have learned that the sampling distribution of the mean follows a family of t-distributions distinguished by the sample size (a.k.a., degrees of freedom). In our case, we identified the t-distribution with degrees of freedom 39 (N-1), which has a mean of 0 (under null hypothesis) and SE of 2.74 as our sampling distribution. Step Four: Identify the Sampling Distribution of the Statistic of interest 20

21.  The “test statistic” is the ratio of the “statistic of interest” to the “standard error” of the same statistic. The test statistic is denoted as “t” if the statistic follows a t-distribution, as “F” if the statistic follows an F- distribution, or as  2 if the statistic follows a  2 distribution, etc. Test statistic =  (t, F,  2 … )  The sstandard eerror (SE). A measure of how much the value of the statistic, for a given sample size, may vary from sample to sample taken from the same distribution. In other words, it is the standard deviation of the sampling distribution of the statistic for given a sample size.  The purpose of the calculating the test statistic is to show the location (on the X-axis) of the statistic on the sampling distribution, so that the probability of the statistic can be obtained.  In our example, the sample statistic is the mean = 65.32, the standard error is 2.74, so the “test statistic” t = 65.32-0/2.74= 23.82. Step Five: Calculate the Test Statistic 21

22. Step Six: Obtain the p-value  The p-value is “the probability of getting a value of the test statistic as large as or larger than that observed by chance alone, given that the null hypothesis is true”, e.g., the probability of getting a t value of 23.82 or greater given that the population mean is zero is... One obtains the p-value by examining the location of the sample statistic on the theoretical sampling distribution.  In the time when PCs were rare, the “test statistic” were compared to the critical value that is pre-calculated and listed on a table (for a particular sampling distribution with specific α levels and sample sizes) to see if the p value is less than the α value.  Today, statistical packages such as SPSS for PCs can directly produces a p-value.  In our example, SPSS produced a p-value< 0.001. Alternatively, using the t-table: 23.82 is larger than the critical value of 2.022 of t- distribution with of degrees of freedom 39 and an α =0.05. Hence, p<0.05. 22

23.  The probability obtained in Step six is compared with the significance level chosen in Step 2. If the probability is less than or equal to the significance level, then the null hypothesis is rejected; if the probability is greater than the significance level then the null hypothesis is retained. When the null hypothesis is rejected, the outcome is said to be “Statistically Significant”; when the null hypothesis is not rejected then the outcome is said be "not statistically significant.“  In our example, because p is less than the significance level of 0.05. the null hypothesis μ kidrate = 0 is rejected. We conclude that the population mean is “significantly different” from zero. Step Seven: Conclude 23

24. Lab Activity- One-Sample t-test Using SPSS Using the injection pain data, run a one-sample t-test in SPSS to test whether the population mean of Kid’s self-reported pain is different from zero. Use the following path of the drop-down menu: Analyze  Compare Means  One-Sample T-test (Specify the test value to be 0). Compare the SPSS results to those that we hand-calculated in the 7-step hypothesis testing procedures. 24

25. Lab Activity- Calculating Confidence Interval NMean Std. Deviation Std. Error Mean kidrate 4065.325017.343272.74221 One-Sample Test Test Value = 0 tdf Sig. (2- tailed) Mean Difference 95% Confidence Interval of the Difference LowerUpper kidrate 23.82239.00065.3250059.778470.8716 SPSS Results for One-Sample t-test with  set to 0.05 Using the above information, hand-calculate the 95% confidence interval of the population injection pain, which is between ________ and ________. Compare your answers to the results of SPSS. Note that the critical value for t-distribution of df= 39 at  = 0.05 is 2.0227. 25

26. Lab Activity- Calculating Confidence Interval SPSS Results for One-Sample t-test with  set to 0.01 Using the above information, hand-calculate the 99% confidence interval of the population injection pain, which is between ________ and ________. Compare your answers to the results of SPSS. Note that the critical value for t-distribution of df= 39 at  = 0.01 is 2.7079. NMean Std. Deviation Std. Error Mean kidrate 4065.325017.343272.74221 One-Sample Test Test Value = 0 tdf Sig. (2- tailed) Mean Difference 99% Confidence Interval of the Difference LowerUpper kidrate 23.82239.00065.3250057.899372.7507 26

27. 27 http://davidmlane.com/hyperstat/z_table.ht ml Lab Activity: One- or Two-tailed Hypothesis Testing

28. 28 Partitioning Variance (partitioning the Sum of Squares) F statistic F sampling Distributions Getting Ready for Analysis of Variance

29. 29 One-way ANOVA Partitions the Total Sum of Squares by the IV 123245346855 46 4 2

30. 30 One-way ANOVA Partitions the Total Sum of Squares of the DV by One IV SSt tot = 42 SS w-g SS b-g IV 10 32

31.  Between-subject one way aanalysis of variance (ANOVA) is used to test a hypothesis about differences between means of two or more independent groups.  The t-test can only be used to test difference between two means. When there are more than two means, it is possible to compare all possible pairs of means using multiple t-tests. However, conducting multiple t-tests can lead to severe inflation of the Type-I error rate.  ANOVA can be used to test differences among several means “without” inflating the Type-I error rate.  Data Assumption: Same as independent-samples t-test Between-subject (Independent) One Way Analysis of Variance 31

32. Activity: Hypothesis Testing Using Between-subject) One Way Analysis of Variance 1.Specify the null hypothesis (H 0 ) and the alternative Hypothesis (H 1 ). 2.Specify the significance level (α) 3.Calculate the statistic of interest. 4.Calculate the test statistic 5.Identify the sampling distribution of the statistic of interest 6.Obtain the p-value 7.Conclude 32

33. Within-group Variance  Within-group variance is the variation among the individuals’ raw score from the mean of a particular group (of the independent variable).  For a given data, say with three groups, the total within-group variance is the sum of the within-group variances across the three groups.  The within-group variation is caused by chance. It is the sample to sample variation of the individuals being recruited (i.e., random error or noise), namely, the individual differences due to sampling. 33

34. Between-group variance can be caused by (1) Chance (i.e., random error) and (2) True group difference (e.g., there is a true difference between the mean of the control group and the treatment groups (from the grand mean). Such variation is NOT caused randomly by chance but systematically by the independent variable (i.e., groups). Between-group variance is the variation of the group means from the grand mean Between-group Variance 34

35. Logic behind ANOVA Under the null hypothesis that there is no true mean difference among any of pairs of groups, the between- group variance will be caused ONLY by the random error but NOT the true difference among the groups. Thus, the within-group variance will be equal to the between-group variance because there is no true between- group variance. Hence, the F = 1. B-G Variance = random variance + true between-group variance W-G Variance = random variance random variance + true between-group variance random variance = 35

36. Lab Activity: Calculate the Test Statistic: F Ratio First Step: Total Sum of Squares 36

37. Lab Activity: Calculate the Test Statistic: F Ratio Second Step: Within-group Sum of Squares 37

38. Lab Activity: Calculate the Test Statistic: F Ratio Third Step: Between-group Sum of Squares Between-group Sum of Sqaures GroupRaw scoreDeviation scoreSum of Square yellow M=2 blue M=4 green M=6 Grand M=4 38

39. The family of F-distributions 39

40. F Table 40

41. df- between = P-1 (P is the number of groups) df-within = N-P (N is the overall sample size) df- total = N-1 Calculate the p-value and Conclude 41 Note that the number of units for calculating the Var b-g. Var w-g, and Var tot should be the same and equals to the total sample size. Also note that the F value for an one-way ANOVA would equal to t 2, where t is the test statistic in a independent -sample t test.

42. 42  Note that the number of units involved in the calculation of the Var b-g. Var w-g, and Var tot should be the same and equal to the total sample size.  Also note that the F value for an one-way ANOVA  With two groups would equal to t 2, where t is the test Statistic for an independent -sample t test. Side Notes

43. Research Question Does the dose of the drug treatment have an effect on the patients’ depression level? This data was originally contrived by Dr. Karl L. Wuensch and was modified by the instructor for pedagogy reasons. The independent variable is the daily does of the new drug with 3 levels (control, 10mg, & 20mg) that were randomly prescribed to 3 groups of patients (20 in each group). The dependent variable is the patients’ depression level in quantity measured after two months of the new treatment. SPSS Activity: Run the descriptive statistics separately for each group and report what you’ve observe. Lab Activity: Between-Group One Way ANOVA in SPSS 43

44. Lab Activity: Between-Group One Way ANOVA in SPSS 44

45. Lab Activity: SPSS Output of One Way ANOVA 45