Kin 304 Inferential Statistics Probability Level for Acceptance Type I and II Errors One and Two-Tailed tests Critical value of the test statistic “Statistics means never having to say you're certain”

Inferential Statistics As the name suggests Inferential Statistics allow us to make inferences about the population, based upon the sample, with a specified degree of confidence

Inferential Statistics The Scientific Method Select a sample representative of the population. The method of sample selection is crucial to this process along with the sample size being large enough to allow appropriate probability testing. Calculate the appropriate test statistic. The test statistic used is determined by the hypothesis being tested and the research design as a whole. Test the Null hypothesis. Compare the calculated test statistic to its critical value at the predetermined level of acceptance.

Inferential Statistics Setting a Probability Level for Acceptance Prior to analysis the researcher must decide upon their level of acceptance. Tests of significance are conducted at pre-selected probability levels, symbolized by p or α. The vast majority of the time the probability level of 0.05, is used. – A p of.05 means that if you reject the null hypothesis, then you expect to find a result of this magnitude by chance only 5 in 100 times. Or conversely, if you carried out the experiment 100 times you would expect to find a result of this magnitude 95 times. You therefore have 95% confidence in your result. A more stringent test would be one where the p = 0.01, which translates to 99% confidence in the result.

Inferential Statistics No Rubber Yard Sticks Either the researcher should pre-select one level of acceptance and stick to it, or do away with a set level of acceptance all together and simply report the exact probability of each test statistic. If for instance, you had calculated a t statistic and it had an associated probability of p = 0.032, you could either say the probability is lower than the pre-set acceptance level of 0.05 therefore a significant difference at the 95% level of confidence or simply talk about as a percentage confidence (96.8%)

Inferential Statistics Significance of Statistical Tests The test statistic is calculated The critical value of the test statistic is determined – based upon sample size and probability acceptance level (found in a table at the back of a stats book or part of the EXCEL stats report, or SPSS output) The calculated test statistics must be greater than the critical value of the test statistic to accept a significant difference or relationship

DegreesProbabilityDegreesProbability of Freedom of Freedom , Table 2-4.2: Critical Values of the Correlation Coefficient

Kin 304 Tests of Differences between Means: t-tests SEM Visual test of differences Independent t-test Paired t-test

t Tests Comparison Is there a difference between two or more groups? Test of difference between means – t-test (only two means, small samples) – ANOVA - Analysis of Variance Multiple means – ANCOVA covariates

t Tests Standard Error of the Mean Describes how confident you are that the mean of the sample is the mean of the population

Visual Test of Significant Difference between Means Overlapping standard error bars therefore no significant difference between means of A and B 1 Standard Error of the Mean Mean A B No overlap of standard error bars therefore a significant difference between means of A and B at about 95% confidence

t Tests Independent t-test Two independent groups compared using an independent T-Test (assuming equal variances) – e.g. Height difference between men and women The t statistic is calculated using the difference between the means in relation to the variance in the two samples A critical value of the t statistic is based upon sample size and probability acceptance level (found in a table at the back of a stats book or part of the EXCEL t-test report, or SPSS output) the calculated t based upon your data must be greater than the critical value of t to accept a significant difference between means at the chosen level of probability

t Tests t statistic quantifies the degree of overlap of the distributions

t Tests standard error of the difference between means The variance of the difference between means is the sum of the two squared standard deviations. The standard error (S.E.) is then estimated by adding the squares of the standard deviations, dividing by the sample size and taking the square root.

t Tests t statistic The t statistic is then calculated as the ratio of the difference between sample means to the standard error of the difference, with the degrees of freedom being equal to n - 2.

t Tests Critical values of t Hypothesis: – There is a difference between means Degrees of Freedom = 2n – 2 t calc > t crit = significant difference

t Tests Paired Comparison Paired t Test – sometimes called t-test for correlated data – “Before and After” Experiments – Bilateral Symmetry – Matched-pairs data

t Tests Paired t-test Hypothesis: – Is the mean of the differences between paired observations significantly different than zero the calculated t statistic is evaluated in the same way as the independent test

9 Subjects All Lose Weight Paired Weight Loss Datan = 9 Weight Before (kg)Weight After (kg)Weight Loss (kg) Mean of differences = +1.13

MS EXCEL t-Test: Independent WRONG ANALYSIS BeforeAfter Mean Variance Observations99 Pooled Variance Hypothesized Mean Difference0 df16 t Stat P(T<=t) one-tail t Critical one-tail P(T<=t) two-tail t Critical two-tail

MS EXCEL t-Test: Paired CORRECT ANALYSIS BeforeAfter Mean Variance Observations99 Pearson Correlation Hypothesized Mean Difference 0 df8 t Stat P(T<=t) one-tail t Critical one-tail P(T<=t) two-tail t Critical two-tail

Kin 304 Tests of Differences between Means: ANOVA – Analysis of Variance One-way ANOVA

Tests of Difference – ANOVA ANOVA – Analysis of Variance Used for analysis of multiple group means Similar to independent t-test, in that the difference between means is evaluated based upon the variance about the means. However multiple t-tests result in an increased chance of type 1 error. F (ratio) statistic is calculated and is evaluated in comparison to the critical value of F (ratio) statistic

Tests of Difference – ANOVA One-way ANOVA One grouping factor – H O : The population means are equal – H A : At least one group mean is different Two or more levels of grouping factor - Exposure = low, medium or high - Age Groups = 7-8, 9-10, 11-12, 13-14

Tests of Difference – ANOVA F (ratio) Statistic The F ratio compares two sources of variability in the scores. The variability among the sample means, called Between Group Variance, is compared with the variability among individual scores within each of the samples, called Within Group Variance.

Tests of Difference – ANOVA Formula for sources of variation

Tests of Difference – ANOVA Anova Summary Table SSdfMSF Between Groups SS(Between)k-1 SS(Between) k-1 MS(Between) MS(Within) Within Groups SS(Within)N-k SS(Within) N-k Total SS(Within) + SS(Between) N-1.

Tests of Difference – ANOVA Assumptions for ANOVA The populations from which the samples were obtained are approximately normally distributed. The samples are independent. The population value for the standard deviation between individuals is the same in each group. If standard deviations are unequal transformation of values may be needed.

CFS Kids 17 – 19 years (Boys) ANOVA Dependent - VO 2 max Grouping Factor - Age (17, 18, 19) No Significant difference between means for VO 2 max (p>0.05)

CFS Kids 17 – 19 years (Girls) ANOVA Dependent - VO 2 max Grouping Factor - Age (17, 18, 19) Significant difference between means for VO 2 max (p<0.05)

Tests of Difference – ANOVA Post Hoc tests Post hoc simply means that the test is a follow- up test done after the original ANOVA is found to be significant. One can do a series of comparisons, one for each two-way comparison of interest. E.g. Scheffe or Tukey’s tests The Scheffe test is very conservative

Scheffe’s – Post Hoc Test Boys – no significant differences, would not run post hoc tests Girls – VO2max for age19 is significantly different than at age17 Girls Boys

Tests of Difference – ANOVA ANOVA – Factorial design Multiple factors Test of differences between means with two or more grouping factors, such that each factor is adjusted for the effect of the other Can evaluate significance of factor effects and interactions between them 2 – way ANOVA: Two factors considered simultaneously

Example: 2 way ANOVA Dependent - VO 2 max Grouping Factors – AGE (17, 18, 19) – SEX (1, 2) Significant difference in VO 2 max (p<0.05) by SEX=Main effect Significant difference in VO 2 max (p<0.05) by AGE=Main effect No Significant Interaction (p<0.05) AGE * SEX

Tests of Difference – ANOVA Analysis of Covariance (ANCOVA) Taking into account a relationship of the dependent with another continuous variable (covariate) in testing the difference between means of one or more factor Tests significance of difference between regression lines

Scatterplot showing correlations between skinfold-adjusted Forearm girth and maximum grip strength for men and women

Use of T tests for difference between means? Men are significantly (p<0.05) bigger than women in skinfold-adjusted forearm girth and grip strength

ANCOVA Dependent – Maximum Grip Strength (GRIPR) Grouping Factor – Sex Covariate – Skinfold-adjusted Forearm Girth (SAFAGR) SAFAGR is a significant Covariate No significant difference between sexes in Grip Strength when adjusted for Covariate (representing muscle size) Therefore one regression line (not two, for each sex) fit the relationship

Tests of Difference – ANOVA 3-way ANOVA For 3-way ANOVA, there will be: - three 2-way interactions (AxB, AxC) (BxC) - one 3-way interaction (AxBxC) If for each interaction (p > 0.05) then use main effects results Typically ANOVA is used only for 3 or less grouping factors

Tests of Difference – ANOVA Repeated Measures ANOVA Repeated measures design – the same variable is measured several times over a period of time for each subject Pre- and post-test scores are the simplest design – use paired t-test Advantage - using fewer experimental units (subjects) and providing a control for differences (effect of variability due to differences between subjects can be eliminated)