Chapter 13 Understanding research results: statistical inference

Inferential statistics Are necessary to give meaning to sample data Population data is harder to acquire Used to determine if we can make statements that the results reflect what would happen if we were to conduct the experiment repeatedly w/ multiple samples Allow us to arrive at conclusions on the basis of sample data

Null and research hypotheses Null hypothesis -- H 0 the population means are equal The observed difference is due to random error  The H 0 is rejected when there is a very low probability that the obtained results could be due to random error. The IV had no effect Research hypothesis -- H 1 The population means are not equal The IV had an effect on the DV If we can determine the null hypothesis is false, we then accept the research hypothesis is true. Statistical significance A very low probability that the obtained results could be due to random error. Results are due to the IV’s effect on the DV

Probability and sampling distributions Probability The likelihood of the occurrence of some event or outcome  What is the likelihood this occurrence will differ in the population Based on specific information Sampling distributions Based on the assumption that the null hypothesis is true When it is highly unlikely the null hypothesis is true (0.05 or 5% chance), the researcher rejects the H 0 and accepts the H 1 Sample size Larger sample sizes are good With more observations, the greater the likelihood of obtaining an accurate estimate of the true population value

The t test t test Commonly used to test if 2 groups significantly differ from each other Does the mean of one group significantly differ from the mean of another group? = group difference / within group variability  Group difference – the difference between the obtained means  Within group variability – the amount of variability around the group mean If the t value has a low probability (a value of 0.05 or less) of occurring, the H 0 is rejected

The t test Degrees of freedom df = the total # of participants in the groups – the number of groups The number of scores free to vary once the means are known One tailed vs. two tailed tests One tailed tests are used if the research hypothesis specified the direction of difference between the groups  H 1 = Group 1 will be > than Group 2 Two tailed tests are used if the research hypothesis did not specify a predicted direction of difference  H 1 = Two groups will differ

The F test F test is a.k.a. the ANOVA (analysis of variance) An extension of the t test Asks if there is a significant difference between 3 or more groups Used to evaluate the results of factorial designs  Used when two or more independent variables are used A ratio of two types of variance  Systematic variance – the deviation of the group means from the mean score of all individuals in all groups Between group variance  Error variance – the deviation of the individual scores in each group from their respective group means Within group variance The larger the F ratio, the more likely it is that the results are significant

The F test Calculating effect size Effect size – the magnitude of the effect Can be in terms of t test or standard deviation Provides information on the size of the relationship between the variable studied Confidence intervals A 95% confidence interval indicates that we are 95% sure that the population value lies within the range As the sample increases the confidence interval narrows  This occurs because larger sample sizes are more likely to reflect the population mean

Type I and II errors Correct decisions Rejecting the null hypothesis and the research hypothesis is true Accept the null hypothesis and the null hypothesis is true Type I error Rejecting the null hypothesis when the null hypothesis is true False positive; false alarm The probability of making a Type I error is deterined by the alpha level, α Type II errors Accepting the null hypothesis when the research hypothesis is true False negative The probability of a type II error is called β

Choosing a significance level Researchers use a.05 or.01 significance level Generally researchers think the consequences of making a Type I error are more serious than a Type II error Interpreting non-significant results Even though significance is not reached does not mean there is no relationship between the variables  Statistical significance does not always mean practical significance To troubleshoot, modify the dependent measure for better reliability and sensitivity If there are small sample sizes, it is harder to do statistical tests Do multiple studies to ensure reliability

Analysis Choosing a sample size: power analysis Power: the probability of correctly rejecting the null hypothesis Power = 1 – p (Type II error) A power between.70 and.90 is usually used Usually done by a stats computer program Significance of a Pearson R correlation coefficient A statistical significance test allows you to decide whether to reject the null hypothesis and conclude that the correlation is greater than 0.0  i.e. perform a t test to compare the obtained coefficient w/ the null hypothesis correlation (0.0) Computer analysis of data MS Excel, SPSS, SAS, Minitab…

Selecting the appropriate significance test One IV – 2 groups only Nominal scale data  Chi square Ordinal scale data  Independent groups: Mann Whitney U  Repeated measures or matched participants: Wilcoxon’s T or the sign test Interval or ratio scale data  Independent groups: t test  Repeated measures or matched participants: t test or RM ANOVA One IV – 3 or more groups Nominal scale data  Chi square

Selecting the appropriate significance test One IV – 3 or more groups Ordinal scale data  Independent groups: Kruskal – Wallace H test  Repeated measures: Friedman T test Interval or ratio scale data  1 way ANOVA Two or more IVs Nominal scale data  Chi square Ordinal scale data  No appropriate test available Interval or ratio scale data  2 way ANOVA