 # Significance Testing.  A statistical method that uses sample data to evaluate a hypothesis about a population  1. State a hypothesis  2. Use the hypothesis.

## Presentation on theme: "Significance Testing.  A statistical method that uses sample data to evaluate a hypothesis about a population  1. State a hypothesis  2. Use the hypothesis."— Presentation transcript:

Significance Testing

 A statistical method that uses sample data to evaluate a hypothesis about a population  1. State a hypothesis  2. Use the hypothesis to predict the characteristics the sample should have  3. Obtain a random sample from the population  4. Compare the obtained sample data with the prediction made from the hypothesis

 Suggests that the difference or relationship is systematic and not due to chance  A significance level gives the risk that the effect is due to chance  At p <.05, there is about a 5% (5/100) chance that an effect just happened in a normal distribution and is not related to any other variable

 p <.05 for 2-tailed occurs at z = 1.96

 Difference in size between the sample mean and the original population mean  The variability of scores  The number of scores in the sample

 We make certain assumptions about the impact of the methods used to obtain the data sample  Random sampling ◦ That the sample represents the population  Independent observations ◦ The first measurement has no relationship to the probability of the second measurement  The value of variability is unchanged by the treatment ◦ In most hypothesis-testing situations, we do not have the “original” population variability  A normal distribution

 Researchers can never be sure that their hypothesis is “true” ◦ Sample may not perfectly reflect the population ◦ Other influences (confounds) may cause the results ◦ It just might be one of those few chances  These concerns are lessened every time that a finding is replicated

 The null is rejected, but there really is no effect (Type I error)  Or fail to reject the null (accept null), but there really is an effect (Type II error)

 Never know the truth about the null hypothesis  The likelihood of a Type I error is defined by the level of significance ◦ p <.05 means there is a 5% chance of rejecting the null when the null is true (conclude there is a difference when there is none)  Type II error is related to power and sample size

 Rather than rely on hypothesis testing, it is now typical to include effect sizes ◦ Most hypothesis tests simply state that a finding is UNLIKELY (or not) ◦ Significance does not mean it’s important ◦ Effect sizes give a way to capture the SIZE of an effect and make stronger statements about the relationship ◦ This also sidesteps issues of power  Power is strongly linked to sample size and therefore effects may be underestimated in small samples and overestimated in large samples

 Studies may show significant differences but the differences may not be meaningful ◦ Small differences in large samples ◦ Small differences that come with a large cost  Studies may not show significance but the differences may be meaningful ◦ Small differences in small samples ◦ Small differences that come with a big benefit and little cost

 Inferences made about the population based on a sample  Which test determined by ◦ Continuous vs. categorical variables ◦ Number of variables ◦ Whether variables vary between subjects or within subjects *Your book has a useful chart

 1. State the null and research hypotheses  2. Set the level of risk (usually.05)  3. Select the appropriate test statistic  4. Compute the test statistic  5. Determine the critical value for rejection of the null  6. Determine whether the statistic exceeds the critical value (usually at p <.05)  7&8. If over the critical value, the null hypothesis is unlikely THEREFORE effect must be due to other variable  If not over the critical value, the null is accepted  INTERPRET

 One-sample z-tests are inferential statistics that allow you to compare a mean from a sample to the average of a population.

 1. State hypotheses ◦ Null hypothesis: there is no difference between civic engagement in Texas A&M students and the national average  H 0 : X A&M = μ students ◦ Research hypothesis: there is a difference between civic engagement at A&M and the national average  H 1 : X A&M ≠ μ students

Z = X – μ SEM X = mean of sample μ = population average SEM = standard error of the mean SEM = σ/square root of n σ = standard deviation for the population n = size of the sample

 Civic engagement of A&M students ◦ X = 3.42 organizations, n = 200 ◦ μ = 1.5 organizations, σ = 1.2

 Civic engagement of A&M students ◦ X = 3.42 organizations, n = 200 ◦ μ = 1.5 organizations, σ = 1.2 ◦ SEM = σ/square root of n

 5. Determine the critical value for rejection of the null  We know that a z of 1.96 or larger is significant at p <.05  Remember this is the smallest value of z that corresponds to a p of.05 or less (falls in the “extreme range”  Bigger zs = more likely to be a significant difference

 6. Determine whether the statistic exceeds the critical value ◦ 24 > 1.96 ◦ So it does exceed the critical value ◦ THE NULL IS REJECTED IF OUR STATISTIC IS BIGGER THAN THE CRITICAL VALUE – THAT MEANS THE DIFFERENCE IS SIGNIFICANT AT p <.05!!  7. If not over the critical value, fail to reject the null

 In results ◦ Student participants at Texas A&M University joined more civic organizations per year (M = 3.42) than the national average, z = 24.00, p <.05.

Similar presentations