2Learning Objectives Compute by hand and interpret Single sample tIndependent samples tDependent samples tUse SPSS to compute the same tests and interpret the output
3Review 6 Steps for Significance Testing Set alpha (p level).State hypotheses, Null and Alternative.Calculate the test statistic (sample value).Find the critical value of the statistic.State the decision rule.State the conclusion.1. Set Alpha level, probability of Type I error, that is probability that wewill conclude there is a difference when there really is not. Typically setat .05, or 5 chances in 100 to be wrong in this way.2. State hypotheses. Null hypothesis: represents a position that thetreatment has no effect. Alternative hypothesis is that the treatment hasan effect. In the light bulb exampleHo: mu = 1000 hoursH1: mu is not equal to 1000 hours3. Calculate the test statistic. (see next slide for values)4. Determine the critical value of the statistic.5. State the decision rule: e.g., if the statistic computed is greater than thecritical value, then reject the null hypothesis.Conclusion: the result is significant or it is not significant. Write up theresults.
4t-testt –test is about means: distribution and evaluation for group distributionWithdrawn form the normal distributionThe shape of distribution depend on sample size and, the sum of all distributions is a normal distributiont- distribution is based on sample size and vary according to the degrees of freedom
5What is the t -testt test is a useful technique for comparing mean values of two sets of numbers.The comparison will provide you with a statistic for evaluating whether the difference between two means is statistically significant.T test is named after its inventor, William Gosset, who published under the pseudonym of student.t test can be used either :to compare two independent groups (independent-samples t test)to compare observations from two measurement occasions for the same group (paired-samples t test).
6What is the t -testThe null hypothesis states that any difference between the two means is a result to difference in distribution.Remember, both samples drawn randomly form the same population.Comparing the chance of having difference is one group due to difference in distribution.Assuming that both distributions came from the same population, both distribution has to be equal.
7What is the t -test Then, what we intend: “To find the difference due to chance”Logically, The larger the difference in means, the more likely to find a significant t test.But, recall:VariabilityMore (less) variability = less overlap = larger difference2. Sample sizeLarger sample size = less variability (pop) = larger difference
8TypesThe one-sample t test is used to compare a single sample with a population value. For example, a test could be conducted to compare the average salary of nurses within a company with a value that was known to represent the national average for nurses.The independent-sample t test is used to compare two groups' scores on the same variable. For example, it could be used to compare the salaries of nurses and physicians to evaluate whether there is a difference in their salaries.The paired-sample t test is used to compare the means of two variables within a single group. For example, it could be used to see if there is a statistically significant difference between starting salaries and current salaries among the general nurses in an organization.
9Assumption Dependent variable should be continuous (I/R) The groups should be randomly drawn from normally distributed and independent populationse.g. Male X FemaleNurse X PhysicianManager X StaffNO OVER LAP
10Assumption the independent variable is categorical with two levels Distribution for the two independent variables is normalEqual variance (homogeneity of variance)large variation = less likely to have sig t test = accepting null hypothesis (fail to reject) = Type II error = a threat to powerSending an innocent to jail for no significant reason
11Story of power and sample size Power is the probability of rejecting the null hypothesisThe larger the sample size is most probability to be closer to population distributionTherefore, the sample and population distribution will have less variationLess variation the more likely to reject the null hypothesisSo, larger sample size = more power = significant t test
12One Sample Exercise (1) Set alpha. = .05 State hypotheses. Testing whether light bulbs have a life of 1000 hoursSet alpha. = .05State hypotheses.Null hypothesis is H0: = 1000.Alternative hypothesis is H1: 1000.Calculate the test statisticLet’s do the steps:1. Set alpha = If there is no difference, we will be wrong only 5 times in 100.2. State hypotheses. (Null) H0: = (Alternative) H1: We are testing to see if our light bulbs came from a population where average life is 1000 hours.3. Calculate the test statistic.
13Calculating the Single Sample t What is the mean of our sample?8007509409707909808207601000860= 867What is the standard deviation for our sample of light bulbs?SD= 96.73Go over the answers to exercise with them.M = 867SD =SE=t = -4.35Reject H0Bulbs were not drawn from population with 1000 hr life.Any questions?
14Determining Significance Determine the critical value. Look up in the table (Munro, p. 451). Looking for alpha = .05, two tails with df = 10-1 = 9. Table saysState decision rule. If absolute value of sample is greater than critical value, reject null.If |-4.35| > |2.262|, reject H0.4. Determine the critical value of the statistic. We look this up in a table. We need to know alpha (.05, two-tailed) and the degrees of freedom (df). For this test, the df are N-1, in our case 10-1 = 9. According to the table, the critical value is5. State the decision rule: If the absolute value of the test statistic is greater that the critical value, we reject the null hypothesis. In our case, if |-.33| is greater than 2.262, we reject the hypothesis that = This is not the case here.
15Finding Critical Values A portion of the t distribution table
16t Values Critical value decreases if N is increased. Critical value decreases if alpha is increased.Differences between the means will not have to be as large to find sig if N is large or alpha is increased.
17Stating the Conclusion 6. State the conclusion. We reject the null hypothesis that the bulbs were drawn from a population in which the average life is 1000 hrs. The difference between our sample mean (867) and the mean of the population (1000) is SO different that it is unlikely that our sample could have been drawn from a population with an average life of 1000 hours.State the conclusion.Our results suggest that GE’s claim that their light bulbs last 1,000 hours is FALSE. (because we had a sample of 10 GE light bulbs and our sample mean was so far away from 1,000 hours that it is highly unlikely that these bulbs came from a population of bulbs whose mean is really 1,000.) There is a 5% chance that this conclusion is wrong (I.e., we may have gotten a difference this big just by chance factors alone).
18SPSS ResultsOn Brannick’s website, Research Methods, Labs, Lab Presentations. Click on Lab 5 SPSS Examples, the Open. SPSS should run and the data for this lab should appear. In the middle is the column ltbulb. This has the data for the ligtbulb example. In the SPSS data editor, click Analyze, Compare Means, One-Sample T Test. Select ltbulb and put it in the Test Variables box. Type 1000 in the Test Value box. Click OK. You get the output on this slide.Computers print p values rather than critical values. If p (Sig.) is less than .05, it’s significant.
20t-tests with Two Samples Independent Samples t-testDependent Samples t-test
21Independent Samples t-test Used when we have two independent samples, e.g., treatment and control groups.Formula is:Terms in the numerator are the sample means.Term in the denominator is the standard error of the difference between means.Here we have two different samples, and we want to know if they were drawn from populations with two different means. This is equivalent to saying whether a treatment has an effect given a treatment group and a control group. The formula for this t is on the slide.Here t is the test statistic, and the terms in the numerator are the sample and population means. The term in the denominator is SEdiff, which is the standard error of the difference between means.You can see from the subscripts for both t and SE, that we are now dealing with the Sampling Distribution of the DIFFERENCE between the means. This is very similar to the sampling distribution that we created last week. However, what we would do to create a sampling distribution of the differences between the means is rather than selecting 5 scores and computing a mean, we would select 5 pairs of scores, subtract one value from the other, then calculate the mean DIFFERENCE value.If we are doing a study and have two groups, what do we EXPECT that the difference in their mean scores will be?[They should say zero].Thus, the mean of the sampling distribution of the differences between the means is zero.The subscripts are here to tell you which Sampling Distribution we are dealing with (for the Sampling Distribution of Means last week, we had a subscript X-bar. For the sampling distribution of the differences between the means, we have a notation specifying a difference, specifically, the difference between X-bar1 and X-bar2.
22Independent samples t-test The formula for the standard error of the difference in means:Suppose we study the effect of caffeine on a motor test where the task is to keep a the mouse centered on a moving dot. Everyone gets a drink; half get caffeine, half get placebo; nobody knows who got what.Suppose we have two samples taken from the same population. Suppose we compute the mean for each sample and subtract the mean for sample 2 from sample 1. We will get a difference between sample means. If we do this a lot, on average, that difference will be zero. Most of the time it won’t be exactly zero, however. The amount that the difference wanders from zero on average is , the standard error of the difference.
23Independent Sample Data (Data are time off task) Experimental (Caff)Control (No Caffeine)12211418108201611519391315N1=9, M1=9.778, SD1=4.1164N2=10, M2=15.1, SD2=4.2805So let’s say we do the following study. We bring in our volunteers and give each of them a psychomotor test where they use a mouse to keep a dot centered on a computer screen target that keeps moving away (pursuit task). One hour before the test, both groups get an oral dose of a drug. For every other person (1/2 of the people), the drug is caffeine. For the other half, it’s a placebo. Nobody in the study knows who got what. All take the test. The results are in the slide.
24Independent Sample Steps(1) Set alpha. Alpha = .05State Hypotheses.Null is H0: 1 = 2.Alternative is H1: 1 2.1. Set alpha = .05, two tailed (just a difference, not a prediction of greater nor less than).2. Null Hypothesis: . This is the same as . This says that there is nodifference between the drug group and the placebo group in psychomotorperformance in the population. The alternative hypothesis is that thedrug does have an effect, or
25Independent Sample Steps(2) Calculate test statistic:3. Calculate the test statistic (see the slide).
26Independent Sample Steps(2) Calculate test statistic:3. Calculate the test statistic (see the slide).
27Independent Sample Steps (3) Determine the critical value. Alpha is .05, 2 tails, and df = N1+N2-2 or = 17. The value is 2.11.State decision rule. If |-2.758| > 2.11, then reject the null.Conclusion: Reject the null. the population means are different. Caffeine has an effect on the motor pursuit task.4. Determine the critical value of the statistic. We look this up in a table.Alpha is .05, t is 2-tailed and the df are n1+n2-2, or in our case, 17. Thecritical value is5. State the decision rule. If the absolute value of the test statistic is largerthan the critical value, reject the null hypothesis. If |-2.758| > 2.110, rejectthe null.6. Conclusion, the population means are different. The result is significant atp < .05.
28Using SPSS Open SPSS Open file “SPSS Examples” for Lab 5 Go to: “Analyze” then “Compare Means”Choose “Independent samples t-test”Put IV in “grouping variable” and DV in “test variable” box.Define grouping variable numbers.E.g., we labeled the experimental group as “1” in our data set and the control group as “2”Make sure that they look at the data in SPSS to see how the groups were defined and how that relates to the “define groups” task.
29Independent Samples Exercise ExperimentalControl1220141810816Have them work this one. Assume again that this is time off task for the DV.Here are the answers for the independent samples exerciseM1 = 12, M2 = 18SD1 = , SD2 =Std Error = 2t = -6/2 = -3; df = = 7t(.05) = ; 3 >Reject null hypothesisWe conclude that caffeine has an effect.Be sure to cover the relevant areas of the SPSS printout. You should show them where everything that they calculate by hand is on the printout. Also cover the Levene’s test. Explain that if the Levene’s test is significant, we need to use the row that says ‘equal variances NOT assumed”. We do NOT want the Levene’s test to be significant as it violates an assumption of the t-test.Work this problem by hand and with SPSS.You will have to enter the data into SPSS.
32Dependent Samples t-test Used when we have dependent samples – matched, paired or tied somehowRepeated measuresBrother & sister, husband & wifeLeft hand, right hand, etc.Useful to control individual differences. Can result in more powerful test than independent samples t-test.We use this when we have measures on the same people in both conditions (or other dependency in the data). Usually there are individual differences among people that are relatively enduring. For example, suppose we tested the same people on the psychomotor test twice. Some people would be very good at it. Others would be relatively poor at it. The dependent t allows us to take these individual differences into account. The scores on the variable in one treatment will be correlated with the scores on the other treatment.If the observations are positively correlated (most people score either high on both or low on both) and if there is a difference in means, we are more likely to show it with the dependent t-test than with the independent samples t-test. [Emphasize this point, they need to know it for their homework.]
33Dependent Samples t Formulas: t is the difference in means over a standard error.We are still dealing with the Sampling Distribution of the Difference between the means. Our subscript is different here, but says basically the same thing. We are looking at the MEAN DIFFERENCE SCORE. The subscript for the independent samples t said we were looking at the DIFFERENCE BETWEEN THE MEANS.The standard error is found by finding the difference between each pair of observations. The standard deviation of these difference is SDD. Divide SDD by sqrt (number of pairs) to get SEdiff.
34Another way to write the formula In this formula, we just put the formula for Sediff in the denominator instead of having you calculate it separately. [this is the formula that appears on the “Guide to Statistics” sheet they can download.
35Dependent Samples t example PersonPainfree (time in sec)PlaceboDifference16055523520153701045045M487SD13.2316.815.70Suppose that we are testing Painfree, a drug to replace aspirin. Five people are selected to test the drug. On day one, ½ get painfree, and the other get a placebo. Then all put their hands into icewater until it hurts so bad they have to pull their hands from the water. We record how long it takes. The next day, they come back and take the other treatment. (Counterbalancing & double blind.)
36Dependent Samples t Example (2) Set alpha = .05Null hypothesis: H0: 1 = 2. Alternative is H1: 1 2.Calculate the test statistic: 1. Set alpha = .05, two tailed (just a difference, not a prediction of greater or less than).2. Null Hypothesis: This is the same as This says that there is no difference between the pain killer and the placebo in the population. The alternative hypothesis is that the pain killer does have an effect, or3. Calculate the test statistic (see slide).
37Dependent Samples t Example (3) Determine the critical value of t.Alpha =.05, tails=2df = N(pairs)-1 =5-1=4.Critical value is 2.776Decision rule: is absolute value of sample value larger than critical value?Conclusion. Not (quite) significant. Painfree does not have an effect.4. Determine the critical value of the statistic. We look this up in a table. Alpha is .05, t is 2-tailed and our df are N-1, where N is the number of pairs. In this case df = 5-1 = 4. The critical value is5. State the decision rule. If the absolute value of the test statistic is larger than the critical value, reject the null hypothesis. If |2.75| > 2.776, reject the null.6. Conclusion. he population means are not (quite) different. The result is not significant at p < .05.
38Using SPSS for dependent t-test Open SPSSOpen file “SPSS Examples” (same as before)Go to:“Analyze” then “Compare Means”Choose “Paired samples t-test”Choose the two IV conditions you are comparing. Put in “paired variables box.”Point out that the data for an independent t-test and dependent t-test must be entered differently in SPSS.[They should choose “painfree” and “placebo” to put in the paired variables box.]
39Dependent t- SPSS output Go over output. Have them start on their homework or project.
40Relationship between t Statistic and Power To increase power:Increase the difference between the means.Reduce the varianceIncrease NIncrease α from α = .01 to α = .05
41To Increase PowerIncrease alpha, Power for α = .10 is greater than power for α = .05Increase the difference between means.Decrease the sd’s of the groups.Increase N.
42In this example Power (1 - β ) = 70.5% Calculation of PowerFrom Table A.1 Zβ of .54 is 20.5%Power is20.5% + 50% = 70.5%In this example Power (1 - β ) = 70.5%
43Calculation of Sample Size to Produce a Given Power Compute Sample Size N for a Power of .80 at p = 0.05The area of Zβ must be 30% (50% + 30% = 80%) From Table A.1 Zβ = .84If the Mean Difference is 5 and SD is 6 then 22.6 subjects would be required to have a power of .80
44PowerResearch performed with insufficient power may result in a Type II error,Or waste time and money on a study that has little chance of rejecting the null.In power calculation, the values for mean and sd are usually not known beforehand.Either do a PILOT study or use prior research on similar subjects to estimate the mean and sd.
45Independent t-TestFor an Independent t-Test you need a grouping variable to define the groups.In this case the variable Group is defined as1 = Active2 = PassiveUse value labels in SPSS
46Independent t-Test: Defining Variables Be sure to enter value labels.Grouping variable GROUP, the level of measurement is Nominal.
51Independent t-Test: Output Assumptions: Groups have equal variance [F = .513, p =.483, YOU DO NOT WANT THIS TO BE SIGNIFICANT. The groups have equal variance, you have not violated an assumption of t-statistic.Are the groups different?t(18) = .511, p = .615NO DIFFERENCE2.28 is not different from 1.96