Education Research 250:205 Writing Chapter 3

Objectives Subjects Instrumentation Procedures Experimental Design Statistical Analysis  Displaying data  Analyzing data Descriptive statistics Derived scores Inferential statistics  Introduction  Confidence intervals  Comparison of means  Correlation and regression

Introduction Statistical inference: A statistical process using probability and information about a sample to draw conclusions about a population and how likely it is that the conclusion could have been obtained by chance

Distribution of Sample Means Assume you took an infinite number of samples from a population  What would you expect to happen?

Assume a population consists of 4 scores (2, 4, 6, 8) Collect an infinite number of samples (n=2)

Total possible outcomes: 16 p(2) = 1/16 = 6.25%p(3) = 2/16 = 12.5% p(4) = 3/16 = 18.75%p(5) = 4/16 = 25% p(6) = 3/16 = 18.75%p(7) = 2/16 = 12.5% p(8) = 1/16 = 6.25%

Central Limit Theorem The CLT describes ANY sampling distribution in regards to: 1. Shape 2. Central Tendency 3. Variability

Central Limit Theorem: Shape All sampling distributions tend to be normal Sampling distributions are normal when:  The population is normal or,  Sample size (n) is large (>30)

Central Limit Theorem: Central Tendency The average value of all possible sample means is EXACTLY EQUAL to the true population mean

µ = / 4 µ = 5 µ M = / 16 µ M = 80 / 16 = 5

Central Limit Theorem: Variability The standard deviation of all sample means is = SEM/√n Also known as the STANDARD ERROR of the MEAN (SEM)

SEM  Measures how well statistic estimates the parameter  The amount of sampling error that is reasonable to expect by chance Central Limit Theorem: Variability

SEM decreases when:  Population  decreases  Sample size increases Other properties:  When n=1, SEM = population SD  As SEM decreases the sampling distribution “tightens” SEM =  /√n

So What? A sampling distribution is NORMAL and represents ALL POSSIBLE sampling outcomes Therefore PROBABILITY QUESTIONS can be answered about the sample relative to the population

Introduction Two main categories of inferential statistics 1. Parametric 2. Nonparametric

Introduction Parametric or nonparametric? What is the scale of measurement?  Nominal or ordinal  Nonparametric  Interval or ratio  Answer next question Is the distribution normal?  Yes  Parametric  No  Nonparametric

Objectives Subjects Instrumentation Procedures Experimental Design Statistical Analysis  Displaying data  Analyzing data Descriptive statistics Derived scores Inferential statistics  Introduction  Confidence intervals  Comparison of means  Correlation and regression

Confidence Intervals Application: Estimation of an unknown variable that is unable or undesirable to be measured directly Confidence intervals estimate with a certain amount of confidence

Confidence Intervals Components of a confidence interval: 1. The level of confidence -Chosen by researcher -Typically 95% -What does it mean? 2. The estimator (point estimate) 3. The margin of error X% CI = Estimator +/- Margin of error

Confidence Intervals: Example A researcher is interested in the amount of $ budgeted for special education by elementary schools in Iowa Select a random sample from the population and collect appropriate data Results:  The average $ spent was $56,789 (95% CI: $51,111 – 62,467)  The average$ spent was $56,789 +/- 5,678 (95% CI)

Objectives Subjects Instrumentation Procedures Experimental Design Statistical Analysis  Displaying data  Analyzing data Descriptive statistics Derived scores Inferential statistics  Introduction  Confidence intervals  Comparison of means  Correlation and regression

Comparing Means  Hypothesis Tests Compare two means  Compare a mean two a known value  Compare means between groups  Compare means within groups Compare three or more means  Compare means between groups  Compare means within groups Compare means as a function of two or more factors (independent variables)  Factorial designs Compare means of multiple dependent variables  Multivariate designs

Hypothesis Tests – A Step by Step Process Step 1: State the null hypothesis Step 2: Select level of significance Step 3: Sample data Step 4: Choose statistic Step 5: Calculate the statistic Step 6: Interpret the statistic

Step 1: Null Hypothesis Recall  Null hypothesis is a statement of no effect The test statistic either accepts or rejects the H0 Create H0 for following tests:  Are females in Iowa taller than 6 feet?  Do 6 th grade boys score differently than 6 th grade females on math tests?  Does an 8-week reading program affect reading comprehension in 3 rd graders?

Step 1: Null Hypothesis The statistic will “test” the H0 based on data No statistic is perfect  The probability of error always exists There are two types of error:  Type I error  Reject a true H0  Type II error  Accept a false H0

Step 1: Null Hypothesis Researcher Conclusion Accept H0Reject H0 Reality About Test No real difference exists Correct Conclusion Type I error Real difference exists Type II error Correct Conclusion How does one control for Type I and II error?

Hypothesis Tests – A Step by Step Process Step 1: State the null hypothesis Step 2: Select level of significance Step 3: Sample data Step 4: Choose statistic Step 5: Calculate the statistic Step 6: Interpret the statistic

Step 2: Significance Level Level of significance: Criterion that determines acceptance/rejection of H0 Level of significance denoted as alpha (  )  = the probability of a type I error  can range between >0.0 – <1.0 Typical values:  0.10  10% chance of type I error  0.05  5% chance of type I error  0.01  1% chance of type I error

Step 2: Significance Level How to determine  ? Exploratory research: Type I error is acceptable therefore set higher    0.05 – 0.10 When is type I error unacceptable?

Hypothesis Tests – A Step by Step Process Step 1: State the null hypothesis Step 2: Select level of significance Step 3: Sample data Step 4: Choose statistic Step 5: Calculate the statistic Step 6: Interpret the statistic

Step 3: Sample Data Parametric statistics assume that data were randomly sampled from population of interest Generalization is limited to population that was sampled

Hypothesis Tests – A Step by Step Process Step 1: State the null hypothesis Step 2: Select level of significance Step 3: Sample data Step 4: Choose statistic Step 5: Calculate the statistic Step 6: Interpret the statistic

Step 4: Choose the Statistic Parametric or nonparametric?  Scale of measurement and distribution How many means are being compared?  Two, three or more? How are the means being compared?  Between or within group? How many independent variables (factors) are being tested?  Factorial design? How many dependent variables are there?  Multivariate design?

Hypothesis Tests – A Step by Step Process Step 1: State the null hypothesis Step 2: Select level of significance Step 3: Sample data Step 4: Choose statistic Step 5: Calculate the statistic Step 6: Interpret the statistic

Step 5: Calculate the Statistic Recall:  H0  exp. design  statistic  The statistic tests the H0 A test statistic can be considered as a ratio between:  Between variance (difference b/w means)  Within variance (variability w/n means)  Statistic = BV/WV Large test statistics imply that:  The difference between the means is relatively large  The variance within the means is relatively small

Example: Researchers compare IQ scores between 6 th grade boys and girls. Results: Girls (150 +/- 50), boys (75 +/- 50) Between Variance Within Variance Distribution overlap?

Statistic = BV/WV Statistic = Big / Big = small valueStatistic = Small / Small = small value Statistic = Small / Big = small valueStatistic = Big / Small = Big value

Step 5: Calculate the Statistic How does sample size affect the statistic? As sample size increases, the within variance decreases  increases size of test statistic

Hypothesis Tests – A Step by Step Process Step 1: State the null hypothesis Step 2: Select level of significance Step 3: Sample data Step 4: Choose statistic Step 5: Calculate the statistic Step 6: Interpret the statistic

Step 6: Interpret the Statistic Calculation of the test statistic also yields a p-value The p-value is the probability of a type I error The p-value ranges from >0.0 – <1.0 Recall alpha (  )  represents the maximum acceptable probability of type I error therefore...

Step 6: Interpret the Statistic If the p-value >   accept the H0  Probability of type I error is higher than accepted level  Researcher is not “comfortable” stating that any differences are real and not due to chance If the p-value <   reject the H0  Probability of type I error is lower than accepted level  Researcher is “comfortable” stating that any differences are real and not due to chance

Statistical vs. Practical Significance Distinction: 1. Statistical significance: There is an acceptably low chance of a type I error 2. Practical significance: The actual difference between the means are not trivial in their practical applications