Quantitative Methods 2 MA in Education

Review of Session Types of variables – Continuous – Categorical Single Variable Analysis – Central Tendency (mean, median) – Variation (standard deviation) – Distributions and histograms Bivariate (two variable) analysis – Comparing group means (categorical/continuous) – Cross-tabulations (categorical/categorical) – Scatter plot and correlations (continuous/continuous)

This Week Understanding statistical significance and hypothesis testing Doing quantitative data analysis in SPSS Corpus linguistics

Hypothetical Scenario You are the director of a school and you need to decide whether to adopt a new textbook for English language comprehension. To do so, you conduct a small research project, and randomly split the 100 students into two groups: Group A: Old textbook Group B: Uses the new text book At the end of the term, you give each group a standard English comprehension test, and gets the following results: Discuss: Adopting the new textbook will be expensive. Should you change books? (Hint: Follow your instinct) Group AGroup B N (Students)50 Mean Standard Deviation

Distributions of the Two Results Group A Group B

What if the results had looked like this… Group AGroup B N (Students)50 Mean Standard Deviation How big would the difference have to be for you to change?

What if the results were like this…? Group AGroup B N (Students)50 Mean Standard Deviation1.92.0

Or Like This? Group AGroup B N (Students)504 Mean Standard Deviation ? But you make decisions like this all the time!

Testing Hypotheses As a director, you are testing a hypothesis: – Null Hypothesis: There is no relationship between the variables – Alternate Hypothesis: There is a relationship between the variables How confident can you be in rejecting the null hypothesis and accepting the alternate? – 50% – 95% – 100% (Never!) To decide, you need to look at three different things: – The sample size (how many students) – The difference in means – The standard deviation

Statistical Signifiance Your level of confidence of rejecting the null hypothesis and assuming a relationship exists. The p value is the inverse of this (1 – confidence): – 0.10 = You are 90% sure a relationship exists (10% unsure) – 0.05 = You are 95% sure a relationship exists (5% unsure) – = You are 50% sure a relationship exists (50% unsure)

Sampling and Significance As director, you are making an inference from your sample of students to all students that you will teach in the future. – Sample: The group of measurements that you are working with – Population: The group you wish to generalize about Population Sample

Sampling and Significance Consider the following population of scores: Let’s draw a sample from the population – 112, 115, 95, 98, 70, 102, 83, 96 – Sample Mean: 96.4 – Sample Standard Deviation: 14.6 – This is a fairly representative sample! Mean = 95.5, SD = 16.2

Sampling and Significance What if we weren’t so lucky? Our new sample – 112, 115, 133, 98, 102, 114, 96, 18 – Sample Mean: – Sample Standard Deviation: 12.2 – This sample was a fluke! We could make erroneous interpretations

Sample Means and the Normal Distribution Repeated sample means will follow the normal distribution: This makes it possible to reach conclusions about the probability of a sample mean, eg: – 68% of sample means fall within 1 standard deviations of population mean – 95% of sample means fall within 2 standard deviations of population mean This means if we notice a difference of more than 2 standard deviations, this is not likely to be due to sampling

Significance and Hypothesis Testing Convention in social science is accept a confidence of 95% (p = 0.05) as sufficient to establish a relationship between variables. The presence of a relationship does not entail causality What this means for your research/dissertation: As sample size increases, so does confidence Sample of 20 is often sufficient to obtain significant results, 30 – 40 is safer

High School and Beyond Data Set Small sample from a larger longitudinal study in the USA 200 cases, 7 variables: – female (Categorical: 1 = female, 0 = male) – prog: programme of study/stream (Categorical: 1 = general, 2 = academic, 3=vocational) – ses: socioeconomic study (Categorical: 1=Low, 2=Medium, 3=High) – read: reading test score (Continuous) – write: writing test score (Continuous) – math: math test score (Continuous) – science: science test score (Continuous)