1. + Quantitative Analysis: Supporting Concepts EDTEC 690 – Methods of Inquiry Minjuan Wang (based on previous slides)

2. + Agenda Quick review of data Why analysis is necessary – beyond descriptive statistics The Culture data posted on BB Descriptive analysis vs. inferential analysis Review Key Concepts of Descriptive Statistics Inferential analysis concepts Types of tests – parametric and non-parametric What test should I use when? Next steps for your studies We will help you with inferential analysis using SPSS or other 2

3. + Our Special Guests: Types of Analysis Descriptive statistics Correlation Measuring a relationship between studied variables Inferential statistics Inferences from a studied sample to a population Parametric analyses Nonparametric analyses 3

4. + What is measurement? Measurement: process of assigning numbers, according to rules defined by the researcher. The numbers are assigned to events or objects, such as responses to items, or to certain observed behaviors Correspondence between event/objective/behavior and number is defined by the researcher 4

5. + Types of Measurement ScalesMeasurement Scales Nominal Categorization, no implied order (e.g., sex, eye color) Ordinal Involves order of the scores/ratings on some basis (e.g., attitude toward the government) Interval Unit interval is the same across the scale, doesn’t necessarily begin at zero (e.g., time, test score) Ratio Equal unit with a true zero point (e.g., the government expenditures; birth weight in pounds) 5

6. + Inferential statistics Making inferences from samples to populations Making inferences, then conclusions, from the statistics of a sample – that’s inferential statistics In practical terms, this means testing your hypothesis 6

7. + Inferential statistics Inferential tests produce a level of significance Significance level or level of significance ( α - level) is a probability (for example, 0.05) used in making a decision about the hypothesis (i.e., rejecting the null hypothesis); it is called the alpha level Significance level is set prior to commencing the study In education, typically.05 Compare the P value with a. P is probability of chance 7

8. + Assumptions of parametric analyses Scale: Dependent variable is measured on an interval scale (or ratio) – not nominal or ordinal Sample: random sampling & normal distribution Normal distribution is required only if sample size is less than 30. More than 30, the sample is large enough to have a normal distribution. Distribution: When two or more populations are being studied, they have homogeneous variance. This means that the populations have about the same dispersion (SD) in their distributions. Mean can differ. When you cannot meet these assumptions (i.e., you have categorical data)… look to non-parametric analyses… 8

9. + Inferential statistics - parametric t-test (difference between two means) testing the statistical significance of the difference between means from two independent samples, or two sets of scores from the same sample (pre to post) Types: T for 1 (paired samples); and T for 2 (unpaired samples) 9

10. + Concepts behind Inferential Statistics Let’s work on the assumption that we’re measuring knowledge. For EDTEC folks – think Kirkpatrick’s Level II – in other words, mastery of objectives Let’s make our audience diesel technicians who work for dealers of a major auto manufacturer Finally, let’s say we have two treatments: 1. Traditional classroom instruction, with limited exercises 2. Fully hands-on curriculum involving “bugged” trucks and problem solving throughout Drawing conclusions from your data

11. + Diesel Technician Scores Pretest Mean Posttest Mean Gain (difference in means) Traditional55.5594.65+39.1 Hands-on53.4597.76+44.31 Course objectives-based Test of Mastery (percent mastery) 11 Are there any statistically significant differences? T test will tell. Compare P with a.

12. + Inferential statistics - parametric But what happens when you have more than two independent variables? For example, what if there were three types of classes for the diesel technicians? Analysis of variance (ANOVA) Tests the statistical significance when 2 or more independent variables are present Consider: A study on student learning with the presence of: no music slow music fast music 12

13. 13 Does Culture Make a Difference in learner perceptions? The survey: Perceptions about being equal with their instructor Chinese, American, Korean students Tests conducted Kruskal-Wallis Analysis of Variance  Non-parametric version of ANOVA Results and Interpretation P=0.02 comparing with a=0.05 ???

14. 14 O. Perceptions about Being Equal with Instructors (the higher the mean, the lower the equality, 1(SA)-5(D))n Rank su m Mean ra nk American31950.030.65 Chinese15682.545.50 Korean29 1217. 541.98 Kruskal-Wallis statistic7.15 p0.028

15. 15 Parametric versus Nonparametric Parametric –  Characteristic is normally distributed in the population; sample was randomly selected; data is interval or ratio Nonparametric  Use when you have a specialized population, you’ve not randomly selected, or data is ranked or nominal “Cooking” Analogy  steamed versus fried  Streamed broccoli versus baked pumpkin pie

16. + Inferential statistics - nonparametric Scale: Can be used with ordinal and nominal scale data Sample/Distribution: Require few if any assumptions about the population under study Nonparametric tests do not emphasize means; they use frequencies and other statistics to investigate significance

17. + What test should I use? Recognize there are many, many statistical tests… And that ED 690 is not intended as a statistics course. Still, you should be conceptually familiar with these statistical tests

18. + Choosing the appropriate test Relationship between variables About means, and parametric assumptions are met About frequencies, etc., and parametric assumptions are met Correlation Coefficient Chi-square Parametric analyses Nonparametric analyses Chi-square t-tests ANOVA Magnitude of Relationship 18

19. 19 Inferential: Parametric tests T-Test for means  T for 1 (pre- post- tests of 1 group)  T for 2 (compare the mean of 2 groups) Analysis of Variance  ANOVA  Compare differences between 2 or more groups Analysis of Covariance

20. 20 Inferential: Non-parametric Nonparametric Techniques for Quantitative Data Wilcoxon Signed Rank Test –Tea for one (Paired Samples T test, Dependent samples T) The Mann-Whitney U Test—for T(ea) for two (Unpaired Samples T Test, Independent Samples T) The Kruskal-Wallis One Way Analysis of Variance—for ANOVA  1 independent variable The Friedman Two-Way Analysis of Variance—for ANOVA  2 or more independent variables

21. + Inferential statistics - nonparametric The Chi-Square (X2) test and distribution Unlike t-distribution, the X2 distribution does not require symmetrical distributions It tests hypotheses about how well a sample distribution fits some theoretical or hypothesized distribution Is there a relationship between eye and hair color? 21

22. 22 Tails of A Test  Two-tailed test (non-directional/both ) There is no difference in content acquisition between "discovery learning" and "direct instruction.“  One-tailed test (directional/upper/lower) difference will be in one direction only Students who use "discovery learning" exhibit greater gains in content acquisition than students who use "direct instruction"

23. + Type I and Type II errors What if we observe a difference – but none exists in the population? What if we do not find a difference – but it does exist in the population? These situations are called Type I and II errors These errors cannot be eliminated; they can be minimized, but unfortunately, minimizing one type of error will increase the probability of committing the other error 23

24. + Type I and Type II errors Conclusion about null hypothesis from statistical test Accept NullReject Null Truth about null hypothesis in population TrueCorrectType I error Observe difference when none exists FalseType II error Fail to observe difference when one exists Correct 24

25. 25 Mini-Data Activity (time to embark on it?) Salary Data Culture Data When you are not heavily cognitively overloaded…..