# How to analyze data What do we do with the collected data?

## Presentation on theme: "How to analyze data What do we do with the collected data?"— Presentation transcript:

How to analyze data What do we do with the collected data?
By Yun Jin Rho

Contents 1. Overview of data analysis MasteringAstronomy
2. Case studies - Group comparison (MyITLab) - Causal relationship (MyMathTest) - Item level analysis (MasteringEngineering) 3. Closing comments How to analyze data l 09/24/11

Overview of data analysis
1 How to analyze data l 09/24/11

Raw data (MasteringAstronomy)
ID ADT-Pre Final Grade ADT-Post 1 15 0.863 18 2 3 0.832 8 11 0.685 4 6 0.855 5 0.853 16 10 0.894 7 0.823 0.773 9 0.787 0.238 0.842 13 12 0.841 0.944 14 0.952 0.749 How to analyze data l 09/24/11

Descriptive statistics
N Mean SD Median Min Max Range SE Pre 314 7.812 3.245 7 1 18 17 0.183 Post 291 11.825 3.909 12 2 21 19 0.229 Final 378 0.796 0.145 0.831 0.967 0.007 Improvement from the Pre scores (M = 7.812) to the Post scores (M = ) was observed.  Is this amount of improvement statistically significant? How to analyze data l 09/24/11

Distributions of test scores
How to analyze data l 09/24/11

Hypothesis testing Let’s assume the distributions of test scores are all normal. How can we test if this difference of is statistically significant?  Dependent (paired) t-test Hypotheses - H0: the improvement is not different from 0 (Pre = Post) - Ha: the improvement is significantly different from 0 (Pre ≠ Post) Results 95% Confidence Interval (C.I.) of the mean difference: [3.889, 4.637] t(250) = , p = < 0.05 (significance level, ) Conclusion The improvement of was statistically significant. How to analyze data l 09/24/11

Effect size Then, how big is this improvement or effect? Effect size :
measure of distance between the two different distributions Cohen’s d = 1.36 Conclusion There was a large learning effect. Cohen’s d < 0.3: small effect Cohen’s d  0.5: medium effect Cohen’s d > 0.8: large effect How to analyze data l 09/24/11

Historical / Control without Mastering Experimental with Mastering
What does this large learning effect mean? Was the students’ improvement in their test scores because of using Mastering? Is this improvement solely from using Mastering? Pretest Posttest Reading textbooks, Homework, TA’s help, Quality of teaching, Self-studying… Self-studying, Mastering… Historical / Control without Mastering Experimental with Mastering How to analyze data l 09/24/11

Case studies 2 How to analyze data l 09/24/11

Group comparison MyITLab
2011 1st semester using the labs 2009 2nd semester without the labs UID Report 25 Word 10 Excel 15 Exam 50 Total 100 22.8 13.8 93 23 9.33 12.6 88 23.2 8.67 4.8 85 20.2 10.2 79 14 11.4 16.5 13.2 40 78 17.4 7.2 76 UID Word  10 Report  30 Excel  15 Exam 45 Total 100 8 27 13.5 76 7 24 73 9 12.75 71 23 13.4 70 17.5 13 26 12 68 21.5 11.75 17 67 How to analyze data l 09/24/11

Group comparison With the labs vs. Without the labs
Mean S.D. S.E. Without Labs Word 123 75.58 16.89 1.52 Excel 122 70.23 17.88 1.62 Exam 124 55.77 10.27 0.92 Total 61.37 9.73 0.87 With Labs 228 79.88 20.13 1.33 231 77.82 21.3 1.4 233 64.86 12.41 0.81 69.39 12.09 0.79 How to analyze data l 09/24/11

Path analysis (causal relationship) MyMathTest
IRB_ID PALG MALG PCALC ACTMATH 1109GRADE 1113GRADE 10766 32 A 10909 B 11291 97 16 F 10736 89 64 10747 28 21 W 10749 84 80 10944 13 20 10618 9 26 10902 34 24 10908 87 62 18 11254 54 27 D 11264 10582 93 92 91 29 10595 85 99 81 .. How to analyze data l 09/24/11

Path analysis Causal model
How to analyze data l 09/24/11

Item level analysis MasteringEngineering
Pre-test Student 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Post-test Student 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 How to analyze data l 09/24/11

Item level analysis: Pre-test
How to analyze data l 09/24/11

Item level analysis: Post-test
How to analyze data l 09/24/11

Item level analysis: Posttest - Pretest
How to analyze data l 09/24/11

Summary 3 How to analyze data l 09/24/11

Closing comments To analyze the data for efficacy studies, the most important thing is the study design to collect data. Educational interpretation will be more important than the data analysis result. How to analyze data l 09/24/11

Thank you How to analyze data l 09/24/11

Download ppt "How to analyze data What do we do with the collected data?"

Similar presentations