Presentation on theme: "How to analyze data What do we do with the collected data? By Yun Jin Rho."— Presentation transcript:
How to analyze data What do we do with the collected data? By Yun Jin Rho
How to analyze data l 09/24/113 Contents 1. Overview of data analysis MasteringAstronomy 2. Case studies - Group comparison (MyITLab) - Causal relationship (MyMathTest) - Item level analysis (MasteringEngineering) 3. Closing comments
How to analyze data l 09/24/114 Overview of data analysis 1
How to analyze data l 09/24/115 Raw data (MasteringAstronomy) IDADT-PreFinal GradeADT-Post ………… …………
How to analyze data l 09/24/116 Descriptive statistics NMeanSDMedianMinMaxRangeSE Pre Post Final Improvement from the Pre scores (M = 7.812) to the Post scores (M = ) was observed. Is this amount of improvement statistically significant?
How to analyze data l 09/24/117 Distributions of test scores
How to analyze data l 09/24/118 Lets assume the distributions of test scores are all normal. How can we test if this difference of is statistically significant? Dependent (paired) t-test Hypotheses - H0: the improvement is not different from 0 (Pre = Post) - Ha: the improvement is significantly different from 0 (Pre Post) Results 95% Confidence Interval (C.I.) of the mean difference: [3.889, 4.637] t(250) = , p = < 0.05 (significance level, ) Conclusion The improvement of was statistically significant. Hypothesis testing
How to analyze data l 09/24/119 Then, how big is this improvement or effect? Effect size : measure of distance between the two different distributions Cohens d = 1.36 Conclusion There was a large learning effect. Effect size Cohens d < 0.3: small effect Cohens d 0.5: medium effect Cohens d > 0.8: large effect
How to analyze data l 09/24/1110 Was the students improvement in their test scores because of using Mastering? Is this improvement solely from using Mastering? PretestPosttest Reading textbooks, Homework, TAs help, Quality of teaching, Self-studying… PretestPosttest Reading textbooks, Homework, TAs help, Quality of teaching, Self-studying, Mastering… Historical / Control without MasteringExperimental with Mastering What does this large learning effect mean?
How to analyze data l 09/24/1111 Case studies 2
How to analyze data l 09/24/1112 Group comparison MyITLab UID Report 25 Word 10 Excel 15 Exam 50 Total ……………… UID Word 10 Report 30 Excel 15 Exam 45 Total ……………… st semester using the labs nd semester without the labs
How to analyze data l 09/24/1113 Group comparison With the labs vs. Without the labs NMeanS.D.S.E. Without Labs Word Excel Exam Total With Labs Word Excel Exam Total
How to analyze data l 09/24/1114 Path analysis (causal relationship) MyMathTest IRB_IDPALGMALGPCALCACTMATH1109GRADE1113GRADE A B F F W W B B B B DD DW A A ………..………
How to analyze data l 09/24/1115 Path analysis Causal model
How to analyze data l 09/24/1117 Item level analysis: Pre-test
How to analyze data l 09/24/1118 Item level analysis: Post-test
How to analyze data l 09/24/1119 Item level analysis: Posttest - Pretest
How to analyze data l 09/24/1120 Summary 3
How to analyze data l 09/24/1121 Closing comments To analyze the data for efficacy studies, the most important thing is the study design to collect data. Educational interpretation will be more important than the data analysis result.