1. PODCAST EFFECTIVENESS AS SCAFFOLDING SUPPORT FOR STUDENTS ENROLLED IN FIRST-SEMESTER GENERAL CHEMISTRY LABORATORIES M. Cynthia B. Powell

2. The Problem: Implementing inquiry- based laboratories Students take an active role in planning experiments Inquiry-based labs can be difficult to implement if students do not have background knowledge and skills Bruck, L. B.; Towns, M. H. Preparing Students to Benefit from Inquiry-Based Activities in the Chemistry Laboratory: Guidelines and Suggestions. J. Chem. Educ., 2009, 86, pp 820-822.

3. DIMENSIONS – Content – Methods – Sequence – Sociology Methods include – Modeling – Coaching – Scaffolding and Fading – Articulation – Reflection – Exploration Content includes –Domain Knowledge –Heuristic Strategies –Control Strategies –Learning Strategies Sequence includes –Global Skills before Local Skills –Increasing Complexity –Increasing Diversity Sociology includes –Situated Learning –Culture of Expert Practice –Intrinsic Motivation –Exploiting Cooperation –Exploiting Competition COGNITIVE APPRENTICESHIP Collins, A., Brown, J.S. & Newman, S.E. Cognitive apprenticeship: Teaching the craft of reading, writing and mathematics. In Knowing, learning and instruction: Essays in honor of Robert Glaser, Resnick, L. B. Ed.; Lawrence Erlbaum Associates: Hillsdale, NJ, 1989. Stewart, K.K.; Lagowski, J.J. Cognitive Apprenticeship and Graduate Chemistry Education. J. Chem. Educ., 2003, 80, 1362.

4. The Plan… Resource files and podcasts that can be accessed on the iPhones as needed to provide modeling, coaching and scaffolding during lab Select two treatment groups: podcast treatment group and a pre-lab lecture treatment group Gather quantitative and qualitative data to compare the effect of the treatments Pilot project to test data collection and data processing methods, TA training and podcasts ACU and UNT IRB approval!

5. Laboratory Resource Files Written laboratory materials MSDS sheets Syllabi, grading rubrics, and safety contract Podcasts in two categories: Chemistry calculations & concepts General laboratory techniques

6. The Fall 2009 sample: ACU General Chemistry Labs Six sections, lead by the same instructor Saturation of mobile devices 4 “podcast treatment” sections, n=81 2 “lecture treatment” sections, n=51 Students research teams of with assigned group roles Curriculum: 1. guided inquiry activity (Q) 2. experiment (E) 3. data processing (P)

7. Demographic Description Fall 2009 semester Podcast treatment n = 81Lecture treatment n = 51 Observed Motivation 40.7% High 40.7% Medium 18.5% Low 39.2% High 39.2% Medium 21.6% Low Classifications75.6% Freshmen 15.8% Sophomores 7.3% Juniors 1.2% Seniors 72.5% Freshmen 15.7% Sophomores 7.8% Juniors 3.9% Seniors Gender43.2% Male 56.8% Female 39.2% Male 60.8% Female Mean ACT25.96 + 3.8425.20 + 3.27 Mean GALT9.32 + 2.009.00 + 2.16 No statistically significant differences

8. Summary of the research design Group Treatment Week 3Week 4Week 5Week 6Week 7Week 8Week 9 Week 10Week 11Week 12 “Podcast” treatment Podcast AccessXXXXXXX Pre-laboratory Lecture XXXX “Lecture” treatment Podcast Access Pre-laboratory LectureXXXXXXXXXX Difference in curriculum? Before experiment information presented in podcast vs. lecture format

9. Research Questions Focus of Research Question 1: Frequency of podcast usage Focus of Research Question 2: Number, types, and topics of scaffolding interactions between students research teams and instructors in two treatment groups Focus of Research Question 3: Performance differences between students in the two treatment groups

10. Research Question 1: Methodology Research Question 1: When relevant chemistry podcasts are available for on- demand access during a general chemistry laboratory taught with an inquiry-based curriculum, how frequently will student research teams access them? – files in a secure file management system – pertinent podcasts loaded at the start of the lab – file usage log to track access events

11. Podcast Usage during the Lab Period Experiment Week Accesses podcastsTotal access events Mean access events per team 3Vernier Gas Pressure Equipment321.33 4Collecting a Gas Sample Using Pipets Simple Statistics 32 39 63 1.33 1.63 2.63 5Filtering Mass Determination Simple Statistics 28 11 8 1.17 0.46 0.33 8Comparing Reactivity of Metals Using Acids Safely 53 39 2.21 1.63 9Using Pipets Titration Techniques 7 46 0.29 1.92 10Filtering Using Acids Safely Simple Statistics 46 4 1.88 0.17 11Planning an Experiment592.46

12. Podcast Usage during the Lab Period Accessed podcastTotal access eventsMean access events per team Simple Statistics753.13 Filtering733.04 Planning an Experiment592.46 Comparing Reactivity of Metals532.21 Using Pipets461.92 Titration Techniques461.92 Using Acids Safely431.79 Vernier Gas Pressure Equipment321.33 Collecting a Gas Sample321.33 Mass Determination110.46

13. Research Question 2: Methodology Research Question 2: What differences are evident in the number, types, and topics of scaffolding interactions between student research teams and instructors in laboratory sections that have access to on-demand podcasts but no pre-laboratory lecture and those who have been instructed using a traditional pre-laboratory lecture? Scaffolding Interaction Categorization Scheme (SICS) Tier 1: Type of interaction Clarifying or Follow-up Tier 2: Topic and Subtopic of interaction Q1 – numerical issue Q2 – ideological issue E3 – tools/equipment E4 – investigative procedure E5 – data (quantitative or qualitative) E6 – safety P7 – claims and evidences P8 – prior knowledge or experiences Tier 2: Topic of Interaction: Q, E, or P Tier 1: Type of Interaction: clarifying or follow-up Tier 3: Subtopic of Interaction: 1-8

14. Inter-rater reliability for teaching assistants Teaching assistants by treatment group NumberTypeTopic (Q, E or P) Subtopic (1-8) Podcast treatment TAs mid-semester end-of-semester 1.00.96.98.96.98.96.82.89 Lecture treatment TAs mid-semester end-of-semester.96.90.96.90.96.95.50.60  all aligned data points (  all aligned data points +  data points not aligned) Ratio of clarifying interactions to follow-up interactions, 15:1

15. Clarifying interactions by experiment week

16. Clarifying interactions per team by treatment block Treatment groupMean clarifying interactions by treatment block Contrasting treatment (Weeks 3, 5, 8, 9, 10, 11) Equivalent treatment (Weeks 6, 7,12) Mixed treatment (Week 4) MSDM Podcast treatment teams (n = 24) 2.942*.6621.942.4853.950 Lecture treatment teams (n = 14) 4.478*.8661.977.6054.210 * Welch’s t-test indicates these values are statistically significantly different at α =.05 level Cohen’s d = ( M t – M c ) / S pooled for contrasting treatment block = 2.18

17. Topic distribution of clarifying interactions Treatment group Mean clarifying interactions by treatment block Contrasting treatment Equivalent treatment Week 4 QEPQEPQEP Podcast treatment teams (n = 24).7091.94*.2921.691.060.503.08.64 Lecture treatment teams (n = 14).9053.07*.5021.711.140.503.07.50 * Welch’s t-test indicates these values are statistically significantly different at α =.05 level Cohen’s d = ( M t – M c ) / S pooled for E interactions in contrasting treatment block = 1.73 c

18. Research Question 3: Methodology Research Question 3: What do the student outcome measures of laboratory report grade average, laboratory quiz average, laboratory final exam grade and laboratory course average indicate about performance differences between students who have access to the on-demand podcasts versus students who have received the same information in a traditional lecture format? Graded laboratory assignments: – team laboratory reports – end-of-lab quizzes – individual student reflections – comprehensive final exam

19. Inter-rater reliability for assignment grading Teaching assistants by treatment group Percent difference between TA and faculty instructor grade assignment QuizzesLaboratory reports with accompanying reflection MSDM Podcast treatment TAs–0.53%2.76%–0.55%1.73% Lecture treatment TAs–0.83%1.87%–0.55%1.84%

20. Mean Values of Outcome Measures Podcast treatment n = 81 Lecture treatment n = 51 Lab Reports91.6 ± 6.7490.61 ± 5.83 Quizzes78.79 ± 11.4975.84 ± 12.33 Lab Final Exam72.38 ± 13.5673.21 ± 11.43 Lab Course Grade87.09 ± 7.9185.92 ± 6.66 No category shows a statistically significant difference at the α =.05 level

21. Demographics and interaction effects? “Highly motivated” Podcast treatment n = 33 Lecture treatment n = 20 Lab Reports95.99 ± 2.7491.80 ± 4.45 Quizzes86.95 ± 6.5679.44 ± 11.00 Lab Final Exam83.24 ± 6.9179.45 ± 10.28 Lab Course Grade93.64* ± 3.1388.72* ± 5.93 * ANOVA and Tukey Post-Hoc tests indicate that these values are statistically significantly different at α =.05 level

22. Conclusions Research Question 1: *The mean number of access events per team was 2.80. *Most frequently accessed podcasts covered calculations or concepts. *Natural “fading” observed in access patterns. Research Question 2: *Fewer clarifying interactions with the instructors for the podcast treatment group during the contrasting treatment weeks, but not during equivalent treatment weeks. (“E” interactions) Research Question 3: *Treatment groups performed comparably on outcome measures. *One observed demographic interaction effect: students with high observed motivation ratings in the podcast treatment group performed at a higher level than highly motivated students in the lecture treatment group.

23. Beyond the quantitative data: other uses for iPhones in the laboratory Ready access to all posted resources Online searching Timers Flashlights Calculators Cameras  Video Cameras

24. Continued research interests…. Continued work on assessment for chemistry laboratory courses and training of teaching assistants Podcasts for upper division biochemistry laboratory with lab practicals to follow student progress Enduring resources for pre-service teachers in General Science course

25. Acknowledgements Dr. Diana Mason and the chemistry education research group Faculty and staff in Departments of Chemistry and Educational Psychology at UNT Colleagues in Chemistry and Biochemistry Department and technical support staff at ACU Mobile Learning Fellowship, ACU (2008-2009, 2009-2010) R. B. Escue Scholarship, UNT