Patterns of online use and improved student performance with repeatable online assessments Ann E. Holmes, Ph.D. Department of Physics, Geology and Astronomy and Jennifer T. Ellis, Ph.D. Walker Teaching Resource Center University of Tennessee at Chattanooga Chattanooga, TN 37403 Ann-Holmes@utc.edu Jennifer-T-Ellis@utc.edu Research Funded by NSF-CCLI Grant #0716290

Patterns of Use

General Usage of Blackboard 52% Content Area contains Lecture outlines, PowerPoint lectures for review, and online quizzes Physical Geology, Fall 2010

Blackboard Access by Date

Blackboard Access by Day of Week Class meets Tuesday & Thursday for 1 hour and 15 minutes Most work is done during the week on Monday-Thursday These data have strong implications for scheduling online class deadlines

Blackboard Access by Hour Most work is done between 8 a.m. – midnight Some work done as late as 3 a.m. Students have 24/7 access to the material These data have strong implications for scheduling online class deadlines

Student Performance

Improved Performance Descriptive Statistics Intro Geol Exam 1 avg Exam 2 avg Exam 3 avg Fall 2010 80.3% 79.4% 80.2% N = 87 N = 86 N = 84 STD = 14.87 STD = 13.12 STD = 13.59 Spring 2010 79.9% 70.8% 73.5% N = 42 STD = 15.71 STD = 22.93 STD = 22.59 Fall 2009 79.2% 72.3% 74.2% N = 75 STD = 14.22 STD = 18.94 STD = 15.23 Spring 2009 74.0% 73.8% 75.3% STD = 14.58 STD = 13.83 Fall 2008 74.8% 71.9% 66.0% N = 115 STD = 15.10 STD = 20.89 STD = 24.73 online quizzes i>Clicker review This is the descriptive statistics, no i>Clicker review

Improved Performance Inferential Statistics Fall 2008 control group Independent Sample T Test Significant Difference Between Fall 2010 (online quizzes) <0.05 Exam 1 Spring 2009 (i>clicker review) (t(87) = -2.821, p = 0.005) Fall 2008 (no i<clicker review) (t(115) = -2.593, p = 0.010) Exam 2 Spring 2010 (i>clicker review) (t(42) = -2.684, p = 0.008) Fall 2009 (i>clicker review) (t(73) = -2.426, p = 0.016) (t(87) = -2.745, p = 0.007) (t(115) = -2.939, p = 0.004) Exam 3 (t(42) = -2.074, p = 0.040) (t(75) = -2.794, p = 0.006) (t(87) = -2.223, p = 0.028) (t(115) = -4.756, p = 0.000) From the ANOVA and the Independent Sample T Test it is safe to conclude that a significant increase in student performance occurred after their exposure to the online tests in comparison to the control group as well as to those students that were exposed to iClicker reviews in class.

Quiz Creation & Examples

Choosing Topics for Quizzes 7 online quizzes ~10 questions each Focused on topics that were confusing, complex, not well-understood in past classes quizzes were designed to give students visual and conceptual opportunities to learn remembering and understanding analysis and application

Topics for Quizzes Lithospheric Plate Names Mineral Group Identification Solidus/Liquidus Igneous melting Sedimentary Depositional Environments Contact Metamorphism and Rock Names Earthquakes Surface and Groundwater

Lithospheric Plate ID Quiz Learning Objective: After completion of this quiz students should be able to recognize plate names. Screen Snapshot From Blackboard Students were given unlimited attempts to correctly identify plate names. Each exam had at least one similar question.

Lithospheric Plate ID Quiz Screen Snapshot From Blackboard

Mineral Group ID by Formula Quiz Learning Objective: Students should be able to identify a mineral group by the anion or anion group Screen Snapshot From Blackboard

Solidus/Liquidus Question QUESTION: Compare the two solidus lines in the diagram.  One is labeled wet solidus line and one is labeled dry solidus line.  Which statement accurately reflects the use of one of these solidus lines? Answer Choices Use the wet solidus line if there is no water present. Use the wet solidus line when water is present. Use the dry solidus if there is water present. Incorrect Feedback Minerals melt at lower temperatures when water is present. The wet solidus represents the appropriate melting curve when water is present, and the dry solidus, when water is not present. Use one or the other, never both.   Learning objective correctly determine the effect of water on melting temperature understand how/when to use lines on graph Solidus/Liquidus Question

Solidus/Liquidus Igneous Quiz Learning Objectives understand how to read a P/T melting graph understand effects of decompression understand effects of increased temperature Answers all solid (crystalline) some melt and some crystals all liquid (melt) Which mineral melts due to increasing temperature? (ID by arrow color) Which mineral or rock melts because of decreasing pressure? (ID by arrow color) What is the state of any mineral or rock that lies in the blue field? What is the state of any mineral or rock that lies in the green field? What is the state of any mineral or rock in the yellow field?

Sedimentary Environments of Deposition Learning objective become adept at recognizing sedimentary environments Screen Snapshot From Blackboard

Earthquake Quiz Match the seismic stations and their seismograms to an appropriate lettered box located on the surface in the cross-section sketch of Earth.  Learning Objective Identify P & S waves on a seismogram Analyze DT length on 3 seismograms Apply knowledge to link DT length and seismic station placement

Earthquake Quiz Which answers correctly match seismic stations (PAN, etc) and their seismograms to an appropriate box located on the surface in the cross-section sketch of Earth?  a.  PAN belongs at B b.  BRG belongs at C c.  RGT belongs at A d.  RGT belongs at C e.  PAN belongs at C f.  PAN belongs at A g.  RGT belongs ad B h.  BRG belongs at B i.  BRG belongs at A

Water Quiz Question Well A penetrates a(n) __________. aquiclude Learning Objective recognize difference between unconfined and confined aquifers Question Well A penetrates a(n) __________. a.  confined aquifer b.  unconfined aquifer

Water Quiz Learning Objectives analyze potential contamination assess which well is less likely to be polluted aquiclude Question Which well, A or B, is likely to be polluted by the factory? A.  neither A nor B is likely to be polluted by the factory B.  only water from well B is likely to be polluted by the factory C.  only water from well A is likely to be polluted by the factory

Conclusion Students conceptual and visual understanding was significantly enhanced via the integration of the online quizzes New formative assessment model required the same level of effort from the instructor to construct concept questions that assess student understanding of the content as used with the i>Clicker method One of the major benefits to students was the ability to access to the material at times that work best for their study habits/preferences (24/7) Increased and repetitive access is interpreted to contribute to enhancing student understanding of physical geology content and improved their overall course performance

Questions? Ann E. Holmes, Ph.D. Ann-Holmes@utc.edu Department of Physics, Geology and Astronomy and Jennifer T. Ellis, Ph.D. Jennifer-T-Ellis@utc.edu Walker Teaching Resource Center University of Tennessee at Chattanooga Chattanooga, TN 37403