Presentation on theme: "1 Effective Feedback to the Instructor from Online Homework Michigan State University Mark Urban-Lurain Gerd Kortemeyer."— Presentation transcript:
1 Effective Feedback to the Instructor from Online Homework Michigan State University www.lon-capa.org Mark Urban-Lurain firstname.lastname@example.org Gerd Kortemeyer email@example.com Copyright Gerd Kortemeyer & Mark Urban-Lurain, 2005. This work is the intellectual property of the authors. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.
2 Abstract LON-CAPA allows instructors to efficiently create and distribute educational materials, assignments, assessments, etc. These include numerous types of formative conceptual and algorithmic exercises for which prompt feedback and assistance can be provided to students as they work on assigned tasks. LON-CAPA records and dynamically organizes a vast amount of information on students' interaction with and understanding of these materials LON-CAPA helps faculty interpret this data to identify students' misconceptions and other areas of difficulty, so that timely corrective action can be taken. This information also facilitates detailed studies of the educational resources help faculty redesign the instructional materials and courses.
4 Excerpt of homework performance –Students in the rows –Problems in the columns –Each character is one online problem for one student Number of attempts each for student to get that particular problem correct –* means more than nine attempts –. unsolved problem, blank an un-attempted problem. Useful before problem deadline –Columns with large number of dots or blank spaces indicate problems students have difficulties with
6 Student thought that the force would increase in the presence of a medium between the charges First answer: force would double –No feedback except "incorrect" was provided by the system Next attempt: changed answer on only this statement to "four times the force" Only ten seconds passed between the attempts, indicating guessing by which factor the force increased
8 Course-wide per-problem view Graphs on left: statements answered correctly the first and on the second attempt Graphs on right: which options students chose if the statement was answered incorrectly. Students have most difficulty with concept of how a medium acts between charges –Majority believe force would increase –20% believe medium has no influence Instructors can address these misconceptions in class
10 Difficulty Ratio of number of submissions to number of correct responses for 17 problems –5 days before assignment due (15% of the 400 students) Five questions are challenging –Each required more than four submissions per success on average. Instructor can adapt lectures and recitations to meet learner needs
12 Concept Analysis Problems highly randomized among students Internally, options grouped into six "concept groups," each focusing on a particular physics aspect of the problem. Every student gets one statement (with the correct labels filled in) from each one of these concept groups
16 Concept Group Second and Third Submissions Examining submission records indicates concept 5 is easiest, across all three submissions Concept 3 is most difficult Consistent regardless of the number of submission attempts Instructor should address concept 3
17 Improved Learning LON-CAPA provides detailed feedback on learner performance on online homework problems LON-CAPA allows faculty to quickly and efficiently analyze this data in several ways Faculty can quickly adjust lectures, recitation sessions and individual help to address learner needs
18 Acknowledgements National Science Foundation (ITR-0085921; CCLI- ASA-0243126). Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. Alfred P. Sloan and Andrew W. Mellon Foundations Michigan State University, and its administrators for over a decade of encouragement and support