Presentation on theme: "Curtis J. Bonk, Professor, Indiana University President, CourseShare "— Presentation transcript:
1 Part I: The 3 T’s of Online Assessment: Tools, Techniques, and (Saving) Time Curtis J. Bonk, Professor, Indiana UniversityPresident, CourseShareVanessa Paz Dennen, Assistant ProfessorFlorida State University
2 Session Objectives Detail online assessment techniques Discuss how to match learning activities with learner assessmentsExamine instructor time and comfort issuesDiscuss ways to limit and detect cheating and plagiarismDocument online tools and resources for assessment
3 Online Assessment Techniques (with some time-savingtips added in…)
4 Is this motivating? How would you feel? You take an online class.You read some Web pages.Maybe you watch some videos or hear some audio clips.Maybe you ponder some study review questions.You take a multiple choice test online.You receive an automated score on the test.Class is over.
5 How about this scenario? You take an online class.You “meet” your fellow students on the d-board.You read some materials. You find and share some materials too.You participate in some discussions of course concepts.You take a multiple choice test.You receive an automated score on the test.Class is over.
6 Commentary on Scenario 1 No interaction with peers.Students don’t feel “missed” if they don’t participate.Not clear why course is online (except perhaps for media elements).Potential for immediate feedback is nice -- but assessment format is limiting.
7 Commentary on Scenario 2 Interaction with peers is great. Serves as a motivator.Community is likely to develop.Students will feel involved and important if they share examples and resources.Assessment format may not be well aligned given the activities.Class lacks closure in a manner appropriate to the activities.
8 Assessment and Learning Course objectives, activities, and assessments should be in alignmentThis tends to be an issue in courses regardless of medium.Example:In class students conduct a debateStudents are tested on their ability to recall facts
9 Mis-aligned Online Learning and Assessment A not-uncommon scenarioDiscussion is used as a learning activityStudents are required to participateParticipation is noted by how many messages were composed by a studentBut does this method measure learning?
10 Common Online Assessment Complaints Instructor perspectiveThere’s too much to assess!I don’t know what activities to assess!I don’t know if students really are ready for the test!How do I know the student actually did the work/took the test?
11 Common Online Assessment Complaints Student perspectiveIf they’re supposed to discuss, why doesn’t that count as part of their grade?If they’re just supposed to do something, why does quality matter?I just got a number, no feedback.I didn’t get participation feedback.
12 The Feedback IssueStudents participating in online activities look for feedback to know:A. the instructor is reading their contributionsB. their participation is valuedC. their participation is adequate, in terms of quality and quantityFeedback need not be individualized to be effectiveWhole class commentary provided on a regular basis was found to be just as satisfactory from the student point of view (Dennen, 2001)
13 The Assessment Issue Often, online activities go unassessed “Add-on” syndrome: Adding an online activity to a previously designed class because it sounds like a good idea
14 The Assessment IssueStudents are more likely to participate when then know there is impact on their gradeDirect impact: graded on participation (quality, quantity or both)Indirect impact: participation should bolster performance on other assessmentsStudents quickly become aware if an online activity is not related to assessed learning objectives
15 The Assessment Issue, Cont. Sometimes the wrong things are assessedExamples:Assessing students’ online moderation skills when the course topic/learning objectives have nothing to do with online moderationAssessing quantity of participation, but not qualitywhich, granted, is easier, but encourages sloppy message posting rather than thoughtful learning dialogues
16 Bloom’s Revised Taxonomy A useful tool for checking alignmentAlso great to guide your course design!
17 To Find Out More… A Taxonomy For Learning Teaching and Assessing By AndersonAnd Krathwohl
18 Assessment Techniques Options…Formative or SummativeStudent-led (Self or Peer) or Teacher-ledPublic or PrivateProcess or ProductOther issues to consider…“Objective” or InterpretiveRubric-based or Wholistic
19 Formative vs. Summative Assessments Alleviate student anxieties re: expectationsSeem especially high in online classesEncourage working toward masteryCan be informalSummativeUsed for student’s gradeAssumed to be “best” effort
20 Example: Online Formative Assessment Paper draft discussion forumStart a discussion forum for papers-in-progressHave each student start a thread and post elements of their papers as they complete them (e.g., topic, major claims, research sources)A schedule for each element is usefulMonitor and provide feedback
21 Student vs. Teacher Led Teacher-led “Traditional” assessment Most often summativeStudent-ledStudents may assess self or peersMay be formative or summativeCan greatly relieve instructor burdenStudents reinforce concepts through feedback process
22 Examples: Online Self-Assessment Self-tests: Use test tool to create self-tests (multiple choice, true false)May wish to track student effortsCan incentivize use (essentially, use as a learning tool)Reflection papers: Have students submit brief, focused papers expressing the strengths and assessments of their assignment(s)
23 Example: Online Peer Assessment (Formative) Feedback groups: Assign students in groups to provide formative feedback on projects and papersOften raises quality of assignmentsNeed a structure with clear deadlinesNeed prompts and models to guide studentsMay wish to assess feedback process/contributions
24 Example: Online Peer Assessment (Summative) Conference Presentations: Have students “present” their work and ask questions/provide feedback to others.In d-board, have students attach papers to messages; post a message with a synopsis; or attach a powerpoint presentationEach student/team should have their own threadFeedback should occur during a defined period of time.May consider allowing students to rate assignments on certain dimensions
25 Public vs. Private Private Work is submitted to the teacher only Entire burden of feedback is on teacherImportant if assessing at fact levelPublicPeers can see each others’ work (either in process or completed)Peers may comment on each others’ workOften increases quality of work submitted
26 Process vs. Product Product Process The end deliverable Look for polish, accuracyProcessHow the student got thereLook for thoughtfulness of approach, intent
27 Assessing Process Easy to do Many technology tools will archive student work/interactionsStudents create a document trail in processHelps students develop metacognitive knowledgeInstructors structure/model/encourage productive work processesStudents learn how to manage their own work processes
28 Why Assess Process? For the instructor … Provides formative feedback on course (e.g., helps gather data about why students have difficulty with product-oriented assessments)Clarifies who is doing most work in small group assignmentsHelps prevent cheating
29 Why Assess Process? For the student … Typically improves the quality of their productsHelps them develop productive work processesPuts on a scheduleShows that you care about individual growth
30 Assessment Project Cycle From Classroom Assessment Techniques by Angelo & Cross (1993)Step 1: PlanChoose classFocus on assessable questionDesign project to answer question
32 I. Term Papers How to do online: Have students each start their own thread and post topic of interestPeers and instructors give feedbackStudents post thesis statements, research sources, etc., with iterations of feedbackFinal paper is posted
33 Term Paper Assessments Product: the paperProcess: quality and timeliness of student work from time when paper is assignedProcess: quality and timeliness of feedback provided to peersProcess: responsiveness to feedback received from instructor and peers
34 II. Discussion Assignments 1. Chain of thoughtHave students develop a solution to a problemHave students indicate what led them to a particular conclusion, method or approachCan be done in a discussion board
35 Discussion Assignments 2. Theory to PracticeHave students match up theories you are learning about to actual problemsPresent students with problems and have them explain what theories they would use to solve these problems and how they would approach itDebrief the assignment
36 Discussion Assignment 3. Synthesizer (i.e., wrapper)Have students take roles being the weekly synthesizer of class discussionAdd a “meta” level in which students narrate their own experiences while reading the weekly discussionReflect on how life relates to discussion
37 III. Group ProjectsTools usedChat: brainstorming ideas, making group decisions, regular way to feel connected (should be archived)Discussion board: commenting on draftsquick feedbackFile exchange: sharing project filesMS Word: Track changes
38 Group Project Assessments Product: project files that are turned inProcess: online archive demonstratingWho contributed whatWho provided peer feedbackWho worked in a timely mannerHow collaborative a group wasProcess: peer ratingsProcess: interim instructor consultations
39 Group Project Assignments 1. Work PlansHave students develop a plan of work for their projectMake them outline topic, schedule, resources needed, division of labor and anticipated form of final deliverablesAt end of project, have students evaluate how well they followed their own plan and how useful it was
40 Project Assignments 2. Research Trail Have students document the steps they took in the research process and the resultsAsk for a brief reflection on how effective their process was and what they might change the next time
41 Project Assignments 3. Process Presentations Have students focus on their process as well as their product in class presentationsTo maintain focus, ask them to share 3 main lessons learnedMight ask for some process documents to be shared, like an early draft
42 Project Assignments 4. Design Journal Have students maintain a journal of all ideas related to their projectEncourage sketches, lists, organizational charts, etc.Require journals to be turned in with final projects
43 IV. Reflection Assignments Have students keep a weekly journal of their thoughts on readings and course content AND real-world related instances that they noticedMay make these public, with each student having their own discussion thread
44 Making it Happen Learners need to see that process is valuable: Model appropriate processesProvide students with scaffolding (guide sheets) to structure their processesGive students feedback on their processRequire students to reflect on their processesGrade students on process
45 Vanessa’s Top Time-Saving Tips Before you assign it, ask yourself “can I reasonably assess it?”Rely on students/peers for providing some feedbackLet students know what to expect up frontChoose the right tool for the jobGet to know the editing and commenting features of your favorite programsUse rubrics!I make them in Word, and then while grading I highlight or bold the section that applies to the student’s projects. Add a few comments at the bottom = speedy grading!