Presentation is loading. Please wait.

Presentation is loading. Please wait.

Part I: The 3 Ts of Online Assessment: Tools, Techniques, and (Saving) Time Curtis J. Bonk, Professor, Indiana University President, CourseShare

Similar presentations


Presentation on theme: "Part I: The 3 Ts of Online Assessment: Tools, Techniques, and (Saving) Time Curtis J. Bonk, Professor, Indiana University President, CourseShare"— Presentation transcript:

1 Part I: The 3 Ts of Online Assessment: Tools, Techniques, and (Saving) Time Curtis J. Bonk, Professor, Indiana University President, CourseShare http://php.indiana.edu/~cjbonk; cjbonk@indiana.edu Vanessa Paz Dennen, Assistant Professor Florida State University http://www.vanessadennen.comhttp://www.vanessadennen.com, vdennen@fsu.edu

2 Session Objectives Detail online assessment techniques Discuss how to match learning activities with learner assessments Examine instructor time and comfort issues Discuss ways to limit and detect cheating and plagiarism Document online tools and resources for assessment

3 Online Assessment Techniques (with some time-saving tips added in…)

4 Is this motivating? How would you feel? You take an online class. You take an online class. You read some Web pages. You read some Web pages. Maybe you watch some videos or hear some audio clips. Maybe you watch some videos or hear some audio clips. Maybe you ponder some study review questions. Maybe you ponder some study review questions. You take a multiple choice test online. You take a multiple choice test online. You receive an automated score on the test. You receive an automated score on the test. Class is over. Class is over.

5 How about this scenario? You take an online class. You take an online class. You meet your fellow students on the d-board. You meet your fellow students on the d-board. You read some materials. You find and share some materials too. You read some materials. You find and share some materials too. You participate in some discussions of course concepts. You participate in some discussions of course concepts. You take a multiple choice test. You take a multiple choice test. You receive an automated score on the test. You receive an automated score on the test. Class is over. Class is over.

6 Commentary on Scenario 1 No interaction with peers. No interaction with peers. Students dont feel missed if they dont participate. Students dont feel missed if they dont participate. Not clear why course is online (except perhaps for media elements). Not clear why course is online (except perhaps for media elements). Potential for immediate feedback is nice -- but assessment format is limiting. Potential for immediate feedback is nice -- but assessment format is limiting.

7 Commentary on Scenario 2 Interaction with peers is great. Serves as a motivator. Interaction with peers is great. Serves as a motivator. Community is likely to develop. Community is likely to develop. Students will feel involved and important if they share examples and resources. Students will feel involved and important if they share examples and resources. Assessment format may not be well aligned given the activities. Assessment format may not be well aligned given the activities. Class lacks closure in a manner appropriate to the activities. Class lacks closure in a manner appropriate to the activities.

8 Assessment and Learning Course objectives, activities, and assessments should be in alignment Course objectives, activities, and assessments should be in alignment This tends to be an issue in courses regardless of medium. This tends to be an issue in courses regardless of medium. Example: Example: In class students conduct a debate In class students conduct a debate Students are tested on their ability to recall facts Students are tested on their ability to recall facts

9 Mis-aligned Online Learning and Assessment A not-uncommon scenario A not-uncommon scenario Discussion is used as a learning activity Discussion is used as a learning activity Students are required to participate Students are required to participate Participation is noted by how many messages were composed by a student Participation is noted by how many messages were composed by a student But does this method measure learning? But does this method measure learning?

10 Common Online Assessment Complaints Instructor perspective Instructor perspective Theres too much to assess! Theres too much to assess! I dont know what activities to assess! I dont know what activities to assess! I dont know if students really are ready for the test! I dont know if students really are ready for the test! How do I know the student actually did the work/took the test? How do I know the student actually did the work/took the test?

11 Common Online Assessment Complaints Student perspective Student perspective If theyre supposed to discuss, why doesnt that count as part of their grade? If theyre supposed to discuss, why doesnt that count as part of their grade? If theyre just supposed to do something, why does quality matter? If theyre just supposed to do something, why does quality matter? I just got a number, no feedback. I just got a number, no feedback. I didnt get participation feedback. I didnt get participation feedback.

12 The Feedback Issue Students participating in online activities look for feedback to know: Students participating in online activities look for feedback to know: A. the instructor is reading their contributions A. the instructor is reading their contributions B. their participation is valued B. their participation is valued C. their participation is adequate, in terms of quality and quantity C. their participation is adequate, in terms of quality and quantity Feedback need not be individualized to be effective Feedback need not be individualized to be effective Whole class commentary provided on a regular basis was found to be just as satisfactory from the student point of view (Dennen, 2001) Whole class commentary provided on a regular basis was found to be just as satisfactory from the student point of view (Dennen, 2001)

13 The Assessment Issue Often, online activities go unassessed Often, online activities go unassessed Add-on syndrome: Adding an online activity to a previously designed class because it sounds like a good idea Add-on syndrome: Adding an online activity to a previously designed class because it sounds like a good idea

14 The Assessment Issue Students are more likely to participate when then know there is impact on their grade Students are more likely to participate when then know there is impact on their grade Direct impact: graded on participation (quality, quantity or both) Direct impact: graded on participation (quality, quantity or both) Indirect impact: participation should bolster performance on other assessments Indirect impact: participation should bolster performance on other assessments Students quickly become aware if an online activity is not related to assessed learning objectives Students quickly become aware if an online activity is not related to assessed learning objectives

15 The Assessment Issue, Cont. Sometimes the wrong things are assessed Sometimes the wrong things are assessed Examples: Examples: Assessing students online moderation skills when the course topic/learning objectives have nothing to do with online moderation Assessing students online moderation skills when the course topic/learning objectives have nothing to do with online moderation Assessing quantity of participation, but not quality Assessing quantity of participation, but not quality which, granted, is easier, but encourages sloppy message posting rather than thoughtful learning dialogueswhich, granted, is easier, but encourages sloppy message posting rather than thoughtful learning dialogues

16 Blooms Revised Taxonomy A useful tool for checking alignment A useful tool for checking alignment Also great to guide your course design! Also great to guide your course design!

17 To Find Out More… A Taxonomy For Learning Teaching and Assessing By Anderson And Krathwohl

18 Assessment Techniques Options… Formative or Summative Formative or Summative Student-led (Self or Peer) or Teacher-led Student-led (Self or Peer) or Teacher-led Public or Private Public or Private Process or Product Process or Product Other issues to consider… Objective or Interpretive Objective or Interpretive Rubric-based or Wholistic Rubric-based or Wholistic

19 Formative vs. Summative Assessments Formative Alleviate student anxieties re: expectations Alleviate student anxieties re: expectations Seem especially high in online classes Seem especially high in online classes Encourage working toward mastery Encourage working toward mastery Can be informal Can be informalSummative Used for students grade Used for students grade Assumed to be best effort Assumed to be best effort

20 Example: Online Formative Assessment Paper draft discussion forum Paper draft discussion forum Start a discussion forum for papers-in- progress Start a discussion forum for papers-in- progress Have each student start a thread and post elements of their papers as they complete them (e.g., topic, major claims, research sources) Have each student start a thread and post elements of their papers as they complete them (e.g., topic, major claims, research sources) A schedule for each element is usefulA schedule for each element is useful Monitor and provide feedback Monitor and provide feedback

21 Student vs. Teacher Led Teacher-led Traditional assessment Traditional assessment Most often summative Most often summativeStudent-led Students may assess self or peers Students may assess self or peers May be formative or summative May be formative or summative Can greatly relieve instructor burden Can greatly relieve instructor burden Students reinforce concepts through feedback process Students reinforce concepts through feedback process

22 Examples: Online Self- Assessment Self-tests: Use test tool to create self-tests (multiple choice, true false) Self-tests: Use test tool to create self-tests (multiple choice, true false) May wish to track student efforts May wish to track student efforts Can incentivize use (essentially, use as a learning tool) Can incentivize use (essentially, use as a learning tool) Reflection papers: Have students submit brief, focused papers expressing the strengths and assessments of their assignment(s) Reflection papers: Have students submit brief, focused papers expressing the strengths and assessments of their assignment(s)

23 Example: Online Peer Assessment (Formative) Feedback groups: Assign students in groups to provide formative feedback on projects and papers Feedback groups: Assign students in groups to provide formative feedback on projects and papers Often raises quality of assignments Often raises quality of assignments Need a structure with clear deadlines Need a structure with clear deadlines Need prompts and models to guide students Need prompts and models to guide students May wish to assess feedback process/contributions May wish to assess feedback process/contributions

24 Example: Online Peer Assessment (Summative) Conference Presentations: Have students present their work and ask questions/provide feedback to others. Conference Presentations: Have students present their work and ask questions/provide feedback to others. In d-board, have students attach papers to messages; post a message with a synopsis; or attach a powerpoint presentation In d-board, have students attach papers to messages; post a message with a synopsis; or attach a powerpoint presentation Each student/team should have their own thread Each student/team should have their own thread Feedback should occur during a defined period of time. Feedback should occur during a defined period of time. May consider allowing students to rate assignments on certain dimensions May consider allowing students to rate assignments on certain dimensions

25 Public vs. Private Private Work is submitted to the teacher only Work is submitted to the teacher only Entire burden of feedback is on teacher Entire burden of feedback is on teacher Important if assessing at fact level Important if assessing at fact levelPublic Peers can see each others work (either in process or completed) Peers can see each others work (either in process or completed) Peers may comment on each others work Peers may comment on each others work Often increases quality of work submitted Often increases quality of work submitted

26 Process vs. Product Product Product The end deliverable The end deliverable Look for polish, accuracy Look for polish, accuracy Process Process How the student got there Look for thoughtfulness of approach, intent

27 Assessing Process Easy to do Easy to do Many technology tools will archive student work/interactions Many technology tools will archive student work/interactions Students create a document trail in process Students create a document trail in process Helps students develop metacognitive knowledge Helps students develop metacognitive knowledge Instructors structure/model/encourage productive work processes Instructors structure/model/encourage productive work processes Students learn how to manage their own work processes Students learn how to manage their own work processes

28 Why Assess Process? For the instructor … For the instructor … Provides formative feedback on course (e.g., helps gather data about why students have difficulty with product-oriented assessments) Provides formative feedback on course (e.g., helps gather data about why students have difficulty with product-oriented assessments) Clarifies who is doing most work in small group assignments Clarifies who is doing most work in small group assignments Helps prevent cheating Helps prevent cheating

29 Why Assess Process? For the student … For the student … Typically improves the quality of their products Typically improves the quality of their products Helps them develop productive work processes Helps them develop productive work processes Puts on a schedule Puts on a schedule Shows that you care about individual growth Shows that you care about individual growth

30 Assessment Project Cycle From Classroom Assessment Techniques by Angelo & Cross (1993) From Classroom Assessment Techniques by Angelo & Cross (1993) Step 1: Plan Step 1: Plan Choose class Choose class Focus on assessable question Focus on assessable question Design project to answer question Design project to answer question

31 Assessment Project Cycle [2] Step 2: Implement Step 2: Implement Teach target lesson Teach target lesson Collect assessment data Collect assessment data Analyze data Analyze data Step 3: Step 3: Interpret results Interpret results Communicate results Communicate results Evaluate assessment project Evaluate assessment project

32 I. Term Papers How to do online: How to do online: Have students each start their own thread and post topic of interest Have students each start their own thread and post topic of interest Peers and instructors give feedback Peers and instructors give feedback Students post thesis statements, research sources, etc., with iterations of feedback Students post thesis statements, research sources, etc., with iterations of feedback Final paper is posted Final paper is posted

33 Term Paper Assessments Product: the paper Product: the paper Process: quality and timeliness of student work from time when paper is assigned Process: quality and timeliness of student work from time when paper is assigned Process: quality and timeliness of feedback provided to peers Process: quality and timeliness of feedback provided to peers Process: responsiveness to feedback received from instructor and peers Process: responsiveness to feedback received from instructor and peers

34 II. Discussion Assignments 1. Chain of thought Have students develop a solution to a problem Have students develop a solution to a problem Have students indicate what led them to a particular conclusion, method or approach Have students indicate what led them to a particular conclusion, method or approach Can be done in a discussion board Can be done in a discussion board

35 Discussion Assignments 2. Theory to Practice Have students match up theories you are learning about to actual problems Have students match up theories you are learning about to actual problems Present students with problems and have them explain what theories they would use to solve these problems and how they would approach it Present students with problems and have them explain what theories they would use to solve these problems and how they would approach it Debrief the assignment Debrief the assignment

36 Discussion Assignment 3. Synthesizer (i.e., wrapper) Have students take roles being the weekly synthesizer of class discussion Have students take roles being the weekly synthesizer of class discussion Add a meta level in which students narrate their own experiences while reading the weekly discussion Add a meta level in which students narrate their own experiences while reading the weekly discussion Reflect on how life relates to discussion Reflect on how life relates to discussion

37 III. Group Projects Tools used Tools used Chat: brainstorming ideas, making group decisions, regular way to feel connected (should be archived) Chat: brainstorming ideas, making group decisions, regular way to feel connected (should be archived) Discussion board: commenting on drafts Discussion board: commenting on drafts E-mail: quick feedback E-mail: quick feedback File exchange: sharing project files File exchange: sharing project files MS Word: Track changes MS Word: Track changes

38 Group Project Assessments Product: project files that are turned in Product: project files that are turned in Process: online archive demonstrating Process: online archive demonstrating Who contributed what Who contributed what Who provided peer feedback Who provided peer feedback Who worked in a timely manner Who worked in a timely manner How collaborative a group was How collaborative a group was Process: peer ratings Process: peer ratings Process: interim instructor consultations Process: interim instructor consultations

39 Group Project Assignments 1. Work Plans Have students develop a plan of work for their project Have students develop a plan of work for their project Make them outline topic, schedule, resources needed, division of labor and anticipated form of final deliverables Make them outline topic, schedule, resources needed, division of labor and anticipated form of final deliverables At end of project, have students evaluate how well they followed their own plan and how useful it was At end of project, have students evaluate how well they followed their own plan and how useful it was

40 Project Assignments 2. Research Trail Have students document the steps they took in the research process and the results Have students document the steps they took in the research process and the results Ask for a brief reflection on how effective their process was and what they might change the next time Ask for a brief reflection on how effective their process was and what they might change the next time

41 Project Assignments 3. Process Presentations Have students focus on their process as well as their product in class presentations Have students focus on their process as well as their product in class presentations To maintain focus, ask them to share 3 main lessons learned To maintain focus, ask them to share 3 main lessons learned Might ask for some process documents to be shared, like an early draft Might ask for some process documents to be shared, like an early draft

42 Project Assignments 4. Design Journal Have students maintain a journal of all ideas related to their project Have students maintain a journal of all ideas related to their project Encourage sketches, lists, organizational charts, etc. Encourage sketches, lists, organizational charts, etc. Require journals to be turned in with final projects Require journals to be turned in with final projects

43 IV. Reflection Assignments Have students keep a weekly journal of their thoughts on readings and course content AND real-world related instances that they noticed Have students keep a weekly journal of their thoughts on readings and course content AND real-world related instances that they noticed May make these public, with each student having their own discussion thread May make these public, with each student having their own discussion thread

44 Making it Happen Learners need to see that process is valuable: Learners need to see that process is valuable: Model appropriate processes Model appropriate processes Provide students with scaffolding (guide sheets) to structure their processes Provide students with scaffolding (guide sheets) to structure their processes Give students feedback on their process Give students feedback on their process Require students to reflect on their processes Require students to reflect on their processes Grade students on process Grade students on process

45 Vanessas Top Time-Saving Tips Before you assign it, ask yourself can I reasonably assess it? Before you assign it, ask yourself can I reasonably assess it? Rely on students/peers for providing some feedback Rely on students/peers for providing some feedback Let students know what to expect up front Let students know what to expect up front Choose the right tool for the job Choose the right tool for the job Get to know the editing and commenting features of your favorite programs Get to know the editing and commenting features of your favorite programs Use rubrics! Use rubrics! I make them in Word, and then while grading I highlight or bold the section that applies to the students projects. Add a few comments at the bottom = speedy grading! I make them in Word, and then while grading I highlight or bold the section that applies to the students projects. Add a few comments at the bottom = speedy grading!


Download ppt "Part I: The 3 Ts of Online Assessment: Tools, Techniques, and (Saving) Time Curtis J. Bonk, Professor, Indiana University President, CourseShare"

Similar presentations


Ads by Google