Assessment in Online Courses: Practical Examples Dr. Roger Von Holzen Ms. Darla Runyon Dr. Phillip Heeler Northwest Missouri State University

2 Heard in the Halls “If we are to be required to assess educational quality and learning by virtue of how long a student sits in a seat, we have focused on the wrong end of the student.” Laura Palmer Noone

3 Heard in the Halls “Every person putting the same amount of time into the same subject matter, regardless of the learner’s previous experience, aptitude for that subject, or ability to learn, no longer makes sense.” William Draves

4 “How do you do online exams?” Question based on notion that online assessment must follow assessment methods used on campus Not necessarily true In an online environment Radical change to the role of the instructor  Shift from the deliverer of content to student mentor Function of assessment techniques must also change*

5 Online Assessment: An Interactive Mentoring Opportunity Need to move beyond the rhetoric that assessment should be utilized as a teaching tool and not as an evaluation mechanism Use quizzes and tests as interactive mentoring opportunities Enable students to evaluate their own progress through the course materials Provide feedback on course content areas that need further enhancement and/or development*

6 Beyond the Rhetoric Quizzes and tests should be viewed as means of promoting learning Open book and extensive testing time Should be only a small component of the overall assessment strategy for the online course*

7 Beyond the Rhetoric Evolution from seat-time/credit hours to outcomes-based education as a measure of learning Acknowledging present reality What matters is whether the student has actually learned something*

8 The New Role of Assessment Assessment techniques should be based on desired learning outcomes Assessment results should be used by students to evaluate progress through course materials Provide the instructor with: evidence of effectiveness of course materials indications of content areas that need further enhancement and/or development*

9 Assessment Strategy Cumulative process Aids in forming student assessment profiles snapshot of student understanding Profile constructed by: building learning outcomes based on critical course content use of applicable assessment methods to determine student’s understanding of learning outcomes*

10 Assessment Strategy Provides guidance to further develop conceptual framework Continuous process (formative) Should guide the student to mastery of the learning outcomes Assessment strategy becomes foundation for developing the instructional design of the online course*

11 Assessment Strategy Learning outcomes: should be measurable through an applicable assessment of that outcome should provide evidence of mastery of the learning outcome through student performance*

12 Learning Outcomes Determine critical course content Discern what the students should know or accomplish based on the critical content  What must the student know in order to function in authentic situations? Decide what evidence is acceptable as proof of knowledge or accomplishment of the learning outcome  Selected student performance must furnish the method of assessment of critical content*

13 Communication of Learning Outcomes Include in syllabus List in course introductory module List for each individual module or unit Convey in related activities and assignments*

14 Assessment Strategy Steps 1. Assist faculty in integrating new assessment techniques and developing overall assessment strategy 2. Administer pre-assessment Provides guidance in the development of appropriate learning activities 3. Present critical content through interactive, instructional concepts and activities*

15 Assessment Strategy Steps 4. Punctuate course with short assessment opportunities Provide student with performance feedback on learning concepts and activities Provide a diverse array of assessment methods to reflect student understanding of the learning outcomes Provide opportunities for relearning and reassessment*

16 Assessment Strategy Steps 5. Develop a post-assessment (summative) Provides evaluation of the overall student performance Indicates ultimate mastery of critical content and ability to incorporate content into appropriate situations*

17 Additional Assessment Results Provide instructor feedback on content delivery methods and techniques Feedback directs the instructional redesign of the course and the instructor’s role*

18 Online Assessment Provides an organized and systematic approach to assessment Digital exam building features  Variety of traditional testing methods are available Multiple Choice True and False Fill-in-the-Blank Multiple Answer Ordering Matching Short Answer/Essay  Options to pool questions and control the delivery of the material*

19 Online Assessment Traditional methods should only be a small component of the overall assessment strategy Learning outcomes should be assessed using applicable assessment techniques Online delivery provides an environment conducive to incorporating a diverse array of assessment techniques strategies that may be employed across a variety of course subject areas*

20 Online Assessment Flexibility of delivery allows for a more student-centered approach to assessment and feedback Proctored exams Some situations may require on-site examinations Expenses and effort involved must be considered Traditional methods should only be a small component of the overall assessment strategy *

21 Issues of Academic Dishonesty Work closely with faculty as they discuss and develop new perspectives on assessment Academic dishonesty and conduct in an online course should be examined as faculty design and develop online assessment strategies*

22 Issues of Academic Dishonesty Academic dishonesty and honor code policies should be clearly stated early in the course Include in the course syllabus*

23 Issues of Academic Dishonesty Policy examples Students are responsible for submitting their own work Students who cooperate on examinations or other work without authorization share the responsibility for violation of academic principles and are subject to disciplinary action*

24 Dealing with Plagiarism Web tracking services Instructional design of course site, assignments and exams Communicate with students*

25 Practical Examples One-Sentence Summary Challenges students to answer the questions "Who does what to whom, when, where, how, and why?" about a given topic, and then to synthesize those answers into a single informative, grammatical, and long summary sentence.*

26 Practical Examples Minute Paper The instructor asks students to submit comments related to the following two questions: "What was the most important thing you learned from this lesson?" and "What important question remains unanswered?" Students then submit their responses by e- mail or in a threaded discussion*

27 Practical Examples Punctuated Lectures Requires students to go through five steps: listen, stop, reflect, write, and give feedback  Students begin by listening/viewing a lecture or demonstration.  Then, after a portion of the presentation has been completed, it is stopped.  The students are asked to reflect on the lecture or demonstration.  They then write down any insights they have gained.  Finally, they submit feedback to the instructor in the form of short notes.*

28 Practical Examples Concept Maps Drawings or diagrams showing the mental connections that students make between a major concept the instructor focuses on and other concepts they have learned. Students are asked to sketch the important features of the geography around major concepts such as democracy, racism, art, or free trade.*

29 Practical Examples Paper or Project Prospectus Paper Prospectus--a brief, structured first- draft plan for a term paper or term project.  Prompts students to think through elements of the assignment, such as the topic, purpose, intended audience, major questions to be answered, basic organization, and time and resources required. Project Prospectus--focuses on the tasks to be accomplished, skills to be improved, and products to be developed.*

30 Practical Examples Analytic Memos Requires students to write a one- or two- page analysis of a specific problem or issue The person for whom the memo is being written is usually identified as an employer, a client, or a stakeholder who needs the student's analysis to inform decision making*

31 Practical Examples Electronic Mail Feedback The instructor poses a question to the class, via , about his or her teaching, and invites student responses Student respond to the question with a personal message sent to the instructor's account*

32 Practical Examples Exam Evaluations Allow instructors to examine both what students think they are learning from exams and tests and students' evaluations of the fairness, appropriateness, usefulness, and quality of tests or exams May help provide verification as to the authorship of exam answers*

33 References Angelo, T, & Cross, P. Classroom Assessment Techniques: A Handbook for College Teachers (Second Edition) Boettcher, J, & Conrad, R. Faculty Guide for Moving Teaching and Learning to the Web Draves, W. Teaching Online Kaczmarczyk, L. Accreditation and Student Assessment in Distance Education: Why We All Need to Pay Attention. Proceedings of the 6th Annual Conference on Innovation and Technology in Computer Science Education

34 Dr. Roger Von Holzen, Director Center for Information Technology in Education Ms. Darla Runyon Assistant Director/Curriculum Design Specialist Center for Information Technology in Education Dr. Phillip Heeler, Chairman Computer Science/Information Systems Department