Teaching Spotlight Workshop: Effective Exams Brandon Campitelli, Sarah EichHorn, Keely Finkelstein, Jennifer Fritz, Kristin Patterson

Workshop Agenda Explore the rationale for giving exams. Define hallmarks of an effective exam. Assessing student mastery of learning outcomes. Accuracy of exam scope and emphasis. Question quality and diversity. Exam wrappers and guided student reflection.

Rationale for giving exams:

Three rationales for giving exams: Well constructed exams motivate students and reinforce learning. Well constructed exams enable teachers to assess the students mastery of course objectives. Exams provide feedback on teaching, often showing what was or was not communicated clearly https://www.k-state.edu/ksde/alp/resources/Handout-Module6.pdf

Hallmarks of effective exams:

Three hallmarks of effective exams: Assesses student mastery of learning outcomes. Accurately reflects the emphasis placed on course content and activities and aligns to learning outcomes. Has variable questions in levels of difficulty and style that measures an appropriate level of student knowledge. https://www.k-state.edu/ksde/alp/resources/Handout-Module6.pdf

Assessing student mastery of learning outcomes: Define the scope of the exam: specific LOs, readings, class activities. Select the types of questions that best address the skill/concept being tested: fixed response vs open response. Focusing on shared assessment goals lets students prepare with intention and reinforces learning. https://utexas.app.box.com/v/Putting-it-all-together Adapted from UT-Faculty Innovation Center

What can mastery look like? • Students can explain concepts, principles, and processes by putting it their own words, justifying their answers, and showing their reasoning. • Students can interpret by making sense of data, text, and experience through images, analogies, stories, and models. • Students can apply by effectively using and adapting what they know in new and complex contexts. http://www.ascd.org/ASCD/pdf/siteASCD/publications/UbD_WhitePaper0312.pdf

Accurately reflects the emphasis placed on course content and activities. Create an exam blueprint: a grid showing the distribution of types of questions over each learning outcome. Develop a grading rubric while writing exam questions. Create assessment tools while building lesson plans or right after lectures. Fair tests use skills that are known to the students. https://www.k-state.edu/ksde/alp/resources/Handout-Module6.pdf

Questions vary in levels of difficulty and style that measures an appropriate level of student knowledge. Cognitive complexity: questions range from simple recall to problem solving, critical thinking and reasoning. Meaningfulness: questions are relevant to the student. Language Appropriateness: Use the terms and lexicon from class. Reliability: Test responses reflect student understanding. Student misconceptions can be gleaned from in-class formative activities. https://www.k-state.edu/ksde/alp/resources/Handout-Module6.pdf

Best Practices for MCQs: Add sequential levels of difficulty -substitute definitions in lieu of terms. -avoid broad generalizations Incorporate competency skills -Use designs and data from classic experiments -Incorporate scientific fluency skills into the answers Use prompts and distractors that are unambiguous and specific -good distractors are relevant to the correct answer but wrong in their degree of precision or hierarchy. Keep a list of Blooms action verbs close at hand. Writing Good Multiple Choice Exams. UT-Austin Faculty Innovation Center

Exam wrappers promote student metacognition. Wrappers are tools given to students that help them reflect on their preparation time and the appropriateness of their study strategies and help them define the nature of their errors to find any recurring patterns. http://serc.carleton.edu/acm_teagle/projects/tibbetts.html https://www.cmu.edu/teaching/designteach/teach/examwrappers/

Resources: https://www.k-state.edu/ksde/alp/resources/Handout-Module6.pdf Writing Good Multiple Choice Exams. https://utexas.app.box.com/v/writing-multiple-choice-qs Check for Learning. https://facultyinnovate.utexas.edu/check-learning Tibbetts, Tim. (November 22, 2016) “Metacognition and Exam Wrappers in Biology” Associated Colleges of the Midwest. Web. http://serc.carleton.edu/acm_teagle/projects/tibbetts.html Anderson, L. W. and Krathwohl, D. R., et al (Eds.) (2001) A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. Allyn & Bacon. Boston, MA (Pearson Education Group) Lovett, M. C. (2013) Make exams worth more than the grade: Using exam wrappers to promote metacognition. In: Kaplan, M., Silver, N., LaVague-Manty, D., Meizlish, D. (eds) Using reflection and metacognition to improve student learning: Across the disciplines, across the academy, Sterling, VA: Stylus, pp. 18–52