Presentation is loading. Please wait.

Presentation is loading. Please wait.

Gregory Light, PhD Brazil Science, Technology, Humanities, Engineering and Mathematics (STHEM) Consortium 1ST WORKSHOP: LORENA, BRAZIL, MAY 2630, 2014.

Similar presentations


Presentation on theme: "Gregory Light, PhD Brazil Science, Technology, Humanities, Engineering and Mathematics (STHEM) Consortium 1ST WORKSHOP: LORENA, BRAZIL, MAY 2630, 2014."— Presentation transcript:

1 Gregory Light, PhD Brazil Science, Technology, Humanities, Engineering and Mathematics (STHEM) Consortium 1ST WORKSHOP: LORENA, BRAZIL, MAY 2630, 2014 Changing Teaching, Transforming Learning Session II: Assessing Learning

2 Overview Session I: Session I: Design Principles & Learning Outcomes Global & National Challenges (Interactive Presentation) Teaching for Learning: a Framework (Interactive Presentation) Constructing Learning Outcomes (Activity) Session II: Session II: Assessing Learning Assessing Student Learning (Discussion) Dimensions of Assessment (Interactive Presentation/Activity) Aligning Assessment (Activity)

3 The quickest way to change student learning is to change the assessment system. -- Elton & Laurillard, 1979, p. 10

4 ASSESSMENT Group Question: Why do we assess students? At Tables Share

5 Purposes of Assessment Summative To pass or fail a student To grade or rank a student To license to proceed To select for future courses To license to practice To predict success in future courses To predict success in employment To select for future employment

6 Purposes of Assessment Formative To provide feedback to students to improve their learning To motivate students To diagnose a students strengths and weaknesses To help students develop their skills of self- assessment To provide a profile of what a student has learnt

7 Purposes of Assessment Evaluative To provide feedback to teachers To improve teaching To evaluate a courses strengths and weaknesses To make the course appear respectable and credit worthy to other institutions and employers

8 Dimensions of Assessment

9 FormativeSummative Reliability The extent to which the results of the assessment method can be trusted Validity The extent to which the assessment methods reflect student learning and the learning goals of the course Essentially designed to sum up someones achievement Essentially designed for use in helping the learning process Practicality Criterion Referenced Norm-Referenced Based on self-reflection of learning achieved on the course Self-Referenced Based on knowledge and skills learned on course Based on comparisons with others in the group Peer-Referenced Based on peer appraisal of learning achieved on the course

10 Dimensions of Assessment FormativeSummative Reliability The extent to which the results of the assessment method can be trusted Validity The extent to which the assessment methods reflect student learning and the learning goals of the course Essentially designed to sum up someones achievement Essentially designed for use in helping the learning process Practicality Criterion Referenced Norm-Referenced Based on self-reflection of learning achieved on the course Self-Referenced Based on knowledge and skills learned on course Based on comparisons with others in the group Peer-Referenced Based on peer appraisal of learning achieved on the course

11 Dimensions of Assessment FormativeSummative Reliability The extent to which the results of the assessment method can be trusted Validity The extent to which the assessment methods reflect student learning and the learning goals of the course Essentially designed to sum up someones achievement Essentially designed for use in helping the learning process Practicality Criterion Referenced Norm-Referenced Based on self-reflection of learning achieved on the course Self-Referenced Based on knowledge and skills learned on course Based on comparisons with others in the group Peer-Referenced Based on peer appraisal of learning achieved on the course

12 Dimensions of Assessment FormativeSummative Reliability The extent to which the results of the assessment method can be trusted Validity The extent to which the assessment methods reflect student learning and the learning goals of the course Essentially designed to sum up someones achievement Essentially designed for use in helping the learning process Practicality Criterion Referenced Norm-Referenced Based on self-reflection of learning achieved on the course Self-Referenced Based on knowledge and skills learned on course Based on comparisons with others in the group Peer-Referenced Based on peer appraisal of learning achieved on the course

13 Dimensions of Assessment FormativeSummative Reliability The extent to which the results of the assessment method can be trusted Validity The extent to which the assessment methods reflect student learning and the learning goals of the course Essentially designed to sum up someones achievement Essentially designed for use in helping the learning process Practicality Criterion Referenced Norm-Referenced Based on self-reflection of learning achieved on the course Self-Referenced Based on knowledge and skills learned on course Based on comparisons with others in the group Peer-Referenced Based on peer appraisal of learning achieved on the course

14 Dimensions of Assessment FormativeSummative Reliability The extent to which the results of the assessment method can be trusted Validity The extent to which the assessment methods reflect student learning and the learning goals of the course Essentially designed to sum up someones achievement Essentially designed for use in helping the learning process Practicality Criterion Referenced Norm-Referenced Based on self-reflection of learning achieved on the course Self-Referenced Based on knowledge and skills learned on course Based on comparisons with others in the group Peer-Referenced Based on peer appraisal of learning achieved on the course TEACHING-CENTERED LEARNING-CENTERED

15 Activity: Assessment Methods Choose an assessment method(s) you currently use and map it on to 1) a learning outcome 2) these dimensions. Do individually, then share at table Then with whole group

16 Assessment Methods (Some Examples) final exam (written) Homework projects reports Presentations observed discussion groups office hours final exam (multiple choice) quizzes cold calling career performance standardized tests personal response systems Higher validity – less reliable Higher reliability – less validity

17 Aligning Assessment Using Rubrics (Making valid assessments more reliable) Quick Share

18 Aligning Assessment with a Learning Outcome (Biology Example) Course Goal Learning Outcome Formative Assessment Summative Assessment What will students learn? If they have learned it, what will students know and be able to do? What will students do to learn it? How will students demonstrate they know it or are able to do it? Students will understand the transfer of information from DNA to proteins. Students will be able to predict changes in amino acid sequences caused by mutations. In groups students are given sequence of DNA corresponding to amino acid sequence. Students identify reading frame and predict amino acid changes due to mutations in that sequence. On exam students will predict the new amino acid sequence that results from a mutation in a given gene sequence. Knight, (2011)

19 A Typical Rubric Format STUDENT EVIDENCE (1) Weak Little or no evidence of outcome (2) Basic Some evidence of outcome (3) Proficient Detailed evidence of outcome (4) Strong Highly creative; outcome TEACHER CRITERIA Criterion 1 Argument Criterion 2 Integration of literature Criterion 3 Writing quality Adapted from Beauchamp et al 1996

20 Assessing Writing: Example Rubric (1) Weak Little or no evidence of outcome (2) Basic Some evidence of outcome (3) Proficient Detailed evidence of outcome (4) Strong Highly creative; outcome Criterion 1 Argument Rambling; poor use of logic; personal opinion Basic structure with some use of evidence Strong structure & logic, evidence used throughout Strong structure & evidence; sophisticated discussion Criterion 2 Integration of literature No or minimal use of sources Sources used, but no integration of writers ideas Sources well integrated into authors ideas Sources well integrated; critique made Criterion 3 Writing quality Poor organization, grammar, syntax Some effort to organize ideas; grammar/syntax problems Good organization; very few grammar/ syntax problems Well-developed flow; error-free; elegant style Adapted from Beauchamp et al 1996

21 Benefits of Rubrics Used for grading (Summative) or feedback (Formative) Clear criteria (Criteria Ref.) Ensure grading aligns with learning outcomes (Validity) Shared with multiple teacher graders/situations (Reliability) Can be used with students (Self Ref.) and peers (Peer Ref.)

22 Activity In small group, identify and design a rubric for a specific Learning outcome Large Group Discussion

23 Aligning Assessment Using Multiple Choice Questions (MSQ) (Making reliable assessments more valid) Quick Share

24 MCQs as formative instruction Answer and provide your rationale: 1. Which of the following has/have intrinsic pacemaker characteristics? a) Medulla c) Sinoatrial node b) Pons d) Atrioventricular node Use rationales (short open-ended explanation for choice) to let students demonstrate their learning - can be graded or not.. Stanford Learning Lab: Nash & Shaeffer, 1999; Schaeffer et al., 1999

25 Ideal rationale: SA node is the normal pacemaker for the entire heart. AV node also has pacemaker potential, but is overshadowed by SA node. Medulla has pacemaker potential for breathing rhythm as well. Pons helps refine rhythm, but does not have pacemaker potential. Less-than-ideal rationales: Offering an incomplete answer: Normally the SA node is responsible for generating heart rate, and it is able to do this because of its intrinsic rhythm. The AV node also has an intrinsic rhythm, but it is overshadowed by that of the SA node. Providing a book definition: The sinoatrial node is the pacemaker of the mammalian heart. Providing irrelevant information: Stretch receptors are located in the aortic arch and the carotid sinus. They have the ability to respond to changes in pressure. Restating the question: The SA node, AV node, and medulla all possess intrinsic pacemaker characteristics as they all serve as intrinsic pacemakers. Blind appeal to authority: This answer is right because Professor Heller said that it was, and Professor Heller is cool.

26 Activity In small group, share other ways in which the validity of MCQs might be enhanced. Share with Large Group

27 Testing low-level knowledge Purely economic loss is recoverable in a product liability action. a)True b)False Purely economic recovery will be barred in which of the following causes of action? a)Negligence b)Fraud c)Defamation d)Product liability Whats the rule? No context, not allowing for interpretation/analysis from Case & Donohue, 2008

28 Revision: Higher-level understanding A restaurant hired an exterminator to eliminate cockroaches from the basement under the restaurant. Around midnight, the exterminator applied to the basement floor and walls an effective pesticide that he had purchased from the manufacturer. A toxic gas released by the pesticide penetrated into the restaurant kitchen and did not disperse by the next day. As a result, the restaurant was required to close that day. The restaurant brought a tort action based on product liability against the pesticide manufacturer for lost profits. Will the restaurant prevail? a)No, because in this action purely economic loss in not recoverable. * b)No, because the exterminator was the proximate cause of the restaurant's damages. c)Yes, because the manufacture of pesticides is an abnormally dangerous activity. d)Yes, because the pesticide was being used as intended. from Case & Donohue, 2008

29 3 ideas to take with you Assessment is teaching Students need to be involved Assessment gives you no more than what you ask for

30 Final QUESTIONS?

31 References Entwistle, N. & Tait, H. (1990) Approaches to learning, evaluations of teaching and preferences for contrasting academic environments, Higher Education, 19 (2): 169–94. Knight, J. (2011) University of Colorado. NAS/HHMI Summer Institute on Undergraduate Biology Education, Madison, WI. Light, G, & Micari, M. (In press) Making Scientists: Six Principles for Effective College Teaching, Harvard University Press. Beauchamp, McConaghy, Parsons & Sanford. (1996) Teaching From the Outside In. Duval: 1996, 37.


Download ppt "Gregory Light, PhD Brazil Science, Technology, Humanities, Engineering and Mathematics (STHEM) Consortium 1ST WORKSHOP: LORENA, BRAZIL, MAY 2630, 2014."

Similar presentations


Ads by Google