More Writing But Less Grading: Calibrated Peer Review™ Share The Future IV - Tempe, AZ John Wise The Pennsylvania State University

Engineering Instructional Services Penn State’s Engineering Instructional Services College of Engineering, University Park PA College of Engineering, University Park PA Associate Dean for Undergraduate Studies, Robert Pangborn Associate Dean for Undergraduate Studies, Robert Pangborn Support and Training for Faculty and Teaching Assistants Support and Training for Faculty and Teaching Assistants

Engineering Instructional Services Review of Bloom’s Taxonomy Knowledge Knowledge Comprehension Comprehension Application Application Analysis Analysis Synthesis Synthesis Evaluation Evaluation Bloom (1956) Taxonomy of educational objectives: the classification of educational goals

Engineering Instructional Services Problem Statement Large Classes in Particular: Large Classes in Particular:  Predominantly Lecture  Multiple Choice / “Scantron” Grading  Feedback and Writing Difficult All Classes All Classes  Student “Final” Papers Often First Drafts  Product vs. Process Orientation  Concept Misunderstandings

Engineering Instructional Services History of Calibrated Peer Review (CPR) Large Molecular Science UCLA Large Molecular Science UCLA Science-Based Model Science-Based Model  Research, Write, Peer Review Now Can Serve Any Discipline and Educational Level Now Can Serve Any Discipline and Educational Level

Engineering Instructional Services How It Works Create (or Select) Assignment Create (or Select) Assignment Set Timing and Standards Set Timing and Standards Manage Assignments Manage Assignments Review Results Review Results

Engineering Instructional Services Create Assignment Choose Source Choose Source Define Writing Assignment Define Writing Assignment Write Exemplar Write Exemplar Write Moderate / Poor Examples Write Moderate / Poor Examples Generate Style and Content Questions Generate Style and Content Questions Assess Own Documents for Calibration Assess Own Documents for Calibration (sample)

Engineering Instructional Services Set Timing and Standards Control Over Text Entry Times Control Over Text Entry Times Control Over Calibration Standards Control Over Calibration Standards Control Over Scoring Scheme Control Over Scoring Scheme

Scoring Template: High Difficulty Calibrations To master each calibration students must: Answer 3.0 of 4 (75%) STYLE questions correctly, and Answer 5.3 of 7 (75%) CONTENT questions correctly, and NOT deviate by more than 2.0 points from the RATING of a calibration text Reviews To master reviews students must: NOT deviate by more than 2.0 points from the AVERAGE RATING of the reviewed text Self-Assessment To master self-assessment students must: NOT deviate by more than 1.0 points from the AVERAGE RATING of their text to receive full credit, or NOT deviate by more than 2.0 points from the AVERAGE RATING of their text to receive half credit

Directions: Enter the points you want associated with each assignment stage. All points must sum to 100. Each point total can range from 0 to Text quality:XX points 2. Calibrations: XX points over 3 Calibrations 3. Reviews: XX points over 3 Reviews 4. Self- assessment: XX points TOTAL:must sum to 100 points

Engineering Instructional Services Management Student Reminders Student Reminders Special Requests Special Requests  Late Submissions  Technical Difficulties

Engineering Instructional Services Review Results Scan Student Results Scan Student Results Identify “Problem” Scores Identify “Problem” Scores Ability to Randomly Review Some or All Student Submissions Ability to Randomly Review Some or All Student Submissions  Can also participate AS a student!

Engineering Instructional Services Student Name Overall Grade (Out of 100) Text Score (Out of 10) Reviewer Competency Index (RCI) (1-6)Complete [name] finished [name]9486finished [name]9175finished

Engineering Instructional Services Student Point of View Review Materials Review Materials Write Write Calibrate Calibrate  Review three examples, compared to professor Review 3 Random Peers Review 3 Random Peers Review Self Review Self

Engineering Instructional Services Calibrated Peer Review ™ (CPR) Available from UCLA Available from UCLA  Contact me for sample assignment Contact me for sample assignment   Contact UCLA regarding traveling workshops Contact UCLA regarding traveling workshops

More Writing But Less Grading: Calibrated Peer Review™ Share The Future IV - Tempe, AZ John Wise The Pennsylvania State University