1. Calibrated Peer Review (CPR): Using Technology to Get the Most Out of a Writing Assignment Dr. Laurie S. Starkey Chemistry Dept., Cal Poly Pomona lsstarkey@csupomona.edu (909) 869-3670 http://www.csupomona.edu/~lsstarkey

2. 2 Calibrated Peer Review What is CPR? Web-based tool which enables peer- reviewed writing assignments Students, faculty log in with password at http://cpr.molsci.ucla.edu to access assignments Use and access are currently free; servers are maintained by UCLA

3. 3 CPR Assignment Components Stage 1: Text Entry Stage Essay assignment is given online –Links to supporting materials, relevant web sites, animations, etc., are provided –Student Instructions, Guiding Questions and Writing Prompt are also available –Essay must be formatted using simple HTML tags (can Preview before submitting) Student submits essay online before text entry deadline (CPR verifies word count)

4. 4 CPR Stage 2: Calibration Stage Begins only after text entry deadline Student reads, evaluates 3 Calibration Essays –faculty-generated essays are of varying quality –Evaluation: Student answers 10-12 Calibration Questions for each essay and assigns 1-10 score Student views Calibration results –Do Students answers match Instructors answers? –Feedback is provided for each Calibration Qn –Student can retake Calibration after failing Student becomes Expert Reviewer

5. 5 Sample Calibration Questions Content (accurate, complete) –Does the essay conclude that 150.0 m has four significant figures? –Does the essay identify George Washington as the first president of the United States of America? –Are there scientific errors in the essay, such as incorrect definitions or misleading statements? Style (coherent, integrated, grammar) –Does the essay have a descriptive topic sentence? –Is the authors argument easy to follow? (Y/N) –Are there spelling errors? (None, Some, Many)

6. 6 CPR Stage 3: Review Stage Begins after passing Calibrations Stage Review= Answer Questions, Score 1-10 Student reviews three classmates essays –Anonymous and randomly assigned –Classmates may include the instructor, TAs, grad students, faculty, etc. Student performs a self-evaluation (reviews his or her own work)

7. 7 CPR Stage 4: Results Stage Did Student master review of peers? –Students 1-10 rating is compared to average –Instructor sets maximum allowable deviation Student views detailed reviews of his/her own text –Four reviews have been performed: three peer reviews plus Self-Assessment –Compare answers for each Calibration Question –View feedback provided by peers

8. 8 Computing CPR Assignment Grade Students Text Entry (30 points) –Calculates weighted average score based on reviews –Score of 10 = 30 pts; Score of 5 = 15 pts Students Calibrations (3 x 10 points each) –Pass = 10 pts; Pass after retake = 5 pts; Fail = 0 pts Reviews performed by Student (3 x 10 points ea) –Within allowable deviation = 10 pts; exceeds = 0 pts Students Self-Assessment (10 points) –Within allowable deviation = 10 pts; exceeds = 0 pts

9. 9 Advantages to using CPR Getting more out of a writing assignment –Time on task: write, read 6 essays, self-reflect –Evaluates and promotes reading comprehension and critical thinking –Ideal for content-heavy assignments Lots of feedback, little Instructors time –Peers are trained to be reviewers –Utilizes time outside of class Give Pre-Service teachers grading experience Provides experience using HTML

10. 10 Is CPR right for me? Ideal for online courses or source materials Assignment runs about 2 weeks Not suitable for every writing assignment –Develop Calibration Qns with Y/N answers Need significant time for initial investment (authoring a CPR assignment)

11. 11 How do I begin using CPR? Library contains over 170 CPR assignments –Chemistry, Biology, English, Theater, etc. Contact CPP CPR Administrator (L. Starkey) –Provide Handouts (Intro to CPR; Authoring Assignments) –Create faculty user; give authoring privileges –Create a course –Create CPR accounts for student users –Add students to CPR course Attend UCLA workshop, July 19-21, 2006

12. 12 Acknowledgments UCLA workshop, Summer 2001 –Arlene A. Russell, Krista Motschiedler Brand Cal Poly Faculty Computing Support Center Cal Poly Pomona Chemistry Department