1. Online Student Peer Reviews William J. Wolfe Professor of Computer Science California State University Channel Islands william.wolfe@csuci.edu

3. NEW MENTAL HOSPITAL MASTER PLAN VOLUME 7 -- NUMBER 7 CAMARILLO, VENTURA COUNTY, CALIFORNIA, NOVEMBER 25, 1932 $2.00 A YEAR IN ADVANCE

10. Teaching and Learning

11. Some Educational Issues

12. ref: http://www.integralworld.net/edwards17.html

14. Vygotsky's Zone of Proximal Development ZPD is “the distance between the actual developmental level as determined by independent problem solving and the level of potential development as determined through problem solving under adult guidance, or in collaboration with more capable peers.”

16. Online Student Peer Reviews

17. Ref: Peer Instruction: A User's Manual. Eric Mazur, Prentice Hall, 1997.

19. "The jury didn't consist of my peers... This jury believed citizens have a civic duty to obey the laws".

20. http://www.stus.com/sv/subjects/lawyerjokes.htm

21. Online Peer Reviews: Outline Pros and cons Implementation Considerations Student Websites Course Website Peer Review Process Results

22. Peer Reviews – Advantages Students learn from each other. Students get lots of feedback. Students develop skills as evaluators. Students learn to appreciate evaluation criteria. Students see how they compare to their peers. Students see the class from teacher’s perspective. Students get to know one another. Teacher plays role of supervisor.

25. Example: ("do the math") 30 students Peer review of 1 assignment: 30 x 30 = 900 copies of the homeworks 5 pages/homework: 5 x 900 = 4500 pages.

26. Software Engineering (CSC 4508): 34 students Theory: 1 Assignment: 1,122 reviews. 15 Assignments: 16,830 reviews. Fact: 1 Assignment:300 – 400 reviews. 15 Assignments: 5,212 reviews. Number of Reviews

28. Peer Reviews – Concerns Students don’t know the subject. Students are not skilled evaluators. Students might send inappropriate messages. Students will not do that much work. Students will copy (cheat)! Keeping track of the reviews is very difficult. Student privacy.

31. Implementation Details What type of assignment? How many reviews does each student do? How many reviews does each student get? Who reviews whom? Does the reviewer have to be “qualified”? Will students grade fairly and accurately? Anonymous reviews? Grades based on peer reviews? Grade the peer reviews? Opportunity to revise based on reviews? Peer review of the peer reviews?

32. Peer Review System

33. Course Website Instructor sets up a course website Web pages Database Scripts Keeps track of all the activity/data

34. Peer Review Process Student: –Logs onto the course website Unique password for each student. –Sees list of URLs List of links to the student web sites –Picks one from the list Accesses a student web site Finds the assignment –Reviews the assignment. –Submits an anonymous review: score (1 – 10) comment –Goes back to the list of URLs and picks another.

35. System Flow

36. List of URLs

37. Student Website Students use their own website. Students must have basic web skills. Students must have access to a web service. Students cannot (easily) hide their identity.

38. COMP 449 Human Computer Interaction John Doe Weekly Assignments COMP 449 Assignment #1 COMP 449 Assignment #2 COMP 449 Assignment #3 COMP 449 Assignment #4 COMP 449 Assignment #5 COMP 449 Assignment #6 COMP 449 Assignment #7 COMP 449 Assignment #8 COMP 449 Assignment #9 COMP 449 Assignment #10 COMP 449 Assignment #11 COMP 449 Assignment #12 COMP 449 Assignment #13 COMP 449 Assignment #14

39. Score + Comment

41. Grading Criteria (Rubric)

42. Peer Reviews Received

43. ” Looks pretty good” Sample Peer Review

44. Perfunctory Reviews perfunctory \pur-FUNGK-tuh-ree\ -- adjective : Done merely to carry out a duty; performed mechanically; done in a careless and superficial manner; characterized by indifference

45. You should have requirements that detail the concepts in section 4.2. Although you had some very good points (i.e. the database should look up student's degree requirements; view should list courses, etc...) almost all your requirements can be more detailed. Go through section 4.2 (each of the sections) and think of what the program would need to do to effective run. Some good examples of what requirements are necessary are on others' websites, however I'll give some to you now: 1.Is there a timeline requirement? 2.Is there a requirement on how much(or how little) this will cost? 3.Is there security requirements? 4.Is there user view requirements? These(and many other questions) are what you should answer in your requirements definition document. Good luck on Assignment #3. Detailed Peer Review

46. Website: http://faculty.csuci.edu/william.wolfe/ucd/online/

55. Software Engineering (CSC 4508) Fall 2002

56. Real Analysis (Math 351) Spring 2003

57. Software Engineering (CSC4508) Fall 2002

62. “perfect” nines eights over 1 under 1 avg dev. avg delta

63. Summary Peer Review Process: Stimulated class activity. Some passionate participation. The “audience effect”: brought up all performance levels. Very accurate evaluations (as a whole). Immediate access to examples of good and poor work. Addressed late and incomplete work. Requires web skills.

66. 1. Online Student Peer Reviews, Proceedings of ACM SIGITE Annual Conference, Salt Lake City, Utah, Oct. 28-30, 2004.Online Student Peer Reviews 2. Student Peer Reviews in an Upper-Division Mathematics Class, exchanges THE ONLINE JOURNAL OF TEACHING AND LEARNING IN THE CSU, (From the Classroom), September, 2003.Student Peer Reviews in an Upper-Division Mathematics Class 3. Course Web Site: http://compsci.csuci.edu/wwolfe/ucd/online Password: GUEST 4. William.Wolfe@csuci.eduWilliam.Wolfe@csuci.edu References