1. The Value of a Self-assessment Rubric in Online Discussion Forums Dr. Rafi Davidson Center of Educational Technology Initiatives Dr. Annette Kahan Communication & Information Educational Center Kaye College of Education, Beer-Sheva, Israel 25-28 June 2007, Beer-Sheva, Tel-Aviv

2. “ Knowledge is located neither in the mind nor in any representation of the mind. All of our understandings are situated in complex webs of experience, action, and interaction. Knowledge is a dynamic, evolving phenomenon, a fabric of relations in which one individual is fundamentally entwined with all others in a community ” (Martin Ryder, 1999). http://carbon.cudenver.edu/~mryder/reflect/enactivism.html

3. The Concept of Collaborative Knowledge Building Knowledge building may be defined as “ The production and continual improvement of ideas of value to a community, through means that increase the likelihood that what the community accomplishes will be greater than the sum of individual contributions and part of broader cultural efforts ” (Scardamalia & Bereiter, 2003). The use of networked computers provides alternatives to traditional teaching and learning as we move from a single-classroom concept to the concept of a knowledge-building community of learners.

4. The importance of online forums in collaborative knowledge building Online discussion forums are one of the most common and accepted tools for collaborative learning in virtual courses. The forums are a shared online environment that enables collaborative knowledge building and supportive social communities ( Harasim 1989).

5. But Mason and others (1993) point out that dialogue in online forums is disappointing in many instances, since the quality of the learning and the knowledge building is poor. So One of the greatest challenges is promoting discourse and developing a high level of discussion and critical thinking in the virtual environment, that will produce meaningful learning, as comprehensive as possible.

6. The Importance of Self-assessment in Teacher Education Use of the self-assessment rubric encourages pre-service and in-service teachers to develop the abilities of critical thinking, reflection and evaluation, abilities that are very important in the teaching profession. Self-assessment is very effective for learners seeking to improve their knowledge and learning strategies (Shaklee et al. 1997), particularly in a collaborative learning setting. 1!

7. What is our innovation? Models for online discussion self- assessment usually provide criteria for the learners but do not require their active decision-making and reflection on the quality of the posts. 2!

8. Therefore we developed a rubric for self-assessment To attempt to create a significant online learning community. To encourage collaborative knowledge building. To assist learners in recognizing their commitment. To encourage re-organization of the discussion in a relevant manner. To transfer the responsibility of evaluation to the learner. To develop the abilities of reflection and evaluation. To ease the lecturer ’ s workload. 3!

9. The rubric was used in the following courses in a college of education: 1. ICT in Teaching and Learning 2. Information Literacy in Education 3. Teaching Heterogeneous Classrooms

10. Stages in the Development of the Self-assessment Rubric Evaluation and Evaluator% of Final GradeYear Participation (amount of posts) Evaluation by lecturer only 10% - 5%2000 Until A few qualitative and quantitative criteria based on participation and contribution to learning- Evaluation by lecturer only 15%2001 Self-assessment rubric prepared by lecturer Evaluation by lecturer only 20%2002 Self-assessment rubric improved - based on students’ comments Self- evaluation by students with monitoring by lecturer (students determined their own grade) 30%2003 Self-assessment rubric Self- evaluation by students with monitoring by lecturer (students determined their own grade) 30-60%2004- 2006

11. Steps in the use of the rubric The rubric is presented to the students at the beginning of the online courses. During the courses, the students collect their own postings and evaluate their contributions to the knowledge building of their fellow students. The final grades in the courses are a combination of the student's and the lecturer ’ s evaluation. 4!

12. Rubric Categories 1. Personal opinion or idea - Offering a reasoned opinion based on reliable sources. 2. Responding to others - Relevant and supported response which adds to what others have said 3. Referral to reliable sources of information 4. Summarizing a topic or issue discussed on the forum – based on the discussion and contribution of others. 5. Helping peers to solve all kinds of problems 6. Asking challenging questions 5!

13. Rubric Categories (continued) 7. Taking personal initiatives in the forum. These may include: 1. Opening a new topic for discussion in a new forum, writing a document on the topic including sources which are presented to the participants 2. Opening a forum and directing the discussion with peers 3. Summarizing the discussion 6!

14. Student's perspective on the value of the self-assessment rubric (Based on content analysis of students ’ responses to the questions on the value of the self assessment rubric)

15. Student's perspective on the value of the self-assessment The students referred to the following contributions: Motivation Transparency and Fairness of Assessment Collaborative Knowledge Building Self Directed Learning Life Skills Professional Development as Teachers 7!

16. Motivation “ Gives the student a huge motivation to learn ” “ Gives the student a clearer picture of the amount of his participation and his contribution to learning" “ The rubric is a way to give students a push to learn and to evaluate the process they are going through. “

17. Transparency and Fairness of Assessment “ I realized that I have been very active in the various forums. ” “… know where I stand, and if I have done what was expected of me. ” "The student understands exactly why (and how) he got this grade. ” “ I very much like the idea of ranking within each criteria “

18. Collaborative Knowledge Building “ In this way it is possible to find out the extent of our participation in the various forums and our contribution to other students ’ learning ”

19. Self Directed Learning “ Working with the rubric is a process that causes us to look at ourselves critically. ” “ Self-assessment changes the status of the student and makes him an active participant, who is held responsible for his contribution. ” “… It is obvious that the rubric is a necessary tool for learning, especially in self-learning. ”

20. Life Skills “ Once they acquire the skill of self- assessment in learning, they can apply it to life in general. ”

21. Professional Development as Teachers “ The rubric is an excellent opportunity to learn more about our pupil - his real ability to assess himself, his self-confidence …” “ I hope that most of us will try, and also succeed in implementing (self-assessment) in our future classrooms as teachers " 8!

22. Difficulties Expressed by Students using the Rubric It takes a long time and a lot of effort to use the self-assessment rubric. Having the rubric in mind while participating in the forums makes the discussion “ forced ” – it doesn ’ t flow freely. The rubric restricts creativity and free choice.

23. Conclusions Students believe that the self-assessment rubric contributes to their knowledge building and their training to be teachers, increases social support, and allows greater visibility as to how they are evaluated in the courses. Preliminary research of content analysis of forum posts shows that the tool raised the level of discussion and increased critical thinking and collaborative knowledge building. 9!

24. Where do we go from here? Adapting the rubric to the needs of other online courses and student populations. Developing online tools to facilitate the self- assessment process.

25. It is important to continue research in the following areas: Research on the effect of the rubric on learning and motivation of students. Research on the effect of the rubric on change in the role of the instructor. Research on the development of concepts during discussion in forums.

26. Maximum number of points for each category Maximum number of points for each post Number of contributions Each contribution is worth a certain number of points, based on its quality Category 12 points3 points III Contributing a relevant reasoned idea/opinion and basing it on reliable sources 3 points II Contributing a relevant reasoned idea/opinion or one based on a reliable source 2 points I Contributing a relevant idea/opinion 1 point 1. Personal opinion/idea (in the opening message of the forum, not in response to others) 12 points3 points VI Same as V but adding to responses of others and also including referral to additional source/s 3 points V Relevant and supported response which adds to what others have said (like an example or information from experience) 2 points IV Relevant response- without explanation and justification 1 point 2. Responding to participants

27. Maximum number of points for each category Maximum number of points for each post Number of contributions Each contribution is worth a certain number of points, based on its quality Category 8 points2 points VIII Referral to relevant online source with comments 2 points VII Referral to relevant online source without description 1 point 3. Adding a post that refers to reliable sources of information 16 points8 points XI Summarizing a topic discussed in the forum and including additional information from outside sources 8 points X Summarizing a topic discussed in the forum and including additional information covered in the course 5 points IX Summarizing a topic discussed in the forum 3 points 4. Summarizing a topic discussed in the forum – based on posts

28. Maximum number of points for each category Maximum number of points for each post Number of contributions Each contribution is worth a certain number of points, based on its quality Category 9 points3 points XIV Asking a question that received at least two responses from peers 3 Points XIII Asking a question which received at least one answer from peers 2 Points XII Asking a question connected to the course and its purposes 1 Point 5. Asking challenging questions 30 points10 points XVII Raising a topic for discussion, moderating the discussion and summarizing the discussion 10 Points XVI Raising a topic for discussion and moderating the discussion 6 Points XV Raising a topic for discussion 4 Points 6. Initiating a new discussion forum

29. Maximum number of points for each category Maximum number of points for each post Number of contributions Each contribution is worth a certain number of points, based on its quality Category 6 points2 points XIX Providing a solution or idea to help resolve a complex problem 2 Points XVIII Providing a solution or idea to help resolve a simple problem 1 Point 7. Helping peers to solve problems – technical and course- related