Presentation is loading. Please wait.

Presentation is loading. Please wait.

Peer evaluation in TBL: A student-developed approach

Similar presentations


Presentation on theme: "Peer evaluation in TBL: A student-developed approach"— Presentation transcript:

1 Peer evaluation in TBL: A student-developed approach
Derek R. Lane, Ph.D. Associate Dean College of Communications and Information Studies H. Lester Reynolds Engineering Endowed Professor June 1, : : Room C150

2 TBL Conference 2007 REFLECTIONS
People have problems working together TBL is distinctive and important Dee Fink “Students don’t like doing peer evaluations.” Interesting examples of TBL Self/Peer Assessment Cooperative Learning Skills Self-directed Learning Interpersonal Skills Rudolph Navari

3 TBL Conference 2007 REFLECTIONS
TBL and the Business Strategy Game Robert A. Herring, III Winston-Salem State University Sample Survey of Student Attitudes Learning Assurance Report Leadership Skills SCALE Collaboration and Teamwork SCALE Making Student Thinking Visible Jim Sibley University of British Columbia Strategies to Create Engaging and Reportable TBL application-oriented activities “Residual Dissent”

4 TBL Conference 2007 REFLECTIONS
iPeer 2.0 can help facilitate TBL peer evaluations Sophie Spiridonoff / Jim Sibley (UBC) Medical students have mixed opinions about satisfaction with peer evaluation Dean Parmlee, Dan DeStephen, Nicole Borges (Wright State U.)

5

6 What Makes a Team Work? “Individual commitment to a group effort: this is what makes a team work, a company work, a society work, and a civilization work.” --Vince Lombardi, Former Green Bay Packers Coach

7 Driving Questions RQ1What is the purpose of peer evaluation?
RQ2 How do we improve our methods of helping students assess professionalism? Students support that which they help to create.

8 Session Overview Purpose of Peer Evaluation: A TBL Retrospective
Characteristics of High Performing Teams Importance of Helpful Feedback Strategies for Establishing Meaningful Formative (PROCESS) and Summative (OUTCOME) Assessment

9 I. Purpose of Peer Evaluation in TBL
A TBL Retrospective

10 TBL Retrospective Michaelsen, L.K., & Black, R. H. (1994). Building Learning Teams: The Key To Harnessing the Power of Small Groups in Higher Education. Growth Partners.

11 Performance Evaluation
Three Essential Components of Grading System Individual Performance Individual Accountability for Reading/Homework Assess Higher-Level Thinking Skills Group Performance Provides Incentives to Support Cohesiveness Justification for Putting Effort into Group Work Peer Evaluation

12 Peer Evaluation Solves Important Motivational Problems
Provides Incentives for Equitable Group Participation Removes students’ fear that they will have to choose between receiving a low grade on the group assignments and having to “carry” group work (when other group members fail to do their fair share)

13 Performance Evaluation
Three Essential Components of Grading System Individual Performance Group Performance Peer Evaluation Give sufficient weight to each of the components Must be clear to students that the instructor thinks each is important (minimum 10%) Must be responsive to student concerns for fairness and equity (reduce social loafing)

14 Necessary Components of TBL Tasks
Heterogeneous composition of diverse interdependent work teams Clear, specific, and widely shared team goals which encourage group cohesion Sufficiently difficult and meaningful team activities that do not allow one person to accomplish the task alone. Regular, descriptive, specific, relevant, timely and usable internal peer feedback External comparisons which are emphasized through immediate and ongoing feedback about organizational performances relative to other teams.

15 II. Characteristics of High Performing Teams

16 Team Performance and Effectiveness
In a well-run team, the overall performance is superior to that of individual efforts “The whole is greater than a sum of its parts” In a poorly run team, the overall performance is worse than what would be obtained by linearly combining the contributions of many individuals

17 The Wisdom of Teams: Creating the High Performance Organization Katzenbach & Smith ©1999
Six team basics define the discipline required for team performance: Small Number (5-7; generally less than 12) Complementary Skills Common Purpose Common Set of Specific Performance Goals Commonly Agreed Upon Working Approach (structure, norms) Mutual Accountability

18 The Team Performance Curve
Impact The Team Performance Curve High-Performance Team Real Team Working Group Potential Team From: Katzenbach & Smith, The Wisdom of Teams: Creating the High-Performance Organization. Used by permission. Pseudo-Team Team Effectiveness

19 Building Team Performance
Establish urgency, demand performance standards and direction Members should be selected for skill and skill potential--not personality Pay particular attention to first meetings and actions Set clear rules of behavior Set immediate performance-oriented tasks and goals Challenge the group with fresh facts and information Spend lots of time together Exploit the power of positive feedback, recognition, and reward Katzenbach & Smith, 1993

20 Eight Characteristics of High Performance Teams
A Clear, Challenging Goal A Results-Driven Structure (clear roles, effective communication) Competent Team Members Unified Commitment Collaborative Climate Standards of Excellence External Support and Recognition Effective Principled Leadership LaFasto, F., & Larson, C. (2001).  When teams work best: 6,000 team members and leaders tell what it takes to succeed.  Thousand Oaks, CA:  Sage.

21 III. Importance of Helpful Feedback
The importance of process feedback can be illustrated with a simple example. If a person Boards a plane in San Francisco and wants to fly to Washington DC they can expect to see a mountain Range in the first minutes. If all they see is Ocean then they are obviously going in the wrong Direction. For students enrolled in TBL classes, this Is what process feedback in the form of peer evaluation Ought to be!

22 Formative vs. Summative Peer Evaluation
Formative (process) peer evaluation procedures can potentially provide a results-driven structure and establish the criteria or standards of excellence to ensure mutual accountability. Summative (outcome) peer evaluations provide an overall standardized assessment of the extent to which individuals contributed to team success. (Helping Behavior)

23 IV. Strategies for Meaningful Assessment

24 TEAM FORMATION Dear God, please don’t make me work with THEM!

25

26 Assign a number from

27 Preliminary Individual Peer Evaluation Criteria
Reflect on past group experiences Consider + - experiences What are the four or five most important issues that contributed to the group success or failure? Develop a list of preliminary criteria you believe are important for group success / peer evaluation

28 Preliminary List of Evaluative Criteria
Instructions: Develop a list of evaluative criteria that you believe is important for students to use when working in TBL instructional teams in order to be successful.

29 Final TBL Evaluative Criteria
Instructions: 1) Make introductions and share contact information. 2) After introductions have been made and contact information shared, decide on a name for your team. The final decision must be unanimous and the name should reflect positively on the qualifications of the individuals in your group. Provide the decision on the line below. Group # ___ Team Name: __________________________________

30 Final TBL Evaluative Criteria
Share individual preliminary criteria with all group members. Consolidate the preliminary list of criteria and develop a final group list of the criteria that the group is comfortable using for peer evaluations. Use the directions on the next page to develop procedures that incorporate the group criteria that will allow efficient feedback. Submit the final peer evaluation procedures and criteria as well as the Group # and Firm Name along with all of the individual criteria sheets at the beginning of the next class.

31 Team Peer Evaluation Procedures & Criteria
Instructions: Before proceeding, read the four-page article written by Michaelsen and Schultheiss’ (1988) Making Feedback Helpful. Develop a system for providing performance feedback to the members of your team. The performance feedback system should include: 1. Provide a statement of the team goals and objectives that you intend to achieve. These goals should reflect an integration of individual team members’ goals for the course. 2. A description of how your team intends to collect the data on which the feedback will be based. Please include a copy of a specific peer grading form that clearly indicates how data will be collected. 3. A description of the feedback process you intend to use. Please specify: a. When the feedback will be given? Be specific. b. Who will give it? Be specific. 4. Assess the difficulties that you are likely to encounter in implementing your performance feedback system. How will the team overcome these difficulties? 5. A statement of how the system provides input into the summative helping behavior grade at the end of the class.

32 Characteristics of Helpful Feedback
descriptive, not evaluative, and is owned by the sender (not anonymous) specific, not general (focus on specific issues and behaviors) honest and sincere expressed in terms relevant to the self-perceived needs of the receiver timely and in context desired by the receiver usable; concerned with behavior over which the receiver has control Michaelsen, L.K., & Schultheiss, E. E. (1988). Making feedback helpful. Organizational Behavior Teaching Review 13(3),

33 Team Peer Evaluation Procedures & Criteria
Your performance feedback system will be evaluated using the following criteria: 1. Is the team collecting data they will need to support the achievement of course goals? 2. Will the procedures they intend to use support the achievement of their objective(s)? 3. Are the procedures they intend to use consistent with the seven characteristics of helpful feedback? 4. Are the procedures practical (i.e., can they be implemented effectively given the specific situation in which they will be used)? 5. Have they accurately anticipated the problems they are likely to encounter in implementing their performance feedback system and have they developed strategies for overcoming the difficulties?

34 Key Elements Critical to Cooperation
Positive Interdependence individuals succeed only when group succeeds Individual Accountability learn together to perform better as an individual Promotive Interaction help, assist, support, encourage, praise efforts Social Skills (Professionalism) communication, leadership, decision-making, trust-building, and conflict management skills Group Processing

35 Code of Cooperation EVERY member is responsible for the team's progress and success. Attend all sessions and be on time. Listen to and show respect for the contributions of other members; be an active listener. Criticize ideas, not persons. Resolve conflicts constructively. Pay attention - avoid disruptive behavior. Avoid disruptive side conversations. Only one person speaks at a time. Everyone participates -- no one dominates. Be succinct, avoid long anecdotes and examples. No rank in the room. Attend to your personal comfort needs at any time but minimize team disruption. HAVE FUN! Adapted from the Boeing Commercial Airplane Group

36 IU School of Medicine - South Bend TBL Self/Peer Assessment Criteria
Cooperative Learning Skills Attendance Listening Interaction / Participation Information Sharing Defer to Needs of the Group Self-Directed Learning Preparation Depth of Knowledge Identifies Limits of Knowledge Problem Solving Confidence Interpersonal Skills Gives instructive feedback Accepts instructive feedback Shows sensitivity and concerns for others Clarity in rationale/reasoning process This is the first mention of a Code of Cooperation and it is an important one. Presenting this Code as a way of defining desirable team norms and acceptable interpersonal communication is very helpful. The Code sets a norm for behavior - a code of ethics, if you will. Teams should add to this code as desired and as the need arises to address necessary new norms. The code should be viewed as a “living document.” In fact, it is advisable to request additions at the time you present this. Jackson, Hamlett, & Navari

37 TBL and the Business Strategy Game
Robert Herring, III Winston-Salem State U. This is the first mention of a Code of Cooperation and it is an important one. Presenting this Code as a way of defining desirable team norms and acceptable interpersonal communication is very helpful. The Code sets a norm for behavior - a code of ethics, if you will. Teams should add to this code as desired and as the need arises to address necessary new norms. The code should be viewed as a “living document.” In fact, it is advisable to request additions at the time you present this.

38 Levine, R. E. , Kelly, A. , Karakoc, T. , & Haidet, P. (2007)
Levine, R. E., Kelly, A., Karakoc, T., & Haidet, P. (2007). Peer evaluation in a clinical clerkship: Student attitudes, experiences, and correlations with traditional assessments. Academic Psychiatry, 31(1),

39 Who seeks formative feedback?
360 Year 3 Undergraduate Medical Students at University of Aberdeen, UK Less than half the students (46%) collected their formative feedback. Females (p=.004) and higher achievers (p=.020) were more likely to seek formative feedback. Sinclair & Cleland (June, 2007) Journal of Medical Education

40 Civil Engineering Design Team Criteria
Attendance Punctuality Attitude Respect Preparedness Completed Readings Completed Homework Ready for Meetings Preparation Goal Oriented Flexibility Quality Work Completed the First Time Accountability Trustworthy and Reliable Gives Proper Notice if Can’t Attend a Meeting Individual Initiative Team commitment Leadership Willingness to accept alternative viewpoints

41 Little and Cardenas (2001) Quality of Technical Work
Ability to Communicate Ability to Provide Leadership Commitment to Team and Project Demonstrated Effectiveness

42 Helping Behavior Summative (outcome) peer evaluations provide an overall standardized assessment of the extent to which individuals contributed to team success. (Helping Behavior) Standardized base 10 n-1 X 10 (6-1)*10 = 50 pts to assign Mean of group scores Percent multiplier 100 point distribution (peer*GRAT)

43 Successful Teams employ self examination and assessment
change basic operating assumptions when necessary monitor their own interaction patterns and progress are willing to make changes are more attentive to group processes begin their discussion by attempting to analyze the problem before trying to search for viable solutions

44 Session Summary Purpose of Peer Evaluation: A TBL Retrospective
Characteristics of High Performing Teams Importance of Helpful Feedback Strategies for Establishing Meaningful Formative (PROCESS) and Summative (OUTCOME) Assessment

45 Student-Developed Peer Evaluation Procedures & Criteria
Feedback—both formative (process) feedback and summative (helping behavior) feedback should be direct, clear, and ensure that group members are accountable to the rest of the team.  The value of the process feedback is that all of the members of the team have the opportunity to discuss the feedback and make performance improvements over the course of the semester.

46

47

48 Ability to Analyze the Problem
Hirokawa & Gouran’s Four Essential Functions that Decision-Making Groups Must Perform Well Ability to Analyze the Problem Ability to Identify Appropriate Criteria for Making a Decision Ability to Develop Alternative Choices from Which to Choose Ability to Evaluate the Positive and Negative Aspects of Alternative Choices Prior to Making a Decision** Order is not important but all functions must be performed well!


Download ppt "Peer evaluation in TBL: A student-developed approach"

Similar presentations


Ads by Google