Presentation on theme: "Peer evaluation in TBL: A student-developed approach Derek R. Lane, Ph.D. Associate Dean College of Communications and Information Studies H. Lester Reynolds."— Presentation transcript:
Peer evaluation in TBL: A student-developed approach Derek R. Lane, Ph.D. Associate Dean College of Communications and Information Studies H. Lester Reynolds Engineering Endowed Professor June 1, : :45 Room C150
TBL Conference 2007 REFLECTIONS People have problems working together TBL is distinctive and important Dee Fink Students dont like doing peer evaluations. Interesting examples of TBL Self/Peer Assessment Cooperative Learning Skills Self-directed Learning Interpersonal Skills Rudolph Navari
TBL and the Business Strategy Game Robert A. Herring, III Winston-Salem State University Sample Survey of Student Attitudes Learning Assurance Report Leadership Skills SCALE Collaboration and Teamwork SCALE TBL Conference 2007 REFLECTIONS Making Student Thinking Visible Jim Sibley University of British Columbia Strategies to Create Engaging and Reportable TBL application-oriented activities Residual Dissent
TBL Conference 2007 REFLECTIONS iPeer 2.0 can help facilitate TBL peer evaluations Sophie Spiridonoff / Jim Sibley (UBC) Medical students have mixed opinions about satisfaction with peer evaluation Dean Parmlee, Dan DeStephen, Nicole Borges (Wright State U.)
What Makes a Team Work? Individual commitment to a group effort: this is what makes a team work, a company work, a society work, and a civilization work. --Vince Lombardi, Former Green Bay Packers Coach
Driving Questions RQ 1 What is the purpose of peer evaluation? RQ 2 How do we improve our methods of helping students assess professionalism? Students support that which they help to create.
Session Overview I. Purpose of Peer Evaluation: A TBL Retrospective II. Characteristics of High Performing Teams III. Importance of Helpful Feedback IV. Strategies for Establishing Meaningful Formative (PROCESS) and Summative (OUTCOME) Assessment
I. Purpose of Peer Evaluation in TBL A TBL Retrospective
TBL Retrospective Michaelsen, L.K., & Black, R. H. (1994). Building Learning Teams: The Key To Harnessing the Power of Small Groups in Higher Education. Growth Partners.
Performance Evaluation Three Essential Components of Grading System Individual Performance Individual Accountability for Reading/Homework Assess Higher-Level Thinking Skills Group Performance Provides Incentives to Support Cohesiveness Justification for Putting Effort into Group Work Peer Evaluation
Solves Important Motivational Problems Provides Incentives for Equitable Group Participation Removes students fear that they will have to choose between receiving a low grade on the group assignments and having to carry group work (when other group members fail to do their fair share)
Performance Evaluation Three Essential Components of Grading System Individual Performance Group Performance Peer Evaluation Give sufficient weight to each of the components Must be clear to students that the instructor thinks each is important (minimum 10%) Must be responsive to student concerns for fairness and equity (reduce social loafing)
Necessary Components of TBL Tasks Heterogeneous composition of diverse interdependent work teams Clear, specific, and widely shared team goals which encourage group cohesion Sufficiently difficult and meaningful team activities that do not allow one person to accomplish the task alone. Regular, descriptive, specific, relevant, timely and usable internal peer feedback External comparisons which are emphasized through immediate and ongoing feedback about organizational performances relative to other teams.
II. Characteristics of High Performing Teams
Team Performance and Effectiveness In a well-run team, the overall performance is superior to that of individual efforts The whole is greater than a sum of its parts In a poorly run team, the overall performance is worse than what would be obtained by linearly combining the contributions of many individuals
Team Effectiveness Performance Impact Working Group Real Team High-Performance Team Pseudo-Team Potential Team The Team Performance Curve From: Katzenbach & Smith, The Wisdom of Teams: Creating the High-Performance Organization. Used by permission.
Building Team Performance Establish urgency, demand performance standards and direction Members should be selected for skill and skill potential--not personality Pay particular attention to first meetings and actions Set clear rules of behavior Set immediate performance-oriented tasks and goals Challenge the group with fresh facts and information Spend lots of time together Exploit the power of positive feedback, recognition, and reward Katzenbach & Smith, 1993
Eight Characteristics of High Performance Teams A Clear, Challenging Goal A Results-Driven Structure (clear roles, effective communication) Competent Team Members Unified Commitment Collaborative Climate Standards of Excellence External Support and Recognition Effective Principled Leadership LaFasto, F., & Larson, C. (2001). When teams work best: 6,000 team members and leaders tell what it takes to succeed. Thousand Oaks, CA: Sage.
III. Importance of Helpful Feedback
Formative vs. Summative Peer Evaluation Formative (process) peer evaluation procedures can potentially provide a results-driven structure and establish the criteria or standards of excellence to ensure mutual accountability. Summative (outcome) peer evaluations provide an overall standardized assessment of the extent to which individuals contributed to team success. (Helping Behavior)
IV. Strategies for Meaningful Assessment
TEAM FORMATION Dear God, please dont make me work with THEM!
Assign a number from
Preliminary Individual Peer Evaluation Criteria Reflect on past group experiences Consider + - experiences What are the four or five most important issues that contributed to the group success or failure? Develop a list of preliminary criteria you believe are important for group success / peer evaluation
Preliminary List of Evaluative Criteria Instructions: Develop a list of evaluative criteria that you believe is important for students to use when working in TBL instructional teams in order to be successful.
Final TBL Evaluative Criteria Instructions: 1)Make introductions and share contact information. 2)After introductions have been made and contact information shared, decide on a name for your team. The final decision must be unanimous and the name should reflect positively on the qualifications of the individuals in your group. Provide the decision on the line below. Group # ___ Team Name: __________________________________
Final TBL Evaluative Criteria 3) Share individual preliminary criteria with all group members. Consolidate the preliminary list of criteria and develop a final group list of the criteria that the group is comfortable using for peer evaluations. 4) Use the directions on the next page to develop procedures that incorporate the group criteria that will allow efficient feedback. 5) Submit the final peer evaluation procedures and criteria as well as the Group # and Firm Name along with all of the individual criteria sheets at the beginning of the next class.
Team Peer Evaluation Procedures & Criteria Instructions: Before proceeding, read the four-page article written by Michaelsen and Schultheiss (1988) Making Feedback Helpful. Develop a system for providing performance feedback to the members of your team. The performance feedback system should include: 1.Provide a statement of the team goals and objectives that you intend to achieve. These goals should reflect an integration of individual team members goals for the course. 2.A description of how your team intends to collect the data on which the feedback will be based. Please include a copy of a specific peer grading form that clearly indicates how data will be collected. 3.A description of the feedback process you intend to use. Please specify: a.When the feedback will be given? Be specific. b.Who will give it? Be specific. 4.Assess the difficulties that you are likely to encounter in implementing your performance feedback system. How will the team overcome these difficulties? 5.A statement of how the system provides input into the summative helping behavior grade at the end of the class.
Characteristics of Helpful Feedback 1. descriptive, not evaluative, and is owned by the sender (not anonymous) 2. specific, not general (focus on specific issues and behaviors) 3. honest and sincere 4. expressed in terms relevant to the self-perceived needs of the receiver 5. timely and in context 6. desired by the receiver 7. usable; concerned with behavior over which the receiver has control Michaelsen, L.K., & Schultheiss, E. E. (1988). Making feedback helpful. Organizational Behavior Teaching Review 13(3),
Team Peer Evaluation Procedures & Criteria Your performance feedback system will be evaluated using the following criteria: 1.Is the team collecting data they will need to support the achievement of course goals? 2.Will the procedures they intend to use support the achievement of their objective(s)? 3.Are the procedures they intend to use consistent with the seven characteristics of helpful feedback? 4.Are the procedures practical (i.e., can they be implemented effectively given the specific situation in which they will be used)? 5.Have they accurately anticipated the problems they are likely to encounter in implementing their performance feedback system and have they developed strategies for overcoming the difficulties?
Key Elements Critical to Cooperation Positive Interdependence individuals succeed only when group succeeds Individual Accountability learn together to perform better as an individual Promotive Interaction help, assist, support, encourage, praise efforts Social Skills (Professionalism) communication, leadership, decision-making, trust-building, and conflict management skills Group Processing
Code of Cooperation EVERY member is responsible for the team's progress and success. Attend all sessions and be on time. Listen to and show respect for the contributions of other members; be an active listener. Criticize ideas, not persons. Resolve conflicts constructively. Pay attention - avoid disruptive behavior. Avoid disruptive side conversations. Only one person speaks at a time. Everyone participates -- no one dominates. Be succinct, avoid long anecdotes and examples. No rank in the room. Attend to your personal comfort needs at any time but minimize team disruption. HAVE FUN! Adapted from the Boeing Commercial Airplane Group
IU School of Medicine - South Bend TBL Self/Peer Assessment Criteria Cooperative Learning Skills Attendance Listening Interaction / Participation Information Sharing Defer to Needs of the Group Self-Directed Learning Preparation Depth of Knowledge Identifies Limits of Knowledge Problem Solving Confidence Interpersonal Skills Gives instructive feedback Accepts instructive feedback Shows sensitivity and concerns for others Clarity in rationale/reasoning process Jackson, Hamlett, & Navari
Robert Herring, III Winston-Salem State U. TBL and the Business Strategy Game
Levine, R. E., Kelly, A., Karakoc, T., & Haidet, P. (2007). Peer evaluation in a clinical clerkship: Student attitudes, experiences, and correlations with traditional assessments. Academic Psychiatry, 31(1),
Who seeks formative feedback? 360 Year 3 Undergraduate Medical Students at University of Aberdeen, UK Less than half the students (46%) collected their formative feedback. Females (p=.004) and higher achievers (p=.020) were more likely to seek formative feedback. Sinclair & Cleland (June, 2007) Journal of Medical Education
Civil Engineering Design Team Criteria Attendance Punctuality Attitude Respect Preparedness Completed Readings Completed Homework Ready for Meetings Preparation Goal Oriented Flexibility Quality Work Completed the First Time Accountability Trustworthy and Reliable Gives Proper Notice if Cant Attend a Meeting Individual Initiative Team commitment Leadership Willingness to accept alternative viewpoints
Little and Cardenas (2001) Quality of Technical Work Ability to Communicate Ability to Provide Leadership Commitment to Team and Project Demonstrated Effectiveness
Helping Behavior Summative (outcome) peer evaluations provide an overall standardized assessment of the extent to which individuals contributed to team success. (Helping Behavior) Standardized base 10 n-1 X 10 (6-1)*10 = 50 pts to assign Mean of group scores Percent multiplier 100 point distribution (peer*GRAT)
Successful Teams employ self examination and assessment change basic operating assumptions when necessary monitor their own interaction patterns and progress are willing to make changes are more attentive to group processes begin their discussion by attempting to analyze the problem before trying to search for viable solutions
Session Summary I. Purpose of Peer Evaluation: A TBL Retrospective II. Characteristics of High Performing Teams III. Importance of Helpful Feedback IV. Strategies for Establishing Meaningful Formative (PROCESS) and Summative (OUTCOME) Assessment
Student-Developed Peer Evaluation Procedures & Criteria Feedbackboth formative (process) feedback and summative (helping behavior) feedback should be direct, clear, and ensure that group members are accountable to the rest of the team. The value of the process feedback is that all of the members of the team have the opportunity to discuss the feedback and make performance improvements over the course of the semester.
Hirokawa & Gourans Four Essential Functions that Decision-Making Groups Must Perform Well Ability to Analyze the Problem Ability to Identify Appropriate Criteria for Making a Decision Ability to Develop Alternative Choices from Which to Choose Ability to Evaluate the Positive and Negative Aspects of Alternative Choices Prior to Making a Decision** Order is not important but all functions must be performed well!