1. Peer Evaluation of Teaching Portfolios John Jones, EDU x6320 etjjones@ K.P. Kwan, EDU x6287 etkpkwan@ EDU Lunchtime Session 29 May 2000

2. 2 Session outline zProblems of evaluating teaching zTeaching portfolio: what it is and why it is preferred? zA suggested framework for peer evaluation of teaching portfolios  Components, criteria and evidence zGuidelines for peer evaluation of teaching portfolios zSuggestions and comments

3. 3 Problem of evaluating teaching zA perfectly precise, valid and objective measure will never exist because  teaching is a complex activity  teaching has multiple goals  effectiveness of teaching depend on goals, teachers, students, and other contextual factors  there is no single ‘best’ way to teach but rather, different approaches to good teaching

4. 4 Good teaching evaluation system zMust be seen to be fair, valid, and credible by staff to be evaluated zProvides incentives and feedback for staff that promote continuous improvements in teaching zProvides useful information for educational administrators for making informed decisions

5. 5 Two things to note zStandardised, objective measure  fair and valid zImportant to consider the consequential validity of the evaluation  Whether the evaluation leads to desirable consequences  Criteria for evaluation reflects the implicit goals/definition of good teaching of the institution  What and how to evaluate  teaching behaviours  effect on teaching that may or may not be beneficial to learning

6. 6 Most indicators are imperfect zUniversity-wide SFQ35.6% zRecommendations by HoD34.9% zFaculty-based SFQ42.9% zPeer ratings of classroom teaching 32.1% zPeer review of learning materials58.4% zExternal examiner’s report 35.5% zMeasures of student learning outcomes35.7% zPublication on teaching & learning40.3% zIndependent observer ratings40.7% zParticipation in LTD projects42.3% zSelf-constructed SFQ49.8% Source: EDU survey (1999)

7. 7 zEvaluation via peer assessment of teaching portfolio is preferred by most PolyU staff  74.3% favoured “a portfolio of evidence from various sources compiled by the staff member” (Source: EDU survey, 1999) zTriangulation of evidence from multiple sources enables a more valid evaluation Portfolio: a preferred approach

8. 8 The teaching portfolio Database of evidence and examples Personal teaching practice Objectives Innovations & changes Student feedback Syllabus Assessment guide Peer feedback Students’ work Teaching duties LTD Projects Teaching portfolio Selection of evidence, Reflection, Action plan. Generate in normal course of teaching According to specified criteria

9. 9 Components zA short reflective document (5-10 pages) created by the staff member which presents evidence from different sources on different aspects of his/her teaching zCommon components  Teaching goals, objectives or philosophy  Approaches & strategies for teaching, advising and supervision  Evidence of teaching xpreparing for teaching; interacting with students assessing students; other teaching-related activities  Reflection on evidence and planning for improvement

10. 10 Providers of information zSelf  Samples of learning materials developed for students  Participation in ongoing professional development, teaching development projects, innovative teaching, etc. zStudents (past and present)  Feedback on teaching, supervision, consultation and learning zPeers  Feedback on curriculum, teaching, assessment, and materials zOther parties  External examiner’s report on student performance  Employer’s satisfaction  Public recognition / awards

11. 11 Evaluators of teaching zSelf  Self evaluation of the teacher is the basis of any attempt to improve zHoD / Peers / Personnel committees  Evaluation by peers is generally more credible especially when xthe peers are respected and trusted xcriteria & standards are shared and clearly-defined xthe process is transparent and understood by all

12. 12 Need for a clear criteria zA clear and commonly agreed-upon set of criteria is needed:  For staff members to select evidence and produce their portfolio  For evaluators to ‘assess’ the portfolio

13. 13 Criteria for evaluating portfolios zExamine the suggested framework and discuss:  Are the criteria for each of the components clear, reasonable, appropriate?  Are there any needs for any of the criteria to be added, modified, or discarded?  What evidence would you include/look for to support/verify the claims for having excelled in the criteria for each of the components?

14. 14 Some issues for further discussion zTo what extent can we trust what has been put down in the portfolio as true representation of competence, efforts, and commitments? zWhat should be the focus:  Competence or continuous improvement?  Evidence of success or commitment to teaching? zHow should the various components and evidence be ‘weighed’ to arrive at the overall evaluation? zCan, and should, the evidences enable one to make fine distinctions among teachers in their teaching performance?

15. 15 Some guidelines zNegotiate and agree on criteria and weighting within department zUse multiple judges (  3) for important personnel decisions zMake holistic judgements in terms of broad categories, avoid numbers or ranking (e.g. outstanding, very good, basically competent, needs much improvement) zExamine consistency between components, and if there are evidences to support claims zEmphasise achievement as well as commitment; competence as well as continuous effort to improve zConsider stage of career and contexts (students, level, class size, subject nature, etc.)

16. 16 Comments & suggestions zSend further comments and suggestions to:  John Jones: etjjones@polyu.edu.hk (ext. 6320) or  K. P. Kwan: etkpkwan@polyu.edu.hk (ext. 6287)