1. Teacher appraisal and job satisfaction an exploration based on the Teaching and Learning International Survey (TALIS)- database 35th Annual Conference of ATEE August 26-30, 2010 Hungary, Budapest

2. 1 Legislative framework 01/09/04: appointment > 104 days  indiv. job description: - in consultation with staff member - duties and tasks related to teaching, school-level tasks, duties and responsibilities related to professional development, personal and developmental objectives 01/09/07: teacher appraisal - legislative framework with minimal requirements; school(communitie)s enter into further agreements - responsibility of principal - process of evaluation and guidance starts with/is based on job description (= objectieve toetssteen) - proces of evaluation = constructive, focussed on staff development (guidance / support) = constructive and positive instrument of personnel management

3. 2 Background Recent research in Flanders (Tuytens & Devos, 2009a en 2009b) provides evidence: - teachers and principals are positive regarding the principle of the implementation of job descriptions and teacher evaluation - teachers are more sceptical regarding the ways in which the evaluations will be implemented in their school - Importance of the perception of principals and teachers may not be underestimated when it comes to succesful evaluations Purpose of paper: exploring if and how characteristics of teacher appraisals/evaluations influence job satisfaction

4. 3 Literature Idea that job satisfaction could be influenced by the appraisal process, has already been raised: Organisational perspective: positive relationship between experiences with appraisals and organisational commitment of the employee, and to the positive relationship between the latter and job satisfaction (Youngcourt, Leiva & Jones, 2007) Human resources/field of education: - clarity of appraisal criteria is related to higher job satisfaction (Pettijohn, Pettijohn & d’Amico, 2001; Kelly, Ang, Chong & Hu, 2007; Conley, Muncey & You, 2005) - p erceived developmental purpose yields direct relationships with job satisfaction. (Youngcourt et al., 2007) - (perception of) fairness of the appraisal relates positively to job satisfaction (Pettijohn et al., 2001; Kelly et al., 2007)

5. 4 Research question + hypotheses  The question is if and how characteristics of teacher appraisals/evaluations influence job satisfaction  3 hypotheses: (a) appraisals held in a developmental perspective have a positive impact on job satisfaction (b) appraisals perceived as being a fair judgement impact positively on job satisfaction (c) transparency and/or clarity of the appraisal system have a positive impact on job satisfaction

6. 5 Research design (1) Teaching and Learning International Survey-database International comparative educational research (OECD): 24 countries Focus on working conditions of teachers and learning environment -Recognizing teachers, teacher appraisal and rewarding teachers -School leadership -Teaching practices, teachers’ attitudes and beliefs -Professional development of teachers Flanders: participation of 203 schools with over 3500 teachers Strict, high quality standards (f.e. sampling, data collection, response rate, …)  Results are representative for Flanders

7. 6 Research design (2) Variables used  operationalisation of the hypotheses: (a) (developmental) perspective of the appraisal system: “I think the appraisal of my work and/or feedback received was helpful in the development of my work as a teacher in this school.” (strongly disagree-disagree-agree-strongly agree) (b) fairness of the appraisal (system): “I think the appraisal of my work and/or feedback received was a fair assessment of my work as a teacher in this school.” (strongly disagree- disagree-agree-strongly agree); (c)clarity of the appraisal criteria: “In your opinion, how important were the following aspects considered to be when you received this appraisal and/or feedback?” (I do not know if it was considered-not considered at all-considered with low importance- considered with moderate importance-considered with high importance)

8. 7 Research design (3) Job satisfaction (JS)  measured twice in TALIS: “JS1”  teachers’ satisfaction with a certain circumstance (being the appraisal and/or feedback received): “Concerning the appraisal and/or feedback you have received at this school, to what extent have they directly led to changes in your job satisfaction?” (a large decrease-a small decrease-no change-a small increase-a large increase) “JS2”  overall construct: “All in all, I am satisfied with my job.” (strongly disagree-disagree-agree-strongly agree) Extra control variables ► Multilevel logistic regression-analyses ◄

9. 8 Perspective: developmental or not Nature: fair or not Degree of clarity/transparency + CONTROL VARIABLES f.e. age, employment status, gender, highest level of formal education, # years of teaching experience, educational network JS1 – JS2 PREDICTORS 3 ≠ models tested: Null model Net model Full model Multilevel logistic regression-analyses

10. 9 Extra information regarding variables: Most variables: dummy variables or dummified clarity/transparancy of appraisal criteria: counting value ‘I do not know if it was considered’ across all appraisal criteria; afterwards the variable was standardised and centred around the grand mean (In other words, the bigger the value on this variable, the less clear the appraisal/feedback) age: centred around the grand mean

11. 10 Results (1) For each of both dependent variables (JS1 en JS2): estimated logistic regression coefficient (and standard error) Statistical significant effects (printed in bold) for: modelJS1: satisfaction with the appraisal and/or feedback received: Net‘fair’: 1.28 (.20) ‘useful in development of work’: 1.02 (.14) ‘transparant/clear’: -.12 (.04) FullIdem: 1.28 (.23),.98 (.18) en -.11 (.05) ‘judgment about quality of work’:.73 (.12) remaining (control)variables: n.s.

12. 11 Results (2) modelJT2: algemene jobtevredenheid Net ‘fair’:.71 (.23) ‘useful in development of work’:.81 (.21) Full ‘fair’:.54 (.25) ‘useful in development of work’:.91 (.23) ‘transparant/clear’: -.15 (.07) remaining (control)variables: n.s.

13. 12 Discussion of results (1) strong evidence for both hypothesis a and hypothesis b: appraisal/feedback with a developmental purpose and appraisal/feedback as a fair assessment have a positive impact on job satisfaction (JS1 and JS2): Teachers who indicate the appraisal and/or feedback was helpful in the development of their work as a teacher have a significantly higher probability of experiencing an increase in job satisfaction (JS1)/ of being satisfied with their job (JS2). (cf. Youngcourt et al. 2007) Teachers who indicate that the appraisal/feedback was a fair assessment of their work as a teacher, have a significantly higher probability of experiencing an increase in job satisfaction (JS1)/ of being satisfied with their job (JS2). (cf. Pettijohn et al., 2001; Kelly et al., 2007)

14. 13 Bespreking resultaten (2) Results relating to hypothesis c are less univocal: JS1: (positive) effect of transparancy and/or clarity of appraisals in both models JS2: no significant effect in net model  confident that our results do not refute this hypothesis; analysis supports findings of Pettijohn et al.(2001), Kelly et al. (2007) and Conley et al. (2005) that clarity of appraisals adds to job satisfaction of teachers

15. 14 Final conclusion Research results are relevant for daily practice (i.c. designing the evaluation process within the school), and consitute relevant input for policy at macro level. Remark: appraisal system of educational staff is currently being evaluated in Flanders  OBPWO 09.03 (01/04/10 – 30/09/11) END OF PRESENTATION Questions? Peter.vanpetegem@ua.ac.be