1. Appraisal Techniques : Contemporary Approaches to appraisal for developmental purposes M21 : Assessment in the Workplace Dr Caroline Bailey

2. Overview Multi-source multi-rater (MSMR) assessment (i.e. 360-Degree Feedback) –themes in research –practical issues Issues in Evaluating Others –cognitive approach : theory and research findings Issues in Being Evaluated –Theoretical model (London & Smither, 1995) Future Research

3. Contemporary approaches to appraising employees for developmental purposes Use mid 1980s : 10% USA, 0% UK; 1990s : $152 million in USA; IPD survey : sig increase in UK Why ? Increasing awareness of limits of traditional appraisal methods Need for cost effective alternative to ACs Increased availability of suitable software Need for continuous measurement of improvement efforts Need for job-related feedback (particularly those at career plateau) Need to maximise employee potential

4. Perspectives on developmental feedback processes Organisations Perspective facilitate culture change particularly useful for executive development reinforce competency framework/business values succession planning legal defensibility of assessments Individuals (targets) Perspective determine strengths and weaknesses determine others perceptions (and where mismatch) Co-workers Perspective means to provide negative feedback

5. The Appraisal Process Typical Feedback Questionnaire Based on competency framework/mission statement 40-60 questions, rating of current level of effectiveness + qualitative (free comments) section ratings provided (anonymously for multiple rater groups); self- assessments + line manager, peers, reports, clients, others The Process target identified (volunteers) target nominates raters (min 3, average 6-8) all raters complete feedback questionnaires ratings collated, typically averaged where more than one rater in rating group feedback report produced feedback interview + action planning

6. Generic Cognitive Model for Performance Evaluation Observe behaviour –sort relevant from irrelevant information Encode information about behaviour –mental representations of behaviour which is is not necessarily a faithful replication of what they have seen Store information –transition from short-term to long-term memory Retrieve information –dependent on how often Performance appraisal process run, demands on memory can be substantial Integrate information –what info is retrieved has to be integrated with info. from other sources as well as other time periods.

7. Information Acquisition Is active Cognitive research on focus of attention –the behaviour itself –the context of observation –the purpose of observation Encoding & Mental Representation Categorisation depends upon the similarity between the target and each of the categories available to the rater Issues : –How are similarity judgements made? –How do categories become more/less available to the rater ? –How does content of categories change ?

8. Storage & Retrieval Short-term memory (STM) and long-term memory (LTM) are distinguishable in terms of –duration –capacity Tulving (1983) : 2 types of memory systems - –semantic : storage for verbal, factual and propositional information –episodic : storage for actions, occurrences, and experiences (direct and vicarious) Recognition, Recall and Reprocessing Recognition memory is typically (but not always) better than recall Murphy and Davidshofer (1988) : PA rating scale used Memory aids (work diaries) Assimilation vs Contrast effects

9. Factors influencing others judgements Demographic Factors : gender (e.g. Schmitt & Lapin, 1980) race (e.g. Schmidt & Johnson, 1973) age (e.g. Cleveland & Landy, 1981) education (e.g. Cascio & Valenzi, 1977) Psychological Factors Personality, Cognitive variables (cognitive complexity), rater motivation, rater goals Job-Related Factorssee Landy, 1983 Job experience Job performance level Leadership style Organisational level Knowledge of job requirements Amount and type of rater contact

10. Example : Interpreting a Feedback Report –Strengths –Weaknesses –Self-Assessment in relation to others –Agreement/Disagreement between raters –Determining a personal development plan Issues –Sell & Tell –obstacles to accurate self-assessment –individuals responsiveness to feedback (Ilgen, Fisher & Taylor, 1979; London & Smither, 1995)

11. Current Themes in 360-feedback research Psychometric properties of ratings from different sources System methodology and its impact on (a) psychometric properties (b) impact of feedback Behaviour change following feedback

12. Does it work? Fletcher, C., Baldry, C. & Cunningham-Snell, N. (1998) : Case study of pilot form of 360-questionnaire Pilot : 27 mgrs (+ 18 bosses, 99 subs & 86 peers) provided ratings for 80 questions (3 dimensions). Looked at : distribution of ratings item discrimination index inter-rater agreement factor structure of questionnaire relationship to external measure of potential Led to revisions being made...

13. Other determinants of utility Purpose London & Beatty (1993) : leniency if for formal appraisal (Garavan et al, 1997 - not for merit) Confidentiality/anonymity no research evi. to date, but would expect less candour Choice of raters ease for system administration; issue of friendship (two-tailed?) Frequency/timeliness of feedback ISSUES : high admin., time for developmental activity Averaging of responses & nature of feedback provided averaging protects anonymity and ameliorates idiosyncratic rating errors BUT lose sensitivity of data (e.g. bimodal distribution); ISSUE 2 : how best to present feedback

14. Determinants of Utility (II) Harris & Schaubroeck (1988) : level of convergence moderated by target employees level within org. Ashford (1989) : higher up org., less likely to receive very negative feedback. Yammarino & Dubinsky (1990) : congruence greater in supportive climate Ashford (1989) : extent to which an individual attends to feedback may be moderated by extent to which organisation (and/or the individual) going through a period of change

15. Some general conclusions of 360-research to date Ratings from different sources can have different psychometric properties, but are largely comparable. 2/3rds of focal individuals benefit from participation in feedback Most people (raters and recipients) regard 360-feedback as a positive process, if used for developmental purposes only. Focal individuals do not regard feedback from all sources as equally credible; different things influence the credibility of different raters The characteristics of feedback received do NOT fully account for individuals reactions and subsequent changes in behaviour following feedback Participation in feedback can have a number of other effects, aside from performance.

16. My recent research in 360-feedback Meta-analysis of the psychometric properties of ratings from different sources (self vs boss vs peer vs reports) How 360-feedback affects focal managers self-efficacy Whether participating in 360-feedback results in (a) reduction in development needs (b) improvement in performance The influence of 360-feedback on goal setting & goal attainment Factors associated to the credibility of a rating source * Characteristics of feedback versus salience of feedback as a determinant of intent to change behaviour * Gender differences in boss vs peer vs reports developmental 360-ratings * Gender differences in the experience of 360 feedback * Cross cultural differences in 360-ratings

17. Efficacy of MS/MR appraisal as a developmental technique London & Smither, 1995 Factors Influencing Targets Reactions

18. Potential Limitations of 360- degree feedback from Moses, Hollenbeck & Sorcher (1993) : –generalised traits - limited frame of reference for making judgements –ratings based on memory (correct inference from behaviour) –participants interpretation influenced by format of feedback report for presenting feedback –Other Peoples Observations vs Other peoples expectations VanVelsor & Wall (1992) : ipsative ratings Kaplan (1993); London et al (1990) : pressure on target Kaplan (1993) : survey fatigue –others...Time consuming, Expensive Importance of providing follow up...

19. Research is yet to establish…. Individuals perspective Motivation Goal-setting Self-esteem, self-image, etc. Organisations Perspective facilitate culture change utility for executive development reinforcement of competency framework/business values succession planning facilitate organisational commitment & from both individual & organisational perspectives negative outcomes (conflict, demotivation, etc.)