1. Stephen J. Porth, Ph.D. for George P. Sillup, Ph.D., M.S. CJBE Conference July 28-30, 2006 A Qualitative Exploration of the Relationship between Performance Appraisal and the Development of Inspired Business Leaders

2. CJBE - July 28-30, 2006Sillup/Porth - 2 Key Findings PA seen as a way to gain recognition for performance and as a link to becoming an IBL Experienced (mean 16.6 years) respondents –Were successful under their PA System - exceeded their objectives –Some problems with bias - none as raters but two when being rated Not enough time to implement –Vast mast majority (15 of 16 respondents) indicated that they did not have enough time to implement the PA system No formal training –None of the respondents received formal training about the PA systems they were using Must go beyond objective measure to develop IBLs –PA System must also introduce the employee to aspects of performance that go beyond the objective measures evaluated in results-oriented PA systems

3. CJBE - July 28-30, 2006Sillup/Porth - 3 Research Questions Does implementation of an effective Performance Appraisal (PA) system help to develop Inspired Business Leasers (IBLs)? What are the key impediments to using a PA system? This includes impediments before and/or during implementation? How can implementation be improved? What are the implications of this research for the development of IBLs? The author wishes to acknowledge the financial support from the Saint Josephs University Pedro Arrupe Center for Business Ethics and the intellectual guidance from John McCall and Stephen J. Porth

4. CJBE - July 28-30, 2006Sillup/Porth - 4 What Is Performance Appraisal (PA)? Systematic description of the job-relevant strengths and weaknesses of an employee or an employee team Combines two processes, observation and judgment, to assess job performance on the basis of objective and subjective indices –Objective indices are appealing because they measure a quantifiable endpoint but often measure factors beyond an employees control or outcomes of behavior (e.g., number of units sold) –Subjective indices measure the job performance behavior (e.g., what did it take to sell the units)

5. CJBE - July 28-30, 2006Sillup/Porth - 5 PA and Its Influence on IBLs PA can improve employees work performance by helping them to realize their full potential as IBLs PA accomplishes this by providing information to employees and managers for use in making work- related decisions Because PA is an inexact human process, it may only effectively evaluate how employees meet objective performance measures –e.g., assessing the number of units sold PA may fail to evaluate attributes that lead to improved performance – e.g., integrity or other attributes especially important to the development of IBLs

6. CJBE - July 28-30, 2006Sillup/Porth - 6 For a Positive Influence on IBLs PA must take steps to ensure preparation for implementation and implementation addresses the following areas: – Setting fair performance standards – Meeting requirements of effective PA systems – Assessing the types of PA systems/determining which one to use – Identifying who should rate performance – Overcoming judgmental biases in rating – Establishing training programs for raters – Determining how often PA should be done – Understanding how PA fits with Total Quality Management

7. CJBE - July 28-30, 2006Sillup/Porth - 7 Research Method Structured interview over 30-45 minutes to assess how the shortcomings of PA potentially impact the development of IBLs 10 questions - 9 designated areas and 1 open-ended question: 1.Does your PA system have clear PS? 2.How well does your PA system meet the requirements of an effective PA system? 3.What type of PA system does you company use? 4.Who rates PA? 5.Are you aware of any biases in rating? 6.Have you had rater training? When? 7.How often is PA conducted? 8.Is your company doing some variation of TQM? If so, how does PA fit with TQM? 9.Have you encountered any barriers to PA? 10.Are there any other factors that help/hinder implementation of PA? Full research instrument available in back-up slides

8. CJBE - July 28-30, 2006Sillup/Porth - 8 Results – Profile of Respondents CompaniesNumber of Respondents (Gender) Departments/Years Experience Aetna 4 (3F, 1M)Marketing = 2 (7, 10) HR = 2 (20, 18) IBM 2 (2F)HR = 1 (22) CS = 1 (11) J&J 6 (1F, 5M)Finance = 2 (15, 17) Marketing = 2 (19, 23) Sales = 2 (9, 15) Wyeth 4 (2F, 2M)R&D = 1 (24) RA = 2 (16, 20) Finance = 1 (20) Total 16 (8F, 8M)Marketing = 4 Finance = 3 HR = 3 Sales = 2 RA = 2 CS = 1 R&D = 1

9. CJBE - July 28-30, 2006Sillup/Porth - 9 Analysis of Structured Interviews Revealed use of results-oriented PA systems on an annual basis Similar responses from the four companies for both the male and female respondents –Summary on table in next slide Top-down support was identified as most important for the successful implementation of the PA system No matter the department, all respondents had their PA conducted by their immediate managers and used multi-rater input (360- feedback) 10 of the 16 used self-appraisal as preparation for PA 2 different responses –Wyeth for TQM because they were the only company with a respondent from Manufacturing, the only department implementing TQM –Aetna where four customers were asked for feedback.

10. CJBE - July 28-30, 2006Sillup/Porth - 10 Results – Frequency of Responses Interview QuestionsResponses as RaterResponses as Ratee 1.) Clear PSYes = 16 2.) Meets requirements of effective PA system Yes = 16 3.) Type of PA systemBehavior-Oriented = 0 Results-Oriented = 16 with non-objective performance attributes included Behavior-Oriented = 16 Results-Oriented = 16 4.) Who rates PAImmediate Manager = 16 Peers = 0 Direct Reports = 0 Self-appraisal = 10; employee preparation for PA at J&J and Wyeth Customers = 4 at Aetna Multi-Rater = 16; 360˚ Computer Monitoring = 0 Immediate Manager = 16 Self-Appraisal = 10 as preparation Multi-Rater = 12 5.) Bias in ratingNone observed = 16No = 16 but two suspected bias in their reviews 6.) Rater trainingYes = 14; informal from their managers No = 2; new hires at company N/A 7.) How often conductedAnnually = 16 with mid-year update 8.) TQM fit with PAYes = 2; Manufacturing at WyethYes =1; Manufacturing at Wyeth 9.) Behavioral barriersNo = 16; all noted the importance of top-down support No = 16 10.) Other factorsNot enough time = 15 Number responses number respondents N = 16

11. CJBE - July 28-30, 2006Sillup/Porth - 11 Results Summary Taken as a whole, the interviews were positive –Regarding PA as a way to gain recognition for performance –Suggesting it as a link to becoming an IBL None of the respondents gave me their individual PA rating but they said they exceeded the annual objectives None of the respondents suspected bias when they were raters but two suspected bias (political) when they were being rated by their managers Vast mast majority (15 of 16 respondents) provided unsolicited responses to Question #10 (Other) and indicated that they did not have enough time to implement the PA system None of the respondents received formal training about the PA systems they were using –2 received no informal training because they were new hires with considerable experience in conducting PA –Those interviewed were experienced (mean of 16.6 years) Most inexperienced respondent had 7 years experience

12. CJBE - July 28-30, 2006Sillup/Porth - 12 Must Go beyond Effective PA System to Inspire IBLs Results reinforce that effective implementation of a PA system is an optimal way to assess an employees performance but…………………………………………… Development must also introduce the employee to aspects of performance that go beyond the objective measures evaluated in results-oriented PA systems if the PA system is to help develop IBLs These include the performance attributes added to complement the behavior-oriented PA systems being implemented by the companies in this study

13. CJBE - July 28-30, 2006Sillup/Porth - 13 Performance Attributes Added to Behavior-Oriented PA Systems Examples of this can viewed in three of the four companies participating in this study –Johnson & Johnson –IBM –Aetna They integrated concerns for employees rights into the very fabric of their PA systems and corporate cultures Despite more recent ethical concerns (e.g., IBM connected to the Holocaust) –Both companies receive strong employee approval for their policies on company attitude surveys

14. CJBE - July 28-30, 2006Sillup/Porth - 14 Allowing Enough Time to Implement PA System is Necessary for Success Enough time to implement a PA system seems to be a determinant for the success of the system Furthermore, when time is constrained –Raters revert to familiar objective measures, such as financial metrics, and tell and sell approach rather than an information exchange –Raters also become more vulnerable to biases, (e.g., recency effect) Although most companies require that PA results be discussed with employees, they do not consider PA as an individual objective –Consequently they miss an opportunity to help improve their PA system and to establish the basis for development of IBLs

15. CJBE - July 28-30, 2006Sillup/Porth - 15 Implementation Practices - Need to be Continued and Started Continue: Annual reviews with a mid-year update –Feedback has maximum impact when given as close as possible to the action Self-appraisal is very useful –Employees who prepare are more satisfied with the PA process and more likely to meet their personal objectives Encourage Participation –Managers welcome employees ideas Start: Training for Raters –Emphasize how to observe behavior more accurately and fairly and not on how to rate or not rate employees performance. Set PA as an Objective –Objectives that are set emphasize the importance of meeting the objective

16. CJBE - July 28-30, 2006Sillup/Porth - 16 Implications for Further Research More companies and more raters/ratees from different departments with less experience –Provide the basis for quantitative assessment of these qualitative observations Larger, more robust dataset would enable other interesting observations –e.g., how managers who exceed their annual objectives rate their employees Consider how PA systems are working globally –Differences between the US and Europe and, especially, Pacific-Rim countries N.B. Contributors to the content of this paper include employees of Aetna, IBM, Johnson & Johnson and Wyeth as well as Susan Givens-Skeaton for her input and introduction to Wayne Casios research and Christopher Field for his critical review.

17. CJBE - July 28-30, 2006Sillup/Porth - 17 Connectedness of PA System, Top Management and IBLs PA System for top management Inspired Business Leaders Balanced Scorecard No correct PA system to stimulate development of IBLs but one alternative that enables a business to manage by values is the Balanced Scorecard

18. CJBE - July 28-30, 2006Sillup/Porth - 18 Balanced Scorecard Creates Environment for IBLs Financial Measures Owners and Stakeholders View Customer Perspective Customers View Future Perspective Continue Internal Perspective Employees View Balanced Scorecard Adaptation of Source: S. Porth, Strategic Management, A Cross-Functional Approach, 2006

19. CJBE - July 28-30, 2006Sillup/Porth - 19 Thank You Questions

20. CJBE - July 28-30, 2006Sillup/Porth - 20 Back-Up Slides

21. CJBE - July 28-30, 2006Sillup/Porth - 21 Research Instrument 1.)Does your PA system have clear PS? 2.)How well does your PA system meet the requirements of an effective PA system? –Relevance – identify what makes a difference –Sensitivity – determine effective/ineffective performance –Reliability – platform for consistent judgment –Acceptability – have the support of those using it –Practicality – easy to understand 3.)What type of PA system does you company use? –Behavior-Oriented Narrative essay Ranking Checklist Critical incidents Graphic rating scale –Results-Oriented Management By Objectives Working Plan and Review

22. CJBE - July 28-30, 2006Sillup/Porth - 22 Research Instrument (continued) 4.)Who rates PA? –Immediate Manager –Peers –Direct Reports –Self-appraisal –Customers –Multi-Rater (360˚-feedback) –Computer Monitoring 5.)Are you aware of any biases in rating? –Leniency and severity –Central tendency –Halo effect –Contrast error –Recency error 6.)Have you had rater training? When? 7.)How often is PA conducted?

23. CJBE - July 28-30, 2006Sillup/Porth - 23 Research Instrument (concluded) 8.)Is your company doing some variation of TQM? If so, how does PA fit with TQM? 9.)Have you encountered any barriers to PA? –Lack of top-down support –Lack of employee buy-in –Political barriers –Interpersonal barriers 10.)Are there any other factors that help/hinder implementation of PA?