Presentation is loading. Please wait.

Presentation is loading. Please wait.

Welcome Today’s course Your facilitator The EQ-360 The facilities.

Similar presentations


Presentation on theme: "Welcome Today’s course Your facilitator The EQ-360 The facilities."— Presentation transcript:

1

2 Welcome Today’s course Your facilitator The EQ-360 The facilities

3 Logistics Schedule Meals Parking Phone/computer access Restrooms
Questions Workbook Feedback practice Networking contact list

4 Objectives By the end of today’s workshop, you should:
Be familiar with the EQ-360 Know its scales and their definitions Understand report components Understand the science behind EQ-360 Know how to interpret the results and provide feedback to respondents Be qualified to purchase the EQ-360

5 Objectives What are your goals for today?

6 Agenda 360° assessments Understanding the EQ-360 Psychometrics
Planning and Ethics Administration Scoring and reports Interpretation and feedback Coaching Evaluation

7 Evaluation Process Written exam After the workshop:
Interpret a sample report 60 minutes in-class today After the workshop: Administer, interpret, and provide feedback for one real-life EQ-360 assessment on a person of your choice

8 Introductions Let’s get to know each other…

9 An Introduction to 360° Assessments
Part One An Introduction to 360° Assessments

10 Part One Objectives By the end of this section, you should be familiar with 360° assessments and their Applications Advantages History Misuses

11 What are 360° Assessments? Applications Self-development
Leadership development Teambuilding Coaching Performance management Change management Stress management Career planning Predicting performance Retaining high performers

12 What are 360° Assessments? Advantages of multiple perspectives
Objective measurement of performance Provides perspective from different angles Increases awareness of work- related behaviors Improves communication between self, raters, and the organization

13 What are 360° Assessments? Advantages of multiple perspectives
Taps many sources of feedback, e.g., customers, suppliers, clients Clarifies differences of opinions Identifies performance gaps Helps with goal setting and development planning

14 History of 360° Assessment
Traditionally, lack of direct feedback hindered organizations. Authoritarian leadership was the norm, with limited supervision given to employees. After World War II, worker satisfaction surveys introduced a more democratic ideology.

15 History of 360° Assessment
The 1990s saw 360° feedback processes incorporated into corporate strategies and goals Now, most organizations believe that 360° assessments are crucial to selecting and developing the best talent for success

16 Misuses of 360° Assessment
Misuses include personnel decisions such as promotions, compensation, demotions, and termination The use of 360° assessments for purposes other than individual development has often failed

17 Misuses of 360° Assessment
Lack of feedback… Removes follow-up or development opportunities Regularly guarantees failure May discourage individuals from providing honest feedback May influence participation in future assessments

18 360° Assessments Discuss your experience using 360° (multi-rater) feedback assessments: What experience do you have with multi-rater assessments? How could the assessment have been improved? What were the strengths of the tool?

19 Questions?

20 Part Two Understanding the EQ-360

21 Part Two Objectives By the end of this section, you should be familiar with the EQ-360’s: Development Participants Rater definitions Basic assessment process

22 Development of EQ-360 First version consisted of 46 items
Detailed data analysis led to improvements Recommendations were implemented (see page 27 of the Technical Manual) and a new version was created

23 Development of EQ-360 Current version consists of 88 items
Rating scale from 1 to 5 (“Very seldom true” to “Very often true”) EQ-360 factor structure is identical to EQ-i - Total EQ - 5 Composite scales - 15 Subscales

24 EQ-360 Scales Adaptability Intrapersonal Problem Solving
Emotional Self-Awareness Assertiveness Self-Regard Self-Actualization Independence Interpersonal Interpersonal Relationship Empathy Social Responsibility Adaptability Problem Solving Flexibility Reality Testing Stress Management Stress Tolerance Impulse Control General Mood Optimism Happiness

25 Participation

26 Participation Rater confidentiality Anonymous responses
Grouped by relationship to facilitate interpretation If fewer than 3, raters go into a “mixed” category (except manager) Rater familiarity Raters are asked how well they know the ratee

27 Rater Familiarity Rater familiarity information is found on the first page of the Coach’s Summary in the Multirater Feedback Report

28 EQ-360 Rater Items

29 How the EQ-360 Works Scoring converts raw scores to standard scores
factor structure Interpretive guidelines Standard Score Guideline 130 and over Atypically well developed emotional and social functioning 120 to 129 Extremely well developed emotional and social functioning 110 to 119 Well developed emotional and social functioning 90 to 109 Effective emotional and social functioning 80 to 89 Consider developing Under 80 Develop

30 How the EQ-360 Works Discuss your experience as a ratee in the EQ-360 process: Who participated? Impressions of your EQ-360 experience? Subscale consistencies or discrepancies? Thoughts on feedback? Were there any surprises?

31 Questions?

32 An Introduction to Psychometrics
Part Three An Introduction to Psychometrics

33 Part Three Objectives By the end of this section, you should be familiar with: What the EQ-i and EQ-360 measure EQ-360 normative data Internal consistency Connections between the EQ-i and EQ-360 response patterns

34 EQ-i Refresher

35 EQ-i Refresher Validity Reliability Norms Norm Groups

36 EQ-360 Normative Data 745 Ratees… Age range from 2170
Mean age of 40.4 years

37 EQ-360 Normative Data 1900 Raters… 909 male & 969 female
22 did not report gender

38 Internal Consistency Reliability of Total EQ is .98
Inter-scale reliability for Composite Scales ranged from .91 to .95 Inter-scale item reliability for Subscales ranged from .77 to .91

39 EQ-i and EQ-360 Connections
Ratee and Rater Group Correlations Scales with significant* relationships: Emotional Self-Awareness Empathy Problem Solving Manager Peer Direct Report *Magnitude was moderate

40 EQ-i and EQ-360 Connections
Ratee and Rater Group Differences Ratee was significantly higher than raters on four scales only: Emotional Self-Awareness Empathy Problem Solving Social Responsibility Manager Peer Direct Report Other

41 EQ-i and EQ-360 Connections
Age Differences Scales with significant* relationships: Managers ̶ Peers Direct Reports Independence Self-Actualization Intrapersonal Composite *Magnitude was very small

42 EQ-i and EQ-360 Connections
Gender Differences Scales with significant* relationships: Empathy Social Responsibility Interpersonal Composite Managers F>M ̶ Peers Direct Reports *Magnitude was very small

43 EQ-i and EQ-360 Connections
What do the differences mean? Ratee may be inclined to inflate responses Context of assessment may affect responses Supervisor unfamiliarity results in inaccurate responses Differing definitions of effective emotional functioning between groups

44 Questions?

45 Part Four Planning and Ethics

46 Part Four Objectives By the end of this section, you should be familiar with: How to plan an EQ-360 assessment Getting organizational buy-in Ethical issues in administration Selecting Raters How to debrief participants

47 Purposeful Planning What is the purpose of assessment?
How will the results be used? What does the ratee know about or expect from the process?

48 Purposeful Planning Issues to Consider: Timing
Extenuating circumstances Personnel involvement How will the results be used? Methods available to confirm confidentiality Feedback Who will be facilitating development and action planning?

49 Purposeful Planning Before you begin, establish:
What the organization hopes to gain from the assessment Connections between purpose and specific business needs or goals Clear participant expectations for accountability

50 Organizational Buy-In
Senior Management support lends credibility to the process Prepare an EQ business case Educate management team on the value and relevance of the process (ROI)

51 Organizational Buy-In
Convey the importance of getting senior management involved: Consider creating a steering committee responsible for guiding the 360 process Show them how they will benefit

52 Selecting Raters Identifying who to involve
Supervisor, colleagues, direct reports, clients, and family members round out the mix Determine who gets input in the rater selection process HR? The ratee? The ratee’s manager?

53 Selecting Raters Defining roles
Will rater nominations be submitted to manager(s) and/or HR for approval or adjustment? The ratee’s manager: Will his/her responses be singled out? Will he/she be involved in skill development and coaching?

54 Selecting Raters Choose raters who: Are credible and trustworthy
Work closely with the ratee Know the ratee well For a fresh perspective, select some raters who have worked with ratee for only a short period of time.

55 Selecting Raters Tips for defining raters:
There must be at least three raters per category. If fewer than three, put raters in the Other/Mixed category to ensure confidentiality. Anyone with a supervisory influence on the Ratee is considered a Manager. If there is only one manager, discuss confidentiality issues. Include Managers, Peers and Direct Reports in the EQ-360 process. Consider clients and family depending on what outcome you want from the assessment.   There must be at least three Raters per category (Peers, Direct Reports, Clients, Family). When there are fewer than three Raters per category, move the Raters into the Other/Mixed category to provide confidentiality.

56 Ethical Issues Who is the client? Individual respondent Group
Organization

57 Ethical Issues Confidentiality EQ-360 ratees own their data
Sharing report/results and determining action steps are decided by the ratee Breaches of confidentiality and rater anonymity can jeopardize feedback and compromise the integrity of the administrator

58 Ethical Issues Ensuring Rater Anonymity
Raters feel more comfortable giving honest responses when they are anonymous If they feel their identity is unprotected, raters may respond overly favorably or be reluctant to participate

59 Ethical Issues Ensuring Rater Anonymity
Each rater category is scored together as an aggregate If there is only one rater in a category, those results will be placed in the Other/Mixed group All online responses are stored in a secure MHS scoring database

60 Ethical Issues Informed Consent Obtain before the administration
Ensure voluntary participation and freedom to withdraw from participating without punishment Provide contact information of administrator Check with your State/Provincial or Country psychology board regarding ethical issues

61 Ethical Issues Debriefing Raters
Explain purpose of the debriefing session How did they find the EQ-360 process? What were their expectations and were they met? How could the process be improved?

62 Ethical Scenarios For each of the scenarios provided, consider the following questions: What is/are the ethical issues with the scenario? What steps should be taken to ensure the privacy of participants? What are the appropriate steps that should be followed in order to rectify the issue? What resources are available in order to rectify the issue?

63 #1: Acquisition An I/O psychologist has collected extensive developmental planning data on the top 100 executives of a very large corporation. The data includes the results from the EQ-360. Participants completing the EQ-360 were promised by the Board of Directors that the results would be used for development purposes only. Now, the organization is being acquired, and the team making staffing decisions for the merged organization has requested the 360 data.

64 #2: Succession Planning
A management consultant has decided to relocate to another city and must leave behind data from a recent EQ-360 assessment used for development purposes with various managers from an Engineering company. The Engineering company indicated that it will not replace the management consultant. Instead, a HR manager will assume responsibility for the EQ-360 feedbacks, as well as the development initiatives.

65 #3: Scope Planning A consultant was hired by a firm to conduct EQ-360 assessments. The organization has not yet decided whether the assessment results will be used for selection in addition to development, or who will have access to the data. The consultant is concerned and wants to tell the ratees and raters that the firm has not yet decided how the data will be used.

66 Part Five Administering the EQ-360

67 Part Five Objectives By the end of this section, you should understand how to: Prepare EQ-360 participants Use your Scoring Organizer account Select an administration format

68 Preparing Participants
Speak with the ratee and raters before administration Discuss with the participants their right to confidentiality, feedback, and the development of skills after the assessment Participants need to know how they will benefit from the assessment

69 Preparing Participants
Brief participants on What emotional intelligence is What a 360° assessment is How the EQ-360 will be used What are the rating scales Definitions of scales

70 Preparing Participants
Discuss the importance of providing honest answers that reflect current functioning Discuss how long it will take to complete the inventory Clarify ratee and rater expectations

71 Exercise Create a correspondence to raters
Invite a group of raters to complete the EQ-360 on their coworker, Jim Smith. Your invitation should contain the following information: What the EQ-360 is Why they are being asked to complete it Reasons for participation That they will complete the assessment online What will be done with the results

72 Administration Options
Online: Link to survey website in an Web scoring Paper Packets shipped to participants Mail/fax scoring

73 Scoring Organizer You will receive an MHS Scoring Organizer account that stores your rater information online Full instructions in the online Help and Getting Started Guide

74 Scoring Organizer

75 Online Administration
Setting up the EQ-360 Each ratee must belong to a different Scoring Organizer Group Decide whether you wish to add open-ended items to the EQ-360 Raters’ short responses will appear on the Coach Report

76 Online Administration
Add a new group for each individual ratee

77 Online Administration
Add up to 7 open-ended items:

78 Mail/Fax Administration
Use the item booklets, scanable response sheets, and scoring service processing form Mail or fax forms to MHS for scoring Reports can be scored from your Scoring Organizer account

79 Administration Service
No time to handle administration and scoring? MHS can contact raters for you, track completion, and send you scored reports

80 Questions?

81 Part Six Scoring and Reports

82 Part Six Objectives By the end of this section, you should know how to: Score EQ-360 reports Differentiate the EQ-360 Client and Coach reports Decide what reports to use Find information in the reports

83 EQ-i Refresher Report Narrative Information Interpretive Guidelines
Bar Graphs Best for… Development Report Clinical with suggestions for development for lowest 3 subscales Numeric 5 areas: “Very Much Below Average” to “Very Much Above Average” Counseling and life coaching (Counselor’s Section stays with clinician) Resource Report Corporate with suggestions for development for lowest 3 subscales Numeric in Counselor’s Section only 3 areas: Enrichment, Effective Functioning, Enhanced Skills; divided by approx 1 SD Human capital development in organizations (Counselor’s Section stays with coach/HR) Business Report Corporate with suggestions for development for all 15 subscales Numeric in Coach’s Section only Development planning with SMART goal setting and activity sheets (Coach’s Section stays with coach/HR) EQ-360 Reports come with your choice of one of the above reports. Individual Summary Reports are generated separately.

84 EQ-i Business Report

85 Scoring Reports Scoring generates 2 EQ-i and 2 EQ-360 reports:
See your Getting Started with EQ-360 Guide for instructions

86 Scoring Reports Under EQ-360, select the EQ-i report type to determine the self-rated report output There is no need to score the EQ-i separately

87 Report Comparison Coach Report Client Report
Provides the administrator, coach, or employer* with in-depth response analysis, including validity and rater information Numeric cut-offs may be distracting to the ratee Not recommended for raters or the ratee Provides the ratee or employer with general response analysis and development information Sensitive language encourages positive change Not recommended for raters *Access to reports requires the consent and permission of the ratee.

88 Report Comparison Coach Report Client Report Interpretive guidelines
< 80 80-89 90-109 >130 Develop Consider Developing Effective Well-Developed Enhanced Coach’s summary section Yes No Open-ended items

89 Report Comparison Coach Report Client Report Numeric Non-numeric

90 Report Components

91 Report Components Development Strategies
Shown for up to three subscales with the highest discrepancy between ratee and rater

92 Report Components Coach’s Report Summary Rater familiarity
Scale scores by rater group (significant differences flagged) Profile graph Item response frequencies (with mean score for each item)

93 Report Components Open-Ended Items (Coach Report)
Respondents’ short descriptions give qualitative information about the ratee:

94 Questions?

95 Interpretation and Feedback
Part Seven Interpretation and Feedback

96 Part Seven Objectives By the end of this section, you should know how to: Gauge validity of responses Understand report results Provide feedback to the ratee Recommend strategies for development Deal with common challenges

97 Assessing Validity Examine the overall validity of the EQ-360 assessment with regards to: - EQ-i validity - Number of raters - Rater familiarity - Selection of raters

98 Assessing Validity EQ-i Validity Omission Rate
Positive Impression Scale Negative Impression Scale (if applicable) Inconsistency Index Ratee response style Time taken to complete the EQ-i

99 Assessing Validity Group Size
Ensure that there are enough raters in each rater group (3 rater minimum) Ensure that there is a similar number of peers and direct reports

100 Assessing Validity Rater Familiarity
Review rater familiarity chart (in the Coach’s Summary) Responses may not be accurate if raters attest to knowing the rater “Not Very Well” or “Under 1 year”

101 Assessing Validity If raters are selected by… The ratee:
Make sure that raters are objective and work with the rater in some current capacity The ratee’s manager or HR: Verify rater relationships with the ratee Manager and HR may not know all of the people with whom the ratee works

102 Understanding the Results
EQ-360 raw scores converted to standard scores Each EQ-360 scale has same mean and SD as EQ-i scales Mean = 100 SD = 15

103 Understanding the Results
Review scores for scales and items: Total EQ Composite scales Subscales Item responses

104 Understanding the Results
What are score discrepancies? Compared to EQ-i scores, are the EQ-360 scores significantly (> 10 points) higher or lower? Do the ratings of one rater group differ substantially from others?

105 Understanding the Results
If ratee scores lower than rater Indicative of underestimating own EI ability May also reflect low self-awareness or self-esteem If ratee scores higher than raters Indicative of overestimating own EI ability May reflect inability to view self critically, ego centrism/narcissism

106 Understanding the Results
Significant score differences between rater groups Indicative that ratee behaves differently with each rater group Consensus of scores between rater groups Indicative of accurate picture of ratee’s EI functioning most of the time and across situations

107 Understanding the Results
EQ-360 results should be integrated with other data and information: Interview data from ratee Additional psychological and behavioral assessments Qualitative feedback of colleagues/raters Direct observation of ratee

108 Feedback Challenges Potentially vulnerable time Defense mechanisms
Areas that need development are for strategic correction, not blame or guilt Talk about strengths Establish your role in the development process

109 Providing Feedback Set the Stage
Verify the client’s understanding of the purpose of the assessment Discuss how the results will be used Assure the client of confidentiality

110 Providing Feedback Explain the Feedback Process
Ask when and where the client completed the assessment Point out that scores are a basis for discussion Discuss the reasons why there are no numbers on the report

111 Providing Feedback Discuss the report
Validity: explore what motivated the client’s answers Select subscales you want to discuss Ask the client what he or she thinks Draw connections between subscales Identify patterns between rater groups

112 Case Studies Feedback Report practice interpretation
Refer to case studies in the appendix of your workbook

113 Case Study #1 Justin 33-year-old Customer Service manager
EQ-360 to aid in professional development Referred by manager, who is grooming Justin for a middle management position

114 Case Study #2 Linda 34-year-old Project Manager
EQ-360 to aid in transition to new role Problems with staff management Referred by manager to HR

115 Case Study #3 Amalie 49-year-old NFP CEO
EQ-360 given to all executives to identify leadership gaps Amalie’s task-oriented work style differs from her sociable coworkers

116 Case Study #4 Christopher 48-year-old commercial airline pilot
Referred by HR due to co-pilot complaints Concern over communication and following logbook protocol

117 Case Study #5 Bill 49-year-old sales manager
Referred by VP due to interpersonal problems Concern over lack of staff commitment

118 Case Study #6 Joan 55-year-old university department chair
Directing a new department committee Instigated committee-wide EQ-360 for teambuilding and performance management Wishes to rekindle dedication and teamwork

119 Questions?

120 Coaching and Development
Part Eight Coaching and Development

121 Part Eight Objectives By the end of this section, you should know how to: Help the ratee create a personal development plan Assess the organizational factors that affect the ratee’s progress Reassess for success

122 Analyzing Development Strategies
Strategies appear for the three subscales with the greatest discrepancy between Ratee and the All Rater Group Choose the strategies that are most appropriate to improving work performance

123 Analyzing Development Strategies
Discuss accessible resources and support systems to help leverage client strategies Encourage the use of development strategies on the job

124 Creating a Personal Development Plan
Develop a plan of action with clear goals and timelines The goal is lasting behavioral change Who, when, how: Who will be involved When will it begin How it will happen

125 Creating a Personal Development Plan
Pick one or two subscales that are relevant to job performance Record the current level of behavior in these subscales List how the attainment of goals will be beneficial to the organization

126 Creating a Personal Development Plan
List internal and external factors that will Enhance your EQ performance Hinder your EQ performance Schedule regular feedback from peers, direct reports, and manager

127 Exercise Create a personal development plan
Jim Smith’s results show that he is low on the Interpersonal Relationship scale, which is important to his job as an engineering supervisor. His plan should contain the following information: The end goal How development will benefit the organization Who, when, how Timelines Factors that will help or hinder progress

128 Follow-up Coaching Summarize prior meeting
Check whether progress towards goals has been made When a goal is met… Congratulate! Choose another EQ-i area to focus on Set new goals

129 Follow-up Coaching When a goal is not met…
Look to see if goals were unrealistic Set new goals with new timelines and action steps Find additional resources to achieve revised goals

130 Measuring Outcomes Begin with organizational objectives
Identify a job-specific competency model with detailed skills and behaviors Quantify performance outcomes such as customer satisfaction, leadership, sales, profit, or deadline adherence Establish cut-offs for poor performance

131 Measuring Outcomes What EQ-i areas are most important to organizational and individual success? What competencies will you investigating? Individual? Organizational? How do the EQ-360 results map to the strategy or needs of the organization?

132 Reassessment to Gauge Progress
Repeat assessment as a measurement of outcomes Annual assessment is the norm but more frequent assessments may be needed Compare effectiveness over time Do performance measures need to change?

133 Questions?

134 Wrap Up Review

135 Goals Scoring and reports 360° assessments Interpretation and feedback
Coaching Exam 360° assessments Using the EQ-360 Psychometrics Planning and ethics Administration

136 Interpret a sample report (60 minutes)
Exam Interpret a sample report (60 minutes)

137 EQ-360 Assessment Complete a EQ-360 administration, interpretation, and feedback on a client of your choice Provide a detailed written description of the following (include copies of forms and letters used if any): How you addressed informed consent How you set up the administration How you conducted your initial interpretation Describe the feedback session(s) Describe any additional plans discussed with the client

138 EQ-360 Assessment Required for qualification Format & length Due date
Submission

139 Please fill out an evaluation form How to contact me
Thank You Please fill out an evaluation form How to contact me


Download ppt "Welcome Today’s course Your facilitator The EQ-360 The facilities."

Similar presentations


Ads by Google