4 Objectives By the end of today’s workshop, you should: Be familiar with the EQ-360Know its scales and their definitionsUnderstand report componentsUnderstand the science behind EQ-360Know how to interpret the results and provide feedback to respondentsBe qualified to purchase the EQ-360
6 Agenda 360° assessments Understanding the EQ-360 Psychometrics Planning and EthicsAdministrationScoring and reportsInterpretation and feedbackCoachingEvaluation
7 Evaluation Process Written exam After the workshop: Interpret a sample report60 minutes in-class todayAfter the workshop:Administer, interpret, and provide feedback for one real-life EQ-360 assessment on a person of your choice
9 An Introduction to 360° Assessments Part OneAn Introduction to 360° Assessments
10 Part One ObjectivesBy the end of this section, you should be familiar with 360° assessments and theirApplicationsAdvantagesHistoryMisuses
11 What are 360° Assessments? Applications Self-development Leadership developmentTeambuildingCoachingPerformance managementChange managementStress managementCareer planningPredicting performanceRetaining high performers
12 What are 360° Assessments? Advantages of multiple perspectives Objective measurement of performanceProvides perspective from different anglesIncreases awareness of work- related behaviorsImproves communication between self, raters, and the organization
13 What are 360° Assessments? Advantages of multiple perspectives Taps many sources of feedback, e.g., customers, suppliers, clientsClarifies differences of opinionsIdentifies performance gapsHelps with goal setting and development planning
14 History of 360° Assessment Traditionally, lack of direct feedback hindered organizations.Authoritarian leadership was the norm, with limited supervision given to employees.After World War II, worker satisfaction surveys introduced a more democratic ideology.
15 History of 360° Assessment The 1990s saw 360° feedback processes incorporated into corporate strategies and goalsNow, most organizations believe that 360° assessments are crucial to selecting and developing the best talent for success
16 Misuses of 360° Assessment Misuses include personnel decisions such as promotions, compensation, demotions, and terminationThe use of 360° assessments for purposes other than individual development has often failed
17 Misuses of 360° Assessment Lack of feedback…Removes follow-up or development opportunitiesRegularly guarantees failureMay discourage individuals from providing honest feedbackMay influence participation in future assessments
18 360° AssessmentsDiscuss your experience using 360° (multi-rater) feedback assessments:What experience do you have with multi-rater assessments?How could the assessment have been improved?What were the strengths of the tool?
21 Part Two ObjectivesBy the end of this section, you should be familiar with the EQ-360’s:DevelopmentParticipantsRater definitionsBasic assessment process
22 Development of EQ-360 First version consisted of 46 items Detailed data analysis led to improvementsRecommendations were implemented (see page 27 of the Technical Manual) and a new version was created
23 Development of EQ-360 Current version consists of 88 items Rating scale from 1 to 5 (“Very seldom true” to “Very often true”)EQ-360 factor structure is identical to EQ-i- Total EQ- 5 Composite scales- 15 Subscales
26 Participation Rater confidentiality Anonymous responses Grouped by relationship to facilitate interpretationIf fewer than 3, raters go into a “mixed” category (except manager)Rater familiarityRaters are asked how well they know the ratee
27 Rater FamiliarityRater familiarity information is found on the first page of the Coach’s Summary in the Multirater Feedback Report
29 How the EQ-360 Works Scoring converts raw scores to standard scores factor structureInterpretive guidelinesStandard ScoreGuideline130 and overAtypically well developed emotional and social functioning120 to 129Extremely well developed emotional and social functioning110 to 119Well developed emotional and social functioning90 to 109Effective emotional and social functioning80 to 89Consider developingUnder 80Develop
30 How the EQ-360 WorksDiscuss your experience as a ratee in the EQ-360 process:Who participated?Impressions of your EQ-360 experience?Subscale consistencies or discrepancies?Thoughts on feedback?Were there any surprises?
32 An Introduction to Psychometrics Part ThreeAn Introduction to Psychometrics
33 Part Three ObjectivesBy the end of this section, you should be familiar with:What the EQ-i and EQ-360 measureEQ-360 normative dataInternal consistencyConnections between the EQ-i and EQ-360 response patterns
35 EQ-i RefresherValidityReliabilityNormsNorm Groups
36 EQ-360 Normative Data 745 Ratees… Age range from 2170 Mean age of 40.4 years
37 EQ-360 Normative Data 1900 Raters… 909 male & 969 female 22 did not report gender
38 Internal Consistency Reliability of Total EQ is .98 Inter-scale reliability for Composite Scales ranged from .91 to .95Inter-scale item reliability for Subscales ranged from .77 to .91
39 EQ-i and EQ-360 Connections Ratee and Rater Group CorrelationsScales with significant* relationships:Emotional Self-AwarenessEmpathyProblem SolvingManagerPeerDirect Report*Magnitude was moderate
40 EQ-i and EQ-360 Connections Ratee and Rater Group DifferencesRatee was significantly higher than raters on four scales only:Emotional Self-AwarenessEmpathyProblem SolvingSocial ResponsibilityManagerPeerDirect ReportOther
41 EQ-i and EQ-360 Connections Age DifferencesScales with significant* relationships:Managers̶PeersDirect ReportsIndependenceSelf-ActualizationIntrapersonal Composite*Magnitude was very small
42 EQ-i and EQ-360 Connections Gender DifferencesScales with significant* relationships:EmpathySocial ResponsibilityInterpersonal CompositeManagersF>M̶PeersDirect Reports*Magnitude was very small
43 EQ-i and EQ-360 Connections What do the differences mean?Ratee may be inclined to inflate responsesContext of assessment may affect responsesSupervisor unfamiliarity results in inaccurate responsesDiffering definitions of effective emotional functioning between groups
46 Part Four ObjectivesBy the end of this section, you should be familiar with:How to plan an EQ-360 assessmentGetting organizational buy-inEthical issues in administrationSelecting RatersHow to debrief participants
47 Purposeful Planning What is the purpose of assessment? How will the results be used?What does the ratee know about or expect from the process?
48 Purposeful Planning Issues to Consider: Timing Extenuating circumstancesPersonnel involvementHow will the results be used?Methods available to confirm confidentialityFeedbackWho will be facilitating development and action planning?
49 Purposeful Planning Before you begin, establish: What the organization hopes to gain from the assessmentConnections between purpose and specific business needs or goalsClear participant expectations for accountability
50 Organizational Buy-In Senior Management support lends credibility to the processPrepare an EQ business caseEducate management team on the value and relevance of the process (ROI)
51 Organizational Buy-In Convey the importance of getting senior management involved:Consider creating a steering committee responsible for guiding the 360 processShow them how they will benefit
52 Selecting Raters Identifying who to involve Supervisor, colleagues, direct reports, clients, and family members round out the mixDetermine who gets input in the rater selection processHR?The ratee?The ratee’s manager?
53 Selecting Raters Defining roles Will rater nominations be submitted to manager(s) and/or HR for approval or adjustment?The ratee’s manager:Will his/her responses be singled out?Will he/she be involved in skill development and coaching?
54 Selecting Raters Choose raters who: Are credible and trustworthy Work closely with the rateeKnow the ratee wellFor a fresh perspective, select some raters who have worked with ratee for only a short period of time.
55 Selecting Raters Tips for defining raters: There must be at least three raters per category.If fewer than three, put raters in the Other/Mixed category to ensure confidentiality.Anyone with a supervisory influence on the Ratee is considered a Manager. If there is only one manager, discuss confidentiality issues.Include Managers, Peers and Direct Reports in the EQ-360 process. Consider clients and family depending on what outcome you want from the assessment. There must be at least three Raters per category (Peers, Direct Reports, Clients, Family). When there are fewer than three Raters per category, move the Raters into the Other/Mixed category to provide confidentiality.
56 Ethical Issues Who is the client? Individual respondent Group Organization
57 Ethical Issues Confidentiality EQ-360 ratees own their data Sharing report/results and determining action steps are decided by the rateeBreaches of confidentiality and rater anonymity can jeopardize feedback and compromise the integrity of the administrator
58 Ethical Issues Ensuring Rater Anonymity Raters feel more comfortable giving honest responses when they are anonymousIf they feel their identity is unprotected, raters may respond overly favorably or be reluctant to participate
59 Ethical Issues Ensuring Rater Anonymity Each rater category is scored together as an aggregateIf there is only one rater in a category, those results will be placed in the Other/Mixed groupAll online responses are stored in a secure MHS scoring database
60 Ethical Issues Informed Consent Obtain before the administration Ensure voluntary participation and freedom to withdraw from participating without punishmentProvide contact information of administratorCheck with your State/Provincial or Country psychology board regarding ethical issues
61 Ethical Issues Debriefing Raters Explain purpose of the debriefing sessionHow did they find the EQ-360 process?What were their expectations and were they met?How could the process be improved?
62 Ethical ScenariosFor each of the scenarios provided, consider the following questions:What is/are the ethical issues with the scenario?What steps should be taken to ensure the privacy of participants?What are the appropriate steps that should be followed in order to rectify the issue?What resources are available in order to rectify the issue?
63 #1: AcquisitionAn I/O psychologist has collected extensive developmental planning data on the top 100 executives of a very large corporation. The data includes the results from the EQ-360.Participants completing the EQ-360 were promised by the Board of Directors that the results would be used for development purposes only.Now, the organization is being acquired, and the team making staffing decisions for the merged organization has requested the 360 data.
64 #2: Succession Planning A management consultant has decided to relocate to another city and must leave behind data from a recent EQ-360 assessment used for development purposes with various managers from an Engineering company.The Engineering company indicated that it will not replace the management consultant. Instead, a HR manager will assume responsibility for the EQ-360 feedbacks, as well as the development initiatives.
65 #3: Scope PlanningA consultant was hired by a firm to conduct EQ-360 assessments.The organization has not yet decided whether the assessment results will be used for selection in addition to development, or who will have access to the data.The consultant is concerned and wants to tell the ratees and raters that the firm has not yet decided how the data will be used.
67 Part Five ObjectivesBy the end of this section, you should understand how to:Prepare EQ-360 participantsUse your Scoring Organizer accountSelect an administration format
68 Preparing Participants Speak with the ratee and raters before administrationDiscuss with the participants their right to confidentiality, feedback, and the development of skills after the assessmentParticipants need to know how they will benefit from the assessment
69 Preparing Participants Brief participants onWhat emotional intelligence isWhat a 360° assessment isHow the EQ-360 will be usedWhat are the rating scalesDefinitions of scales
70 Preparing Participants Discuss the importance of providing honest answers that reflect current functioningDiscuss how long it will take tocomplete the inventoryClarify ratee and rater expectations
71 Exercise Create a correspondence to raters Invite a group of raters to complete the EQ-360 on their coworker, Jim Smith.Your invitation should contain the following information:What the EQ-360 isWhy they are being asked to complete itReasons for participationThat they will complete the assessment onlineWhat will be done with the results
72 Administration Options Online:Link to survey website in anWeb scoringPaperPackets shipped to participantsMail/fax scoring
73 Scoring OrganizerYou will receive an MHS Scoring Organizer account that stores your rater information onlineFull instructions in the online Help and Getting Started Guide
75 Online Administration Setting up the EQ-360Each ratee must belong to a different Scoring Organizer GroupDecide whether you wish to add open-ended items to the EQ-360Raters’ short responses will appear on the Coach Report
76 Online Administration Add a new group for each individual ratee
77 Online Administration Add up to 7 open-ended items:
78 Mail/Fax Administration Use the item booklets, scanable response sheets, and scoring service processing formMail or fax forms to MHS for scoringReports can be scored from your Scoring Organizer account
79 Administration Service No time to handle administration and scoring?MHS can contact raters for you, track completion, and send you scored reports
82 Part Six ObjectivesBy the end of this section, you should know how to:Score EQ-360 reportsDifferentiate the EQ-360 Client and Coach reportsDecide what reports to useFind information in the reports
83 EQ-i Refresher Report Narrative Information Interpretive Guidelines Bar GraphsBest for…Development ReportClinical with suggestions for development for lowest 3 subscalesNumeric5 areas:“Very Much Below Average” to “Very Much Above Average”Counseling and life coaching(Counselor’s Section stays with clinician)Resource ReportCorporate with suggestions for development for lowest 3 subscalesNumeric in Counselor’s Section only3 areas:Enrichment, Effective Functioning, Enhanced Skills; divided by approx 1 SDHuman capital development in organizations(Counselor’s Section stays with coach/HR)Business ReportCorporate with suggestions for development for all 15 subscalesNumeric in Coach’s Section onlyDevelopment planning with SMART goal setting and activity sheets(Coach’s Section stays with coach/HR)EQ-360 Reports come with your choice of one of the above reports. Individual Summary Reports are generated separately.
85 Scoring Reports Scoring generates 2 EQ-i and 2 EQ-360 reports: See your Getting Started with EQ-360 Guide for instructions
86 Scoring ReportsUnder EQ-360, select the EQ-i report type to determine the self-rated report outputThere is no need to score the EQ-i separately
87 Report Comparison Coach Report Client Report Provides the administrator, coach, or employer* with in-depth response analysis, including validity and rater informationNumeric cut-offs may be distracting to the rateeNot recommended for raters or the rateeProvides the ratee or employer with general response analysis and development informationSensitive language encourages positive changeNot recommended for raters*Access to reports requires the consent and permission of the ratee.
95 Interpretation and Feedback Part SevenInterpretation and Feedback
96 Part Seven ObjectivesBy the end of this section, you should know how to:Gauge validity of responsesUnderstand report resultsProvide feedback to the rateeRecommend strategies for developmentDeal with common challenges
97 Assessing ValidityExamine the overall validity of the EQ-360 assessment with regards to:- EQ-i validity - Number of raters - Rater familiarity - Selection of raters
98 Assessing Validity EQ-i Validity Omission Rate Positive Impression ScaleNegative Impression Scale (if applicable)Inconsistency IndexRatee response styleTime taken to complete the EQ-i
99 Assessing Validity Group Size Ensure that there are enough raters in each rater group (3 rater minimum)Ensure that there is a similar number of peers and direct reports
100 Assessing Validity Rater Familiarity Review rater familiarity chart (in the Coach’s Summary)Responses may not be accurate if raters attest to knowing the rater “Not Very Well” or “Under 1 year”
101 Assessing Validity If raters are selected by… The ratee: Make sure that raters are objective and work with the rater in some current capacityThe ratee’s manager or HR:Verify rater relationships with the rateeManager and HR may not know all of the people with whom the ratee works
102 Understanding the Results EQ-360 raw scores converted to standard scoresEach EQ-360 scale has same mean and SD as EQ-i scalesMean = 100 SD = 15
103 Understanding the Results Review scores for scales and items:Total EQComposite scalesSubscalesItem responses
104 Understanding the Results What are score discrepancies?Compared to EQ-i scores, are the EQ-360 scores significantly (> 10 points) higher or lower?Do the ratings of one rater group differ substantially from others?
105 Understanding the Results If ratee scores lower than raterIndicative of underestimating own EI abilityMay also reflect low self-awareness or self-esteemIf ratee scores higher than ratersIndicative of overestimating own EI abilityMay reflect inability to view self critically, ego centrism/narcissism
106 Understanding the Results Significant score differences between rater groupsIndicative that ratee behaves differently with each rater groupConsensus of scores between rater groupsIndicative of accurate picture of ratee’s EI functioning most of the time and across situations
107 Understanding the Results EQ-360 results should be integrated with other data and information:Interview data from rateeAdditional psychological and behavioral assessmentsQualitative feedback of colleagues/ratersDirect observation of ratee
108 Feedback Challenges Potentially vulnerable time Defense mechanisms Areas that need development are for strategic correction, not blame or guiltTalk about strengthsEstablish your role in the development process
109 Providing Feedback Set the Stage Verify the client’s understanding of the purpose of the assessmentDiscuss how the results will be usedAssure the client of confidentiality
110 Providing Feedback Explain the Feedback Process Ask when and where the client completed the assessmentPoint out that scores are a basis for discussionDiscuss the reasons why there are no numbers on the report
111 Providing Feedback Discuss the report Validity: explore what motivated the client’s answersSelect subscales you want to discussAsk the client what he or she thinksDraw connections between subscalesIdentify patterns between rater groups
112 Case Studies Feedback Report practice interpretation Refer to case studies in the appendix of your workbook
113 Case Study #1 Justin 33-year-old Customer Service manager EQ-360 to aid in professional developmentReferred by manager, who is grooming Justin for a middle management position
114 Case Study #2 Linda 34-year-old Project Manager EQ-360 to aid in transition to new roleProblems with staff managementReferred by manager to HR
115 Case Study #3 Amalie 49-year-old NFP CEO EQ-360 given to all executives to identify leadership gapsAmalie’s task-oriented work style differs from her sociable coworkers
116 Case Study #4 Christopher 48-year-old commercial airline pilot Referred by HR due to co-pilot complaintsConcern over communication and following logbook protocol
117 Case Study #5 Bill 49-year-old sales manager Referred by VP due to interpersonal problemsConcern over lack of staff commitment
118 Case Study #6 Joan 55-year-old university department chair Directing a new department committeeInstigated committee-wide EQ-360 for teambuilding and performance managementWishes to rekindle dedication and teamwork
120 Coaching and Development Part EightCoaching and Development
121 Part Eight ObjectivesBy the end of this section, you should know how to:Help the ratee create a personal development planAssess the organizational factors that affect the ratee’s progressReassess for success
122 Analyzing Development Strategies Strategies appear for the three subscales with the greatest discrepancy between Ratee and the All Rater GroupChoose the strategies that are most appropriate to improving work performance
123 Analyzing Development Strategies Discuss accessible resources and support systems to help leverage client strategiesEncourage the use of development strategies on the job
124 Creating a Personal Development Plan Develop a plan of action with clear goals and timelinesThe goal is lasting behavioral changeWho, when, how:Who will be involvedWhen will it beginHow it will happen
125 Creating a Personal Development Plan Pick one or two subscales that are relevant to job performanceRecord the current level of behavior in these subscalesList how the attainment of goals will be beneficial to the organization
126 Creating a Personal Development Plan List internal and external factors that willEnhance your EQ performanceHinder your EQ performanceSchedule regular feedback from peers, direct reports, and manager
127 Exercise Create a personal development plan Jim Smith’s results show that he is low on the Interpersonal Relationship scale, which is important to his job as an engineering supervisor.His plan should contain the following information:The end goalHow development will benefit the organizationWho, when, howTimelinesFactors that will help or hinder progress
128 Follow-up Coaching Summarize prior meeting Check whether progress towards goals has been madeWhen a goal is met…Congratulate!Choose another EQ-i area to focus onSet new goals
129 Follow-up Coaching When a goal is not met… Look to see if goals were unrealisticSet new goals with new timelines and action stepsFind additional resources to achieve revised goals
130 Measuring Outcomes Begin with organizational objectives Identify a job-specific competency model with detailed skills and behaviorsQuantify performance outcomes such as customer satisfaction, leadership, sales, profit, or deadline adherenceEstablish cut-offs for poor performance
131 Measuring OutcomesWhat EQ-i areas are most important to organizational and individual success?What competencies will you investigating?Individual?Organizational?How do the EQ-360 results map to the strategy or needs of the organization?
132 Reassessment to Gauge Progress Repeat assessment as a measurement of outcomesAnnual assessment is the norm but more frequent assessments may be neededCompare effectiveness over timeDo performance measures need to change?
135 Goals Scoring and reports 360° assessments Interpretation and feedback CoachingExam360° assessmentsUsing the EQ-360PsychometricsPlanning and ethicsAdministration
136 Interpret a sample report (60 minutes) ExamInterpret a sample report (60 minutes)
137 EQ-360 AssessmentComplete a EQ-360 administration, interpretation, and feedback on a client of your choiceProvide a detailed written description of the following (include copies of forms and letters used if any):How you addressed informed consentHow you set up the administrationHow you conducted your initial interpretationDescribe the feedback session(s)Describe any additional plans discussed with the client
138 EQ-360 Assessment Required for qualification Format & length Due date Submission
139 Please fill out an evaluation form How to contact me Thank YouPlease fill out an evaluation formHow to contact me