41st International Military Testing Association Conference, November 11, 1999 SA-ALC Supervisor Feedback Survey Assessing organizational issues: Using a computer-based supervisor feedback survey at several organizational levels to assess relative unit supervisory climate Charles Holt Institute for Job and Occupational Analysis Carol Maske Metrica, Inc.

41st International Military Testing Association Conference, November 11, 1999 SA-ALC Supervisor Feedback Survey Purposes of Study: To provide insight on how supervisors are perceived to be performing according to their direct subordinates To identify supervisory training needs

41st International Military Testing Association Conference, November 11, 1999 SA-ALC Supervisor Feedback Survey Conducted in the fall of 1998 at Kelly AFB, Texas All SA-ALC personnel surveyed Administered twice per year since 1994 Anonymous

41st International Military Testing Association Conference, November 11, 1999 SA-ALC Supervisor Feedback Survey Feedback Six Critical Performance Areas: Communication Decision Making Human Development Management Support Safety Transition Issues

41st International Military Testing Association Conference, November 11, 1999 SA-ALC Supervisor Feedback Survey Feedback Survey format 30 multiple choice items rated on a 6-point scale with “1” being “Strongly Disagree” and “6” being “Strongly Agree” Items were identical to those used in previous surveys except five additional questions that were related to transition issues we added to this iteration. Also four open-ended items asking respondents to comment on things they would like their supervisor to Stop Doing, Start Doing, Continue Doing and one question asking for General Comments

41st International Military Testing Association Conference, November 11, 1999 SA-ALC Supervisor Feedback Survey Feedback Utilized the Air Force Survey Authoring Software (AUTHOR) Surveys were completed on 3.5” floppy disks Respondents completed the surveys at work The disks were returned to IJOA and uploaded using AUTHOR’s Collect function The data was analyzed and summarized with the Statistical Package for the Social Sciences (SPSS), Version 8.0 Data “rolled up” from lowest level of hierarchy to highest

41st International Military Testing Association Conference, November 11, 1999 SA-ALC Supervisor Feedback Survey Feedback 4,840 Valid cases returned 76th ABW20% of total 76th ABG8% of total SA-ALC72% of total

41st International Military Testing Association Conference, November 11, 1999 SA-ALC Supervisor Feedback Survey Feedback Reports generated: 1099 individual supervisor reports with means and standard deviations 38 division reports rolled up from supervisor reports 3 organizational reports rolled up from division reports 1 overview report 1141 total reports generated

41st International Military Testing Association Conference, November 11, 1999 SA-ALC Supervisor Feedback Survey Feedback Overall Means by Critical Area

41st International Military Testing Association Conference, November 11, 1999 SA-ALC Supervisor Feedback Survey Feedback Only one division appeared problematic

41st International Military Testing Association Conference, November 11, 1999 SA-ALC Supervisor Feedback Survey Feedback Individual results 40 supervisors identified as scoring less than 2.0 standard deviations below the overall mean. Remedial training to be provided by appropriate organization

41st International Military Testing Association Conference, November 11, 1999 SA-ALC Supervisor Feedback Survey Feedback Comment Analysis Over 1000 pages of narrative comments generated No full semantic analysis performed No attempt to group comments by areas of concern or topic Reviewed for threatening or abusive statements Very few threatening comments found.

41st International Military Testing Association Conference, November 11, 1999 SA-ALC Supervisor Feedback Survey Feedback Discussion Demonstrated the versatility of AUTHOR Demonstrated the willingness of respondents to make comments when appropriate opportunities are provided Raised issues regarding confidentiality of “anonymous” responses Demonstrated the need for easily accessible semantic analysis tools

41st International Military Testing Association Conference, November 11, 1999 SA-ALC Supervisor Feedback Survey Feedback Thank you !