Presentation on theme: "5/2/20151 Guide to MN WorkForce Center System Job Seeker Satisfaction Survey Making Sense of the Semi-Annual Charts and Reports."— Presentation transcript:
5/2/20151 Guide to MN WorkForce Center System Job Seeker Satisfaction Survey Making Sense of the Semi-Annual Charts and Reports
5/2/20152 About the Survey Telephone interviews are conducted with: Exiters from Workforce Investment Act (WIA) Adult, WIA and State Dislocated Worker, WIA and State Youth, and Minnesota Family Investment Program (MFIP) Exiter and in-progress Vocational Rehabilitation Services (VRS) and State Services for the Blind (SSB) customers Core Services customers (“universal customer,” formerly Job Service). Visit the link below for specific selection criteria information: http://www.positivelyminnesota.com/About_Us/Customer_Satisfaction/Job_Seeker_Satisfaction/Background_2.aspx
5/2/20153 About the Survey Survey Vendor: Strategic Research Group (SRG) Chosen based on best response to our Request for Proposals Based in Columbus, Ohio Has conducted WIA customer satisfaction surveys for State of Ohio (our vendor since May 2005) Out-of-state vendor helps limit survey bias
5/2/20154 About the Survey Survey questions provide a “score” as well as descriptive comments: Minnesota Customer Satisfaction Index (MnCSI) is a numeric score based on 3 standard questions about general satisfaction (more on this in the ‘Questions’ slide). Descriptive comments are from open-ended questions asking about what services/activities program or WFC should START doing, STOP doing, KEEP doing.
5/2/20155 About the Survey The survey presents a broad view of customer satisfaction: Local area results are based on balanced input from local area customers. Results represent a range of customers: most satisfied, least satisfied, and everyone in-between. Results are produced semi-annually from on-going survey.
5/2/20156 Why is this survey important? Job Seeker Satisfaction Survey (JSSS) is based on large, random samples; these produce a credible picture of overall satisfaction for the whole system / program / local area. JSSS focuses on the customer’s entire services experience, particularly in relation to finding a job. JSSS complements results of Mystery Shoppers that focus on quality of WorkForce Center facilities and short- term interaction with staff.
5/2/20157 Why is this survey important? (cont.) JSSS is on-going, with new results each quarter; this allows tracking over time of customer response to changes in service strategy. JSSS provides detailed understanding of the customer’s experience by capturing verbatim responses to “which services were most help [in getting a job]” and “what about the services could be improved”.
5/2/20158 Reports Produced WorkForce Center System (minus MFIP & any additional extra sample, e.g. VRS, DW) Each major program at the statewide level Each local area, all programs combined. Each local area, extra sample reports; VRS Team report & DW Provider report
5/2/20159 Samples For a WorkForce Service Area (WSA), the proportion of interviews from a program roughly equals that program's portion of the total customer population. One exception: Core Services customers receive less- intense services than customers in other programs, so the Core Services population is halved. Example: ProgramPopulation in WSA Sample Adult 80 8 Dislocated Worker 600 60 Youth 240 24 VRS 900 90 Core Services 2800 (use 1400) 140
5/2/201510 Samples SSB and MFIP are not included in local area samples or results. SSB program results are based on a statewide sample (~262 interviews over 12 months). MFIP program results are based on a statewide sample (~376 interviews over 12 months).
5/2/201511 Questions Three questions ask about general satisfaction: Overall satisfaction with the services? Overall satisfaction with the services? To what extent have the services met expectations? To what extent have the services met expectations? How well do the services compare with ideal ? How well do the services compare with ideal ? Questions are asked on a 1-to-10 scale where “1” is least satisfied and “10” is most satisfied. Responses to the three questions are averaged and converted to 0-to-100 scale to create MnCSI (Minnesota Customer Satisfaction Index). MnCSI can be used to compare results over time.
5/2/201512 Questions Two questions asked about staff: How satisfied that staff understand your needs? How satisfied that staff understand your needs? How satisfied that staff responded to request for help in reasonable amount of time? (response scale - very satisfied to very dissatisfied) How satisfied that staff responded to request for help in reasonable amount of time? (response scale - very satisfied to very dissatisfied) Two additional questions asked (for all programs except Core Services) also use “very satisfied” to “very dissatisfied” How satisfied that were given enough information to make good choices for employment plan? How satisfied that were given enough information to make good choices for employment plan? How satisfied that had an active role in decisions about services? How satisfied that had an active role in decisions about services? Core Services questions: How satisfied that were given enough help in the Resource Area? Have you used Minnesotaworks.net to look for openings? If YES to using MNW, then job seeker is asked “Did you find that it was quite easy, somewhat easy, somewhat difficult, or quite difficult to use? If YES to using MNW, then job seeker is asked “Did you find that it was quite easy, somewhat easy, somewhat difficult, or quite difficult to use?
5/2/201513 Questions Three open-ended questions ask (customers asked only 2 of these 3 questions): What would you like the program/WFC to START doing? What would you like the program/WFC to START doing? What would you like the program/WFC to STOP doing? What would you like the program/WFC to STOP doing? What would you like the program/WFC to KEEP doing? What would you like the program/WFC to KEEP doing? Verbatim responses sorted into categories ( e.g., service quality, staff, level of service) Each category reported as a percent of total responses Customers often mention more than one topic in their response (e.g., “the computer class, learning what they are looking for in a resume, and the interview techniques...”)
5/2/201514 Three Things to Keep in Mind When Viewing Quarterly MnCSI Results Results fluctuate from quarter to quarter; watch them over a longer period. After five or six quarters, ask: Is there any consistent pattern to our WSA [program] score? Is there any consistent pattern to our WSA [program] score? Is the score generally rising or falling? Is the score generally rising or falling? If we changed our services or processes, did the score change? If we changed our services or processes, did the score change? Smaller samples give less exact results. Always check the width of the confidence intervals. It’s best not to compare scores among WSAs. Differences may be due to variations in sample make-up or: local economic conditions local economic conditions customer-to-staff ratios customer-to-staff ratios financial resources financial resources program strategies program strategies customer characteristics, etc. customer characteristics, etc.
5/2/201515 Overall Satisfaction with Services: Local Area, Region, State Track the changes over time. Remember, the larger the sample (the bigger the “N”) the more precise the result. Local area MnCSI scores are put in context by comparing to State and corresponding region (Metro or Greater MN) MnCSI scores. The chart uses an enlarged scale range (60.0 – 80.0) so you can better see trends in MnCSI scores. Customer Satisfaction: Lake Woebegone WSA Average MnCSI over past 5 quarters (12-month rolling) (N=Current N to date) 60.0 65.0 70.0 75.0 80.0 Apr 05-Jun 05Apr 05-Sept 05Apr 05-Dec 05Apr 05-Mar 06April 05-June 06 Year-to-date by Quarter Minnesota Customer Satisfaction Index (MnCSI) Lake Woebegone (N=455) Minnesota (N=4781) Greater MN (N=2834)
5/2/201516 Overall Satisfaction with Services: Local Area/Program Scores This chart shows average MnCSI score over time. Examine the confidence intervals, the black bars extending above and below the average scores. The bar indicates we are 95 percent confident the true score lies within the length of the bar. The narrower the bar, the more precise the MnCSI score. Think of confidence intervals as “margin of error,” much like political polls that refer to a “margin of error of plus or minus three points.” Customer Satisfaction: Lake Woebegone WSA Average MnCSI over past 5 quarters (12-month rolling) (Current to date N=455) 71.6 73.5 74.374.4 0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 100.0 Apr 05-Jun 05Apr 05-Sept05Apr 05-Dec05Apr 05-Mar 06Apr 05-June06 Year-to-date by Quarter Minnesota Customer Satisfaction Index (MnCSI) Lake Woebegone
5/2/201517 Summaries of answers to open-ended questions Summaries of answers to open-ended questions Tables provide a quick summary of topics mentioned in answers to: What would you like the programs/WFC to START, STOP, KEEP doing? The local area summary is put in context by comparing to State and region (Greater MN or Metro) summaries.
5/2/201518 What would you like the program/WFC to START, STOP, KEEP doing? Responses categorized into 6 to 7 major groups (issues or help finding and searching for a job, Staff helpfulness or improvements, etc.) and many more detailed sub-groups. Compare to State and Region (Metro or Greater MN) by looking across the rows.
5/2/201519 Comments on Good Services, September 2009 Results Youth “ “My counselor went way beyond just finding me a job, he hooked me up with finding scholarships, that is what I am most thankful for.” (WSA 3) Rehab Services “A program called Brighter Futures. All of these programs boil down to the individual that has hands on contact. The person that I had contact with was wonderful.” ( WSA 5) “My job coach. They helped me adjust problems to my employer properly, and also helped me find affordable transportation.” (WSA 17) Adult / Dislocated Worker “The weekly group session, job club, we got a lot of different ideas on how to react in interviews.” (WSA 8)
5/2/201520 Feedback on Good Services (cont.) “Resume writing, interviewing skills, creative job searching. They seemed to be very current in terms of what employers wanted.” (WSA 12) “I went to a three day boot camp; it was beneficial to see how things changed over the years. I also received two computer classes and I really enjoyed those.” (WSA 16) Core Services “I had a one on one meeting with a job placement worker and he spent a lot of time going over possible classes, training and useful internet job searching tools.” (WSA 4) “They are actually helping me look for jobs by providing referrals via email and phone calls.” (WSA 12) “The combination of everything is important, talking one-on-one with a counselor is important so that they can get a feel for who you are and your needs. The classes and knowing the website is important. There is a wide range of needs so one-on-one talking is important.” (WSA 17)
5/2/201521 Flexibility of the Survey Local areas that would like more detailed results may purchase a larger sample so their results can be broken-out by program. (Two programs, Voc Rehab Services and the Dislocated Worker program have enlarged their samples to break-out results down to local levels. VRS to 18 RAM or Team level and DW to 22 local area providers) (Two programs, Voc Rehab Services and the Dislocated Worker program have enlarged their samples to break-out results down to local levels. VRS to 18 RAM or Team level and DW to 22 local area providers) Local areas or programs can explore additional topics by purchasing additional questions asked just of their customers. (State Services for the Blind and LMI Research have purchased additional questions.)
5/2/201522 Sample Job Seeker Web Page Look for the link to your local area or program at these following links System http://www.positivelyminnesota.com/About_Us/Customer_Satisfaction/Job_Seeker_Satisfaction/Entire_WorkForce_System_Results.aspx http://www.positivelyminnesota.com/About_Us/Customer_Satisfaction/Job_Seeker_Satisfaction/Entire_WorkForce_System_Results.aspx Program http://www.positivelyminnesota.com/About_Us/Customer_Satisfaction/Job_Seeker_Satisfaction/Results_by_Program.aspx http://www.positivelyminnesota.com/About_Us/Customer_Satisfaction/Job_Seeker_Satisfaction/Results_by_Program.aspx WSA’s http://www.positivelyminnesota.com/About_Us/Customer_Satisfaction/Job_Seeker_Satisfaction/Results_by_Workforce_Service_Area.aspx http://www.positivelyminnesota.com/About_Us/Customer_Satisfaction/Job_Seeker_Satisfaction/Results_by_Workforce_Service_Area.aspx
5/2/201523 Need Further Information? Visit http://www.positivelyminnesota.com/About_Us/Customer_Satisfaction/index.aspx to view the entire Customer Satisfaction Survey results. http://www.positivelyminnesota.com/About_Us/Customer_Satisfaction/index.aspx Contact Devin Colvin of the Policy, Planning and Measures Unit via email, email@example.com or by phone, 651-259-7190, with questions, comments or requests about the presentation or results. firstname.lastname@example.org