Presentation is loading. Please wait.

Presentation is loading. Please wait.

Common Measurements Tool Client Feedback Workshop.

Similar presentations


Presentation on theme: "Common Measurements Tool Client Feedback Workshop."— Presentation transcript:

1

2 Common Measurements Tool Client Feedback Workshop

3 Introductions Welcome Introductions Objectives Agenda

4 Objectives Learn why,when,how to use CMT Understand basics of survey research Survey operationalisation, question design, sampling, administration, analysis 101 Produce 2 draft surveys (Homecare, Emergency Care) Produce action plans for implementation

5 Resources To assist you additional resource material has been provided in your binder CMT Managers Guide Customer Feedback and Customer Satisfaction Measurement - EPA Detailed Handouts, Articles, Case Studies

6 Why Customer Feedback? Performance measurement is basic element of management. Management functions Plan Organize Lead Control (Measurement)

7 Managements Contribution To coordinate the behaviour of: individuals groups organizations To attain: individual effectiveness organizational effectiveness Management performs the functions of: planning organizing leading controlling Feedback

8 Customer Service Model Satisfied Customers Principles Systems, Processes Enabling Environment Performance Measurement Team-Oriented Customer-Focused Manage by Fact Continuous Improvement Leadership Customer Feedback

9 What level of service should you get from government, compared to the private sector? Percent of respondents Governments have a more difficult task than the private sector. EXPECTATIONS OF SERVICE QUALITY Percent of respondents

10 Implement, Evaluate, Repeat Develop and implement action strategies, test outcomes Listen to Customers Analyze Results Implement Changes

11 Customer Feedback Framework Plan Construct Conduct Analyze Act

12 Step 1. Plan the Customer Feedback Project Define objectives of feedback What do I want to accomplish with this feedback? Why am I conducting this feedback activity Determine how findings will be used What will we do with these findings Will they be used for performance indicator to improve a process to ID customer needs/expectation planning, decision making

13 What is most appropriate feedback technique? Survey is often not the best technique Options include Focus Groups In Depth Interviews Customer Advisory Panels Employee Observations Performance Audits Customer Site Visits Mystery Shoppers Benchmarking surveys –see handbook for description of strengths and weaknesses

14 Strengths/Weaknesses Employee Observations Customer feedback, complaints Service audits Internal customers Walk-throughs Focus groups, interviews Surveys

15 Two key questions What information do you need? Who (which segment of the population) can provide you with this information? Internal vs external customers Demographic segmentation (age, income, geography Program function

16 Exercise # 1 In 2 groups answer the following questions What do you want to know? What technique would you recommend? Who would ask to provide this information? In 2 groups answer the following questions What do you want to know? What technique would you recommend? Who would ask to provide this information?

17 FOI/FIPPA Major Concerns include privacy notification approval etc. Major Concerns include privacy notification approval etc.

18 Warning: How ready is your organization for customer feedback? Do managers and staff sincerely intend to pay attention and take action? Are key managers committed to action? Have staff members participated directly in defining needs and approaches Are there possible barriers to using feedback effectively ( concerns re: workload, negative findings, change )

19 Top Reasons for Failure Failure to involve the right person in developing customer service measures customers sponsors owners,users Failure to capture data that leads to action Scorekeeping vs.understanding Failure to link customer data to operational, quality, or performance data Absence of closed- loop process for measuring and managing satisfaction Brandt,D.R. Build Actionability Into Your Customer Surveys 1995

20 Customer Feedback Framework Plan Construct Conduct Analyze Act

21 Step 2.Constructing the Survey Types of Questions Dos and Donts Scales Ordering Number of Questions Layout Types of Questions Dos and Donts Scales Ordering Number of Questions Layout

22 Phase 2 Constructing your survey Operationalizing your research questions What kinds of questions will get you the information you are looking for This crucial step is often overlooked Examples The Common Measurements Tool

23 Common Measurements Tool Objectives: Provide a ready-made tool for public service organizations Enable like-organizations to compare results Enable organizations to build internal benchmarks Facilitate the sharing of information gained and lessons learned between organizations

24 Elements of the Service Experience... Client expectations Client perceptions of service experience Level of satisfaction Level of importance Priorities for improvements

25 CMT - Service Dimensions Responsiveness »Timely delivery of service »# of contacts to receive service »Waiting time »Courtesy of service staff Reliability »What was promised was delivered »Errors detected and corrected Access & Facilities »Hours of service »Telephone access Communications

26

27 Question Types Open ended questions allows respondents to provide his or her own answer –e.g.. What one thing could we do to improve service? Closed ended questions limit the response an individual can make –e.g. (likert scale) CMT includes both types

28 Closed Ended Pros Questions are relatively easy to complete Easy to analyze Unlikely to receive irrelevant or unintelligible responses Pros Questions are relatively easy to complete Easy to analyze Unlikely to receive irrelevant or unintelligible responses Cons no knowledge may over simplify Risk missing the boat: For example if you asked what is the main reason for your visit? and 30% respond other. This is a good reason to pre-test questions to ensure you include all the likeliest responses

29 Open ended Pros will receive more information, often richer (intensity, more specific) more likely to receive surprise insights Pros will receive more information, often richer (intensity, more specific) more likely to receive surprise insights Cons difficult to analyze and code responses may be irrelevant or unintelligible demanding and time consuming for respondent

30 Guidelines Remember you want actionable info what action could you take with this information Avoid questions which are too vague, subject to misinterpretation double barreled questions which imbed more than one item »on a scale of 1-6 please indicate how clear and useful the materials are? extraneous or irrelevant sensitive

31 Avoid questions that do not elicit responses which point to specific remedies –e.g. How satisfied are you with the quality of service provided »Very satisfied »Satisfied »Neutral »Dissatisfied »Very Dissatisfied which mix scales –e.g. were you checked in within five minutes »satisifed »neutral »dissatisfied

32 Avoid questions which include terminology, jargon etc. which includes words and terminology which is too complex »e.g. Are you aware of the impending amalgamation of surrounding constituencies into the New Metro catchement region? with unclear frame of reference »e.g. what is your income? »your individual family, household? »Time reference bi-weekly, monthly, annual »Income - salary, tips or other sources

33 Impact of question wording See hand out for examples Implied alternative

34 Considerations in wording Are the words simple, direct, and familiar to all respondents? Are the questions as clear and specific as possible? Are the questions applicable to all respondents Are the categories mutually exclusive and exhaustive Are any questions leading or loaded?

35 Scale types Likert scale 3,5,7,9 point Thurston

36 Constructing Questionnaire Format Introduction - sets forth the purpose of survey and guide respondent through questions Customer experience - establishes customers level of knowledge regarding various parts of questionnaire Measurement - CMT asks customer to characterize experience, expectations, needs, priorities Demographic information - classify

37 Ordering of Questions Order of questions should encourage respondents to complete questionnaire facilitate respondents recall appear sensible focus on the topic follow a sequence that is logical to the respondents follow smoothly from one to next

38 Sensitive Questions Introduce where respondent has developed confidence and trust locate in section where they have most meaning and context introduce slowly

39 Number of Questions Be careful not to ask too many questions CMT (100+) Do not recommend asking all 11 core questions get 5 drivers,access, communication and cost –(see list in book)

40 Timing of Feedback When you solicit feedback is very important too long after original event will lower response rate and relevancy. Be cautious re: decisions, administrative process etc.

41 Exercise # 2 Group Exercise – Using CMT as a guide each group will be given 1 hour to develop a draft survey based on first exercise. Each group must include a minimum of 5 CMT questions and 5 Custom questions

42 Sampling Census Vs. random What size sample do I need? Depends on use of survey Trade offs Confidence Interval and sampling error How to calculate sample size Why 400 is the magic # –(see handouts)

43 On-line tools

44 Pre-testing Questions Where possible pre- test your instrument. Ensure language is clear Have you missed any important items Testing Methods Focus Groups/Interviews Ask test respondents to talk out loud Remember Pre-testing is extremely valuable, if you consider how much time and effort you have already committed to customer feedback

45 Agenda Day 2 Review of Draft Surveys - Discussion Review of CMT Conducting Surveys (Who,When, Where, How) Analysis Action

46 Citizen-Centered Network A network of Federal/Provincial/Municipal civil servants dedicated to improving the quality of public service in Canada Citizens First Erin Research Inc THE CITIZEN-CENTRED Common Measurements Tool Faye Schmidt for the ANAGEMENT DEVELOPMENT IPAC 1999 Gold Award

47 Citizens First Five Drivers of Satisfaction 1. Timeliness 2. Competence 3. Courtesy 4. Fairness 5. Outcome

48 Customers Priorities for Improving Access to Government Services Making government services easier to locate in the phone book Improving telephone service Expanding electronic access Expanding hours of service Standards for routine services Access a major weak spot 29% citizens cant find service 89% experience at least 1 problem Access a major weak spot 29% citizens cant find service 89% experience at least 1 problem

49 Common Measurements Tool Objectives: Provide a ready-made tool for public service organizations Enable like-organizations to compare results Enable organizations to build internal benchmarks Facilitate the sharing of information gained and lessons learned between organizations

50 Elements of the Service Experience... Client expectations Client perceptions of service experience Level of satisfaction Level of importance Priorities for improvements

51 CMT - Service Dimensions Responsiveness »Timely delivery of service »# of contacts to receive service »Waiting time »Courtesy of service staff Reliability »What was promised was delivered »Errors detected and corrected Access & Facilities »Hours of service »Telephone access Communications

52 The CMT Go to tool

53 Customer Feedback Framework Plan Construct Conduct Analyze Act

54 Step 3. Conducting Surveys Will Focus on Mail Surveys (80-90%) How to achieve a high response rate Data collection and coding Data entry options –CMT Data Entry options

55 Mail Surveys Mail surveys can achieve response rate of 60% Begin with good data base –unique id #, relevant customer characteristics, mailing address, mail out dates and date responses receive Professionalism in all Materials –questionnaire, cover letter, return envelope with self addressed stamp envelope; all professional look

56 Ensuring high response rate Include cover letter; explaining purpose of survey ( print on official letter head ) Booklet format recommended ( see sample ) Use stamp and hand written address Objective is coffee table adhesion factor Follow up letter is a must Generates an additional 50% response

57 Other tips Avoid mailing near holidays use hot-line or contact # use admin # and explain this is to avoid multiple contacts 1st Mail out 2nd Mail out 3rd Mail out

58 Telephone Surveys Very common form of survey administration Advantages are: faster offers more control Call response ratio 10:1 CMT provides Computer Assisted Telephone Interview option

59 Internet Surveys Advantages fast, cheap, automated data collection/analysis Disadvantages susceptible to sample biases too easy to administer security/privacy concerns More info available on request

60 Data entry options CMT External data entry contact Service 1st Manual entry options On-line survey Data entry spreadsheet sheet Bar coding

61 Customer Feedback Framework Plan Construct Conduct Analyze Act

62 Analysis Quantitative Frequencies, percentiles, percentages Central tendencies (mean, mode, median) Cross tabulations that array independent variables against dependent variables –e.g. customer type who were satisfied Quantitative Frequencies, percentiles, percentages Central tendencies (mean, mode, median) Cross tabulations that array independent variables against dependent variables –e.g. customer type who were satisfied Cross tab example

63 7 Tools of Quality Cause and effect diagram Check Sheets Control charts Flow charts Histogram Pareto charts Scatter Diagram Cause and effect diagram Check Sheets Control charts Flow charts Histogram Pareto charts Scatter Diagram See Handout

64 CMT Analysis For each question Bar/Pie charts Frequencies Percentages For each section Satisfaction/Importance Actual Vs. Expected Priorities for Improvement Service Standards For each question Bar/Pie charts Frequencies Percentages For each section Satisfaction/Importance Actual Vs. Expected Priorities for Improvement Service Standards

65 Exercise 3 In groups of 4 review the Case Study What are the main findings? What are the clients priorities for improvement? What further analysis do you recommend? How would you present the results to executive management? 30 Minutes

66 Customer Feedback Framework Plan Construct Conduct Analyze Act

67 Acting on Results What to do with the feedback? Step 1. Assess what the feedback is telling you. CMT for example allows you to evaluate 5 drivers of satisfaction, estimate service standards, understand relationship b/w satisfaction and importance and ID priorities for improvement.

68 Acting on Results How good is good enough? Satisfied vs.. Very satisfied 4.0/5,0 vs. 4.5/5.0 Nature of organization effects rating How good is good enough? Satisfied vs.. Very satisfied 4.0/5,0 vs. 4.5/5.0 Nature of organization effects rating

69 Action Steps Recover Report Brief Prioritize Communicate Improve Enhance Reward Plan Feed into Strategic/Business Planning Adapted from Customer Service Guidelines EPA

70 Defining Action Paths Process involves identifying owners for each issue or area targeted for improvement Getting owners to own readiness for action linking issue to relevant processes, systems using learning achieved above to define next steps See Brandt

71 Satisfaction and Problem Tracking LOGGING Problem description Action required CLASSIFICATION Bridge to Analysis ANALYSIS Statistics Diagnostics Policy PREVENTION Training/Staffing Process Improvement Innovation INVESTIGATION FORMULATION RESPONSE AND FOLLOW-UP REFERRAL OR SPECIAL HANDLING Technical Assistance Research Programs (TARP)

72 Organizational Readiness DO WE UNDERSTAND WHAT CUSTOMER IS TELLING US? GATHER ADDITIONAL INFORMATION DEVELOP AND IMPLEMENT SOLUTION BEGIN PROCESS PERFORMANCE ANALYSIS ALIGN TARGET/SOLUTION WITH CUSTOMER EXPECTATIONS IS OUR TARGET ALIGNED WITH CUSTOMER EXPECTATIONS? IS THE EXISITNG METHOD WORKING? DO WE HAVE A METHOD OF ADRESSING THE ISSUE? NO

73 Wrap - up Review Objectives Tools and References


Download ppt "Common Measurements Tool Client Feedback Workshop."

Similar presentations


Ads by Google