Presentation is loading. Please wait.

Presentation is loading. Please wait.

Common Measurements Tool Client Feedback Workshop

Similar presentations

Presentation on theme: "Common Measurements Tool Client Feedback Workshop"— Presentation transcript:

1 Common Measurements Tool Client Feedback Workshop

2 Introductions Welcome Introductions Objectives Agenda

3 Objectives Learn why,when,how to use CMT
Understand basics of survey research Survey operationalisation, question design, sampling, administration, analysis 101 Produce 2 draft surveys (Homecare, Emergency Care) Produce action plans for implementation

4 Resources To assist you additional resource material has been provided in your binder CMT Managers Guide Customer Feedback and Customer Satisfaction Measurement - EPA Detailed Handouts, Articles, Case Studies

5 Why Customer Feedback? Performance measurement is basic element of management. Management functions Plan Organize Lead Control (Measurement)

6 Management’s Contribution
Management performs the functions of: planning organizing leading controlling To coordinate the behaviour of: individuals groups organizations To attain: individual effectiveness organizational effectiveness Feedback

7 Customer Service Model
Satisfied Customers Principles Systems, Processes Enabling Environment Performance Measurement Team-Oriented Customer-Focused Manage by Fact Continuous Improvement Leadership Customer Feedback

“Governments have a more difficult task than the private sector.” “What level of service should you get from government, compared to the private sector?” Percent of respondents Percent of respondents Two important elements of context: Citizens acknowledge that governments have a more difficult task than the private sector. Despite the difficulty of the task, however, citizens expect government to provide stellar service.

9 Implement, Evaluate, Repeat
Develop and implement action strategies, test outcomes Listen to Customers Analyze Results Implement Changes

10 Customer Feedback Framework
Plan Construct Conduct Analyze Act

11 Step 1. Plan the Customer Feedback Project
Define objectives of feedback What do I want to accomplish with this feedback? Why am I conducting this feedback activity Determine how findings will be used What will we do with these findings Will they be used for performance indicator to improve a process to ID customer needs/expectation planning, decision making

12 What is most appropriate feedback technique?
Survey is often not the best technique Options include Focus Groups In Depth Interviews Customer Advisory Panels Employee Observations Performance Audits Customer Site Visits Mystery Shoppers Benchmarking surveys see handbook for description of strengths and weaknesses

13 Strengths/Weaknesses
Employee Observations Customer feedback, complaints Service audits Internal customers “Walk-throughs” Focus groups, interviews Surveys

14 Two key questions What information do you need?
Who (which segment of the population) can provide you with this information? Internal vs external customers Demographic segmentation (age, income, geography Program function

15 Exercise # 1 In 2 groups answer the following questions
What do you want to know? What technique would you recommend? Who would ask to provide this information?

16 FOI/FIPPA Major Concerns include privacy notification approval etc.

17 Warning: How ready is your organization for customer feedback?
Do managers and staff sincerely intend to pay attention and take action? Are key managers committed to action? Have staff members participated directly in defining needs and approaches Are there possible barriers to using feedback effectively (concerns re: workload, negative findings, change)

18 Top Reasons for Failure
Failure to involve the right person in developing customer service measures customers sponsors owners,users Failure to capture data that leads to action Scorekeeping vs.understanding Failure to link customer data to operational, quality, or performance data Absence of closed- loop process for measuring and managing satisfaction Brandt,D.R. “Build Actionability Into Your Customer Surveys” 1995

19 Customer Feedback Framework
Plan Construct Conduct Analyze Act

20 Step 2.Constructing the Survey
Types of Questions Do’s and Don’ts Scales Ordering Number of Questions Layout

21 Phase 2 Constructing your survey
Operationalizing your research questions What kinds of questions will get you the information you are looking for This crucial step is often overlooked Examples The Common Measurements Tool

22 Common Measurements Tool
Objectives: Provide a ready-made tool for public service organizations Enable like-organizations to compare results Enable organizations to build internal benchmarks Facilitate the sharing of information gained and “lessons learned” between organizations

23 Elements of the Service Experience...
Client expectations Client perceptions of service experience Level of satisfaction Level of importance Priorities for improvements

24 CMT - Service Dimensions
Responsiveness Timely delivery of service # of contacts to receive service Waiting time Courtesy of service staff Reliability What was promised was delivered Errors detected and corrected Access & Facilities Hours of service Telephone access Communications


26 Question Types Open ended questions Closed ended questions
allows respondents to provide his or her own answer e.g.. What one thing could we do to improve service? Closed ended questions limit the response an individual can make e.g. (likert scale) CMT includes both types

27 Closed Ended Pro’s Con’s Questions are relatively easy to complete
Easy to analyze Unlikely to receive irrelevant or unintelligible responses Con’s no knowledge may over simplify Risk “missing the boat”: For example if you asked “what is the main reason for your visit?” and 30% respond other. This is a good reason to pre-test questions to ensure you include all the likeliest responses

28 Open ended Pro’s will receive more information, often richer (intensity, more specific) more likely to receive surprise insights Con’s difficult to analyze and code responses may be irrelevant or unintelligible demanding and time consuming for respondent

29 Guidelines Remember you want actionable info Avoid questions which are
what action could you take with this information Avoid questions which are too vague, subject to misinterpretation double barreled questions which imbed more than one item on a scale of 1-6 please indicate how clear and useful the materials are? extraneous or irrelevant sensitive

30 Avoid questions that do not elicit responses which point to specific remedies e.g. How satisfied are you with the quality of service provided Very satisfied Satisfied Neutral Dissatisfied Very Dissatisfied which mix scales e.g. were you checked in within five minutes satisifed neutral dissatisfied

31 Avoid questions which include terminology, jargon etc.
which includes words and terminology which is too complex e.g. Are you aware of the impending amalgamation of surrounding constituencies into the “New Metro” catchement region? with unclear frame of reference e.g. what is your income? “your” individual family, household? Time reference bi-weekly, monthly, annual Income - salary, tips or other sources

32 Impact of question wording
See hand out for examples Implied alternative

33 Considerations in wording
Are the words simple, direct, and familiar to all respondents? Are the questions as clear and specific as possible? Are the questions applicable to all respondents Are the categories mutually exclusive and exhaustive Are any questions leading or loaded?

34 Scale types Likert scale 3,5,7,9 point Thurston

35 Constructing Questionnaire
Format Introduction - sets forth the purpose of survey and guide respondent through questions Customer experience - establishes customers level of knowledge regarding various parts of questionnaire Measurement - CMT asks customer to characterize experience, expectations, needs, priorities Demographic information - classify

36 Ordering of Questions Order of questions should
encourage respondents to complete questionnaire facilitate respondents recall appear sensible focus on the topic follow a sequence that is logical to the respondents follow smoothly from one to next

37 Sensitive Questions Sensitive Questions
Introduce where respondent has developed confidence and trust locate in section where they have most meaning and context introduce slowly

38 Number of Questions Be careful not to ask too many questions
CMT (100+) Do not recommend asking all 11 core questions get 5 drivers,access, communication and cost (see list in book)

39 Timing of Feedback When you solicit feedback is very important
too long after original event will lower response rate and relevancy. Be cautious re: decisions, administrative process etc.

40 Exercise # 2 Group Exercise – Using CMT as a guide each group will be given 1 hour to develop a draft survey based on first exercise. Each group must include a minimum of 5 CMT questions and 5 Custom questions

41 Sampling Census Vs. random What size sample do I need?
Depends on use of survey Trade offs Confidence Interval and sampling error How to calculate sample size Why 400 is the magic # (see handouts)

42 On-line tools

43 Pre-testing Questions
Where possible pre-test your instrument. Ensure language is clear Have you missed any important items Testing Methods Focus Groups/Interviews Ask test respondents to “talk out loud” Remember Pre-testing is extremely valuable, if you consider how much time and effort you have already committed to customer feedback

44 Agenda Day 2 Review of Draft Surveys - Discussion Review of CMT
Conducting Surveys (Who,When, Where, How) Analysis Action

45 Citizen-Centered Network
Common Measurements Tool Faye Schmidt for the ANAGEMENT DEVELOPMENT Citizens First Erin Research Inc THE CITIZEN-CENTRED IPAC 1999 Gold Award A network of Federal/Provincial/Municipal civil servants dedicated to improving the quality of public service in Canada

46 Citizens First Five Drivers of Satisfaction
For the first time based on Erin’s Survey results we understand the key drivers of satisfaction with government services timeliness competent courtesy fairness, and outcome but of the 5 drivers, timeliness is the single strongest determinant of service quality, yet only about 50% of citizens stated that they were satisfied with the time it took to get the services they needed. 1. Timeliness 2. Competence 3. Courtesy 4. Fairness 5. Outcome

47 Customer’s Priorities for Improving Access to Government Services
Access a major weak spot 29% citizens can’t find service 89% experience at least 1 problem When asked about their priorities for improving services, citizens want improved phone service they want to be able to find these services more easily in the phone book they want expanded electronic access, and ... expanded hours of service Making government services easier to locate in the phone book Improving telephone service Expanding electronic access Expanding hours of service Standards for routine services

48 Common Measurements Tool
Objectives: Provide a ready-made tool for public service organizations Enable like-organizations to compare results Enable organizations to build internal benchmarks Facilitate the sharing of information gained and “lessons learned” between organizations

49 Elements of the Service Experience...
Client expectations Client perceptions of service experience Level of satisfaction Level of importance Priorities for improvements

50 CMT - Service Dimensions
Responsiveness Timely delivery of service # of contacts to receive service Waiting time Courtesy of service staff Reliability What was promised was delivered Errors detected and corrected Access & Facilities Hours of service Telephone access Communications

51 The CMT Go to tool

52 Customer Feedback Framework
Plan Construct Conduct Analyze Act

53 Step 3. Conducting Surveys
Will Focus on Mail Surveys (80-90%) How to achieve a high response rate Data collection and coding Data entry options CMT Data Entry options

54 Mail Surveys Mail surveys can achieve response rate of 60%
Begin with good data base unique id #, relevant customer characteristics, mailing address, mail out dates and date responses receive Professionalism in all Materials questionnaire, cover letter, return envelope with self addressed stamp envelope; all professional look

55 Ensuring high response rate
Include cover letter; explaining purpose of survey (print on official letter head) Booklet format recommended (see sample) Use stamp and hand written address Objective is “coffee table adhesion factor” Follow up letter is a must Generates an additional 50% response

56 Other tips Avoid mailing near holidays use “hot-line” or contact #
use admin # and explain this is to avoid multiple contacts 1st Mail out 3rd Mail out 2nd Mail out

57 Telephone Surveys Very common form of survey administration
Advantages are: faster offers more control Call response ratio 10:1 CMT provides Computer Assisted Telephone Interview option

58 Internet Surveys Advantages Disadvantages
fast, cheap, automated data collection/analysis Disadvantages susceptible to sample biases too easy to administer security/privacy concerns More info available on request

59 Data entry options CMT External data entry Manual entry options
contact Service 1st Manual entry options On-line survey Data entry spreadsheet sheet Bar coding

60 Customer Feedback Framework
Plan Construct Conduct Analyze Act

61 Analysis Quantitative Cross tab example
Frequencies, percentiles, percentages Central tendencies (mean, mode, median) Cross tabulations that array independent variables against dependent variables e.g. customer type who were satisfied Cross tab example

62 7 Tools of Quality Cause and effect diagram Check Sheets
Control charts Flow charts Histogram Pareto charts Scatter Diagram See Handout

63 CMT Analysis For each question For each section Service Standards
Bar/Pie charts Frequencies Percentages For each section Satisfaction/Importance Actual Vs. Expected Priorities for Improvement Service Standards

64 Exercise 3 In groups of 4 review the Case Study
What are the main findings? What are the clients priorities for improvement? What further analysis do you recommend? How would you present the results to executive management? 30 Minutes

65 Customer Feedback Framework
Plan Construct Conduct Analyze Act

66 Acting on Results What to do with the feedback?
Step 1. Assess what the feedback is telling you. CMT for example allows you to evaluate 5 drivers of satisfaction, estimate service standards, understand relationship b/w satisfaction and importance and ID priorities for improvement.

67 Acting on Results How good is good enough?
Satisfied vs.. Very satisfied 4.0/5,0 vs. 4.5/5.0 Nature of organization effects rating

68 Action Steps Recover Report Brief Prioritize Communicate Improve
Enhance Reward Plan Feed into Strategic/Business Planning Adapted from Customer Service Guidelines EPA

69 Defining Action Paths Process involves
identifying owners for each issue or area targeted for improvement Getting owners to own readiness for action linking issue to relevant processes, systems using learning achieved above to define next steps See Brandt

70 Satisfaction and Problem Tracking
LOGGING Problem description Action required INVESTIGATION CLASSIFICATION Bridge to Analysis FORMULATION ANALYSIS Statistics Diagnostics Policy RESPONSE AND FOLLOW-UP PREVENTION Training/Staffing Process Improvement Innovation REFERRAL OR SPECIAL HANDLING Technical Assistance Research Programs (TARP)

71 Organizational Readiness

72 Wrap - up Review Objectives Tools and References

Download ppt "Common Measurements Tool Client Feedback Workshop"

Similar presentations

Ads by Google