Presentation is loading. Please wait.

Presentation is loading. Please wait.

Using Surveys to Support OD projects You are free to copy and distribute the work under the following conditions: you must attribute the work to “ Mick.

Similar presentations


Presentation on theme: "Using Surveys to Support OD projects You are free to copy and distribute the work under the following conditions: you must attribute the work to “ Mick."— Presentation transcript:

1 Using Surveys to Support OD projects You are free to copy and distribute the work under the following conditions: you must attribute the work to “ Mick Verran ; you may not use the work for commercial purposes; if you alter, transform, or build upon this work, you may distribute the resulting work only under the same or similar license to this one. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License Mick Verran for Boston Facilitators Roundtable Handout Password BFR

2 Issues in gathering data Sensitivities about the role of an outside ‘expert’ Impact of the intervention itself Barriers to participation – distribution, trust Challenges in balancing the loudest voices Availability & accessibility of usable data Whole or representative population required Degree of accuracy required Funding & time available 2 Using Surveys to Support OD Projects Mick Verran

3 Data-gathering methods 3 Using Surveys to Support OD Projects Mick Verran ProsCons Observation, Test, Simulation High face-value. Allows for probing & clarification. Observe hidden dynamics. More accurate than e.g. recall. Comparability. Skilled, knowledgeable observer. Time-consuming, high $. Personal Interview Follow-up to nuance data. Unexpected information. > Quality < Quantity. Time consuming. Analysis and comparisons harder. Focus Group Interaction between viewpoints. Buy-in. Time-consuming. Quantifying data. Skillful facilitator. (Electronic) Survey Low $. Reach large or distributed population. Easy for respondents. Easy to nudge completion. Higher response than paper. Easy data analysis & reporting. Requires internet access. No opportunity for clarifying or probing. (Rare) concerns about anonymity.

4 Survey tools Google Constant Contact SearchGizmo Capricity Qualtronics – QuestionPro – Vertical Response ‘… where I do not speak the language’ https://www.questionpro.com/https://www.questionpro.com/ ‘Analysis, vendors who specialize in analysis and reporting, current tools and trends, live poling, web-based live poling’ – Bruce Katcher 4 Using Surveys to Support OD Projects Mick Verran

5 ‘How others are using surveys’ 5 Using Surveys to Support OD Projects Mick Verran Download Handout 1

6 ‘Other ways to leverage surveys’ 6 Using Surveys to Support OD Projects Mick Verran Enter, Scope Initiate, Analyze Design, Plan Implement Sustain, Exit M M O O V V E E MotivationOwnershipVoiceExpectation (Role = moving the process along) Download Handout 2

7 ‘What makes for a good survey?’ Everyone concerned agrees that the survey asks an important question that each respondent you need to hear from will understand, know their answer to, and be willing and able to answer, to produce information you can reliably interpret and use. 7 Using Surveys to Support OD Projects Mick Verran

8 Online survey workload Activity Agree survey goals, measures, population, design considerations, follow-up actions, schedule Draft questions, intro & nudge s, example report Review with client Create online survey Launch Monitor & nudge Draft results Review results with client Finalize report of results 8 Using Surveys to Support OD Projects Mick Verran Additional Workload - 2 hours - < 1 hour - 1 hour - < 1 hour Activity Additional Workload Watch-out for …. Agree survey goals, measures, population, design considerations, follow-up actions, schedule - Vague follow-up actions Push for demographics Shift patterns Draft questions, intro & nudge s, example report 2 hours Review with client- ‘Scope creep’ Create online survey< 1 hour Launch- Monitor & nudge< 1 hour Draft results1 hour Review results with client- Heavy-handed editing Vague follow-up actions Finalize report of results< 1 hour

9 Needs of interested parties 1.Inform the content of what you’re doing 2.Move the process along Client Validation Efficiency > Participation Temperature check Survey Participants A voice Understand what the others involved think. Consultant Reach around the client to uncover hidden issues Platform for success Attune to the language and issues 9 Using Surveys to Support OD Projects Mick Verran

10 Applications – Goals 10 Using Surveys to Support OD Projects Mick Verran Download Handout 3

11 Response rates Online, ‘out of the blue’ market research = 30% average Our pre-survey = 27/30 = 90% What impacts the response rate? Connected with issues of importance to the respondents. Promise of report back on results. Credible commitment of the sponsor to address issues. Confidence in confidentiality. Accommodation of e.g. shift schedules. Easy-to-answer. – Questions & scales – Goal; 20 questions or 5 minutes (5 closed or 2 open per minute) 11 Using Surveys to Support OD Projects Mick Verran

12 Choosing respondents Who to ask? Stakeholders – Knowledgeable – Opinion Leaders ‘Gathering data from a large sample…’ ‘Surveying groups for the same project with variants of the survey’ Separate surveys – no value in complexities of creating too much conditional/skip logic/branching/piping 12 Using Surveys to Support OD Projects Mick Verran

13 Employee Engagement Experiences Which ‘drivers of engagement’ do you experience? Commitment How committed are you to our new strategy? Confidence How confident are you that we will successfully execute our new strategy? Concerns What concerns do you have about our new strategy? What concerns do you have about our ability to successfully execute our new strategy? 13 Using Surveys to Support OD Projects Mick Verran

14 Organization/Team Effectiveness Experience How often does the team ……? Evaluation How well does the team do ……? Satisfaction How satisfied are you with the way the team ………? Opportunities for improvement Improvements in which of the following would have a significant impact on the team’s effectiveness? 14 Using Surveys to Support OD Projects Mick Verran

15 Training Needs Analysis Relevance How often do you use this skill at work? Importance How important is this skill to success in your work? Competence How skilled are you in ……? Development Need I need to improve my ability to …. Commitment to Improve How likely are you to attend ….? 15 Using Surveys to Support OD Projects Mick Verran

16 Applications – Measures 16 Using Surveys to Support OD Projects Mick Verran Download Handout 3

17 Survey sections 1. Why doing survey, benefits, how results will be used 2. Time commitment, when required 3. Contacts – content AND completion 4. Call for Honesty – use full scale, specific examples 5.Confidentiality framing 6. Demographic data 7. Questions 8. Open ended comments 9. Thanks 17 Using Surveys to Support OD Projects Mick Verran

18 ‘Collecting demographic info’ Only those that are vital to the achievement of your objectives Not so narrow that one can guess Optional space for self-identification if open to follow-up SurveyMonkey tracks IP address; individual tracking requires unique survey Positioning Demographic questions At the end of the survey, when respondents have developed confidence in the survey and it’s objective OR Early, straight after the reassurance of anonymity, so don’t feel cheated if asked at the end 18 Using Surveys to Support OD Projects Mick Verran

19 So many options! How many times in the last (period of time) have you bought a Coca-Cola? 0 1 – 5 6 – 12 More than 13 How much do you enjoy Coca-Cola? I enjoy Coca-Cola I neither enjoy nor do not enjoy Coca-Cola I do not enjoy Coca-Cola Given a choice, which would you choose? Coca-Cola Pepsi I prefer Coca-Cola to Pepsi; Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree If you were to buy 10 soft drinks, how many of each would you buy? Coca-Cola Pepsi I have chosen Coca-Cola over Pepsi; Always Often About half and half Rarely Never I enjoy Coca-Cola; More than I enjoy Pepsi As much as I enjoy Pepsi Less than I enjoy Pepsi Put these soft drinks in order of your preference; Coca-Cola Pepsi Mountain Dew Are you willing to; Drink Coca-Cola Buy Coca-Cola Offer Coca-Cola to your guests Buy Coca-Cola for your children Consult to Coca-Cola Inc. Invest in Coca-Cola Inc. Work for Coca-Cola Inc.? 19 Using Surveys to Support OD Projects Mick Verran

20 Question types 20 Using Surveys to Support OD Projects Mick Verran Download Handout 4

21 Respondent Tendencies Agree with statements as presented. Respond from general opinion of positives & negatives. Have own definition of e.g. ‘frequently’ v. ‘occasionally’. Avoid using extreme response categories/poles. “I Tend to Disagree” is more negative than “I Tend to Agree” is positive. Portray selves or group in a more favorable light when asked about; Feelings of powerlessness, "kindness" or "benevolence“, honesty, willingness to contribute. Ability to do job, stress levels, company loyalty, value of own contribution, compliance with procedures. 21 Using Surveys to Support OD Projects Mick Verran

22 How the question and the scale work together How important are the following to the success of a restaurant? Very Unimportant Somewhat Unimportant Neither Important Nor Unimportant Somewhat Important Very Important Service Parking Cost 22 Using Surveys to Support OD Projects Mick Verran How important are the following in attracting families to a restaurant? Very Unimportant 1234 Very Important 5 Quality of Service Ease of Parking Value for Money I take the following into account when choosing a restaurant for a family outing; I Strongly Disagree I Mostly Disagree I Neither Agree Nor Disagree I Mostly AgreeI Strongly Agree Quality of Service Ease of Parking Value for Money

23 ‘Preventing bias’ – ‘Examples of what not to do’ Reliability – Problem questions & scales 23 Using Surveys to Support OD Projects Mick Verran Download Handout 5

24 Anchored rating scales Process X produces an acceptable quality output. Strongly Agree Somewhat Agree Neither Agree nor Disagree Somewhat Disagree Strongly Disagree 24 Using Surveys to Support OD Projects Mick Verran Effectively examine events issues, problems and generate solution alternatives. Strongly Agree Somewhat Agree Neither Agree nor Disagree Somewhat Disagree Strongly Disagree Behaviorally-anchored Effectively examine events issues, problems and generate solution alternatives Strongly Agree 1 Makes sound decisions on everyday issues and problems. Somewhat Agree2 Neither Agree nor Disagree 3 Makes sound decisions on complex functional issues and problems. Somewhat Disagree4 Strongly Disagree 5 Makes decisions that take into account broad, system-wide implications Anchored thru question AND scale The output of Process X compared with our competitors is; Far Above Average Somewhat Above Average Average Somewhat Below Average Far Below Average

25 Rating Scale Template – ‘gold standard’ 25 Using Surveys to Support OD Projects Mick Verran Download Handout 6

26 Favorite questions & scales 26 Using Surveys to Support OD Projects Mick Verran Download Handout 7

27 Questions about scales 1 How many points should there be on the scale? How many degrees of difference can respondents discriminate? How many responses will give rise to different actions? Should I include a mid-point? Use is heavily influenced by “social desirability bias”. “… a four point scale without a mid-point appears to push more respondents towards the positive end of the scale.” (a four point scale) “… can … (force) subjects to indicate their opinion when they actually have no opinion and are not simply reluctant to reveal it.” “This study found the opposite … the way in which people will respond to a balanced Likert type scale without a mid-point is content-specific.” (To minimize the use of the mid-point) “… either not include it at all or use scales with many points.” 27 Using Surveys to Support OD Projects Mick Verran

28 Questions about scales 2 Should I start with the “agree” or “disagree” options? Some evidence of a small bias towards left-side options. No evidence of significant bias whether start with positive option or negative in a survey (different when face-to-face due to “social desirability bias”). Should I give people the option to NOT respond to a question? I Don’t Know – I Have No Evidence – I Have No Experience – I Have No Opinion. SurveyMonkey: “By allowing respondents to opt out due to non-familiarity with the question or if it does not apply to their situation, this will help increase the response rate and quality of the collected data.” Exclude the question if too many respondents will be unable to respond. How will high “don’t know” responses impact your interpretation of the results? Omit if it’s impossible NOT to have an opinion on or experience of the issue. 28 Using Surveys to Support OD Projects Mick Verran

29 Sequencing the questions “Respondents asked to answer questions dealing with fraud and waste in government programs, were subsequently MORE likely to oppose welfare programs than those who were asked the questions in the reverse order.” Meeting Review Content we covered Actions you will take What’s left to be done (Logistics) 29 Using Surveys to Support OD Projects Mick Verran Team Assessment Goal clarity Results Roles & responsibilities Goal clarity Processes Roles & responsibilities Resources Processes Results Resources

30 Ok, …. one ‘Tip on using SurveyMonkey’ To allow multiple entries from same computer or IP address 1.Create the survey 2.Click ‘Collect Responses’ 3.Select ‘Web Link’ & click ‘Next Step’ 4.Click ‘Change Settings’ 5.Under ‘Allow Multiple Responses’ click ‘Yes’ 6.Click ‘Save Settings’ 30 Using Surveys to Support OD Projects Mick Verran

31 Deployment schedule (time zones, shift patterns) YouSponsoring Manager Launch -2 days -Heads-up Day 1 a.m. -Distribute link; 3-day deadline Day 2 a.m. Send manager x/n- Day 2 p.m. -Nudge Day 3 a.m. Send manager x/n- Day 3 p.m. -Extend deadline by one day Day 4 a.m. Send manager x/n- Day 4 noon -Count down to deadline Day 5 a.m. Send manager x/n (Leave open for stragglers) Thank you Day 5 p.m. Draft report- 31 Using Surveys to Support OD Projects Mick Verran Download Handout 8

32 Analyzing responses ‘Evaluating data based on … demographics’ SurveyMonkey filters What to do about missing responses What response rate & coverage is critical to the goals? 32 Using Surveys to Support OD Projects Mick Verran

33 ‘Communicating results to the event sponsor’ ‘Improving the presentation of data’ Boston, March 11–14 $380

34 Reporting principles Separate results & interpretation. Low response; how representative are the responses you got? ‘Use-by’ date on the report? Summarize themes in open-ended comments. Respondents’ own words – including typos..pptx for the report,.docx for the record. Present all the data at end in a Table. 34 Using Surveys to Support OD Projects Mick Verran

35 Means, standard deviations, confidence interval A = 2.60 : B = 2.95 : 5 point scale : 12 respondents Very Strongly Agree is not 20% > Mostly Agree “I can do that for you, but I’m afraid I’ll have to add $nx2 to my bill”. 35 Using Surveys to Support OD Projects Mick Verran

36 Presenting the results Responses Strongly Agree Mostly Agree Mostly Disagree Strongly Disagree Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 36 Using Surveys to Support OD Projects Mick Verran Q Mean/ 4.00 Q12.50 Q22.50 Q32.50 Q42.75 Q52.75 Q63.00 Q73.00 Q83.25 Q93.25 Number of Responses to each Rating Option 4 Strongly Agree 3 Mostly Agree 2 Mostly Disagree 1 Strongly Disagree

37 Format options All responses Bar chart % responses Pie chart Changes over time Line graph Side-by-side bar chart Comments Verbatim Themes Word frequency You can struggle with Excel & CSv or PDFs or ….. Manual Templates! 37 Using Surveys to Support OD Projects Mick Verran

38 Priorities for a new CEO Business Development Creating Bank’s vision for growth Generating new sources of revenue Capturing new customers Differentiating Bank through customer service & marketing Operations Management Protecting capital & asset quality Managing the regulatory process Restructuring operating costs Generating capital for investment Organization Leadership Developing an effective executive team Attracting & developing leadership talent Maintaining a healthy culture External Relations Engaging in the political process Connecting with the local community Retaining the confidence of the Board 38 Using Surveys to Support OD Projects Mick Verran

39 Hospital Housekeeping 1234 N/Comment I Strongly DisagreeI Somewhat DisagreeI Somewhat AgreeI Strongly Agree Housekeeping staff adhere to all health and safety standards %39%36%4% Housekeeping staff act appropriately toward patients and visitors %4%61%14%7% Housekeeping staff interact collaboratively with medical staff % 57%0%7% Housekeeping staff respond flexibly to changing needs %50%18%0%4% Housekeeping Supervisors are responsive to Unit needs %43%36%0% Housekeeping staff respond in a timely manner to urgent needs %50%18%0% 39 Using Surveys to Support OD Projects Mick Verran

40 Team surveys 40 Using Surveys to Support OD Projects Mick Verran Team Effectiveness Initiative Implementation

41 Team survey with leader’s response 41 Using Surveys to Support OD Projects Mick Verran

42 360 feedback 42 Using Surveys to Support OD Projects Mick Verran

43 Repeat Using Surveys to Support OD Projects Mick Verran JanuaryAprilJuly Others NROFNROFNROF JanuaryAprilJuly Others NROFNROFNROF 1 Making condescending or dismissive statements N = Never R = Rarely O = Often F = Frequently 2 Talking over and/or interrupting people N = Never R = Rarely O = Often F = Frequently

44 ‘How to summarize open-ended data’ 1 Read through once List the themes Cut ‘n’ paste sentences to themes Cluster overlapping themes Editing decisions; Every theme? Every sentence, or representative quotes? Edit easily-identified sources? Omit obvious grudges? Edit same point by same person over and over? Correct spelling, grammar? 44 Using Surveys to Support OD Projects Mick Verran

45 Strengths Leadership Team members step up & take a leadership role when required. Leadership Team meetings & conference calls are effective. Leadership Team members collaborate with & support each other. Leadership Team members give & receive feedback without becoming defensive or aggressive. XX is willing to listen & be influenced, & offers significant support. Leadership Team members are optimistic about the team's future & ability to accomplish it's mission. Noteworthy Limitations Team goals & responsibilities are not as clear as individual assignments. Team capabilities and resources are a concern. ‘How to summarize open-ended data’ 2 45 Using Surveys to Support OD Projects Mick Verran

46 46 Using Surveys to Support OD Projects Mick Verran ‘How to summarize open-ended data’ 3

47 Event reviews Goals Assess achievement of objectives Isolate impact of content – process – presenter Assess commitment to Actions Identify need for further support & ‘next agenda’ Items Pre-work relevance & use Content Relevance Presenters: Relevance of examples – Responses to questions – Interactions Clarity of action plan Outstanding issues Personal commitment to change Ongoing support requirements 47 Using Surveys to Support OD Projects Mick Verran

48 Handout Password BFR Thank you. You are free to copy and distribute the work under the following conditions: you must attribute the work to “ Mick Verran ; you may not use the work for commercial purposes; if you alter, transform, or build upon this work, you may distribute the resulting work only under the same or similar license to this one. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License


Download ppt "Using Surveys to Support OD projects You are free to copy and distribute the work under the following conditions: you must attribute the work to “ Mick."

Similar presentations


Ads by Google