Presentation is loading. Please wait.

Presentation is loading. Please wait.

Designing Your Survey. By the end of this lesson you will be able to: Define the 6 goals of the questionnaire survey. Differentiate between a biased and.

Similar presentations


Presentation on theme: "Designing Your Survey. By the end of this lesson you will be able to: Define the 6 goals of the questionnaire survey. Differentiate between a biased and."— Presentation transcript:

1 Designing Your Survey

2 By the end of this lesson you will be able to: Define the 6 goals of the questionnaire survey. Differentiate between a biased and an unbiased survey question. Develop survey questions from preliminary objectives.

3 Designing Your Survey Design a survey that: — has appropriate sample sizes and strategy — measures the impact of your Pride campaign on SMART objectives for each target audience — measures exposure to your Pride campaign activities

4 Designing Your Survey Learning to Detect Bias and Poorly written questions Reading Activity Writing the best possible survey questions SESSION TOPICS AND TIMING Creating Questions from Preliminary Objectives Drafting Your Stages of Behavior Questions Determining Sample Sizes and Calculating Your Sites Sample Size Assignment, Rubrics and Research Plan Review

5 Designing Your Survey Activity 1: Learning to Detect Bias and Poorly Written Questions (Basics) PURPOSE:  To practice recognizing “bias” on questions and identify what makes a question biased 45 minutes

6 Designing Your Survey Revisit the Theory of Change

7 Designing Your Survey INSTRUCTIONS: 1.Read the Designing Your Survey Lesson Content (pp ) – 20 minutes 2.Proceed to your workshop group (3-4 persons per groups) and answer the guiding questions in pages – 10 minutes 3.Record your groups responses and report your results (5 minutes) ACTIVITY 2: READING

8 Designing Your Survey Why Conduct an Interview Survey Surveys are research tools that provide quantitative information about people in your target area. The systematic sampling will make your respondents “representative” of the population of interest. Surveys are ideal tools to measure change in your target audiences because you can measure the “baseline” and post-campaign levels. — The difference between baseline and post- campaign survey results provides an estimate of the impact of your campaign on that variable.

9 Designing Your Survey Questionnaires and your Pride Campaign You will implement 2 surveys during your campaign: — During the planning phase (baseline) — At the conclusion of your camp

10 Designing Your Survey Goals for your survey 1. Describe your Target Audience(s) “ 90 % of all fishers surveyed are male and 60 % of those are over the age of 40”

11 Designing Your Survey Goals for your survey– pre campaign 2. Assign Respondents to their Appropriate Stage-of-Behavior- Change “Among fishermen surveyed, 5 percent are currently setting rattraps on their boats, 5 percent are currently thinking about setting rattraps...”

12 Designing Your Survey Goals for your survey– pre campaign 3. Establish a Baseline for your SMART Objectives “30% of Fishers know that rats get to Serena on fishing boats” can be the basis for setting the SMART Objectives: “By 2013, the percentage of Fishers who know that rats travel to Serena on their boats will increase from 30% to 50%

13 Designing Your Survey Goals for your survey– post campaign 4. Measure the impact of your PRIDE campaign in achieving your SMART Objectives IC Interpersonal communications A Attitude K Knowledge Control site

14 Designing Your Survey 5. Measure the Audience Exposure to your Pride Campaign and Attribute Effects to your Pride Campaign CARE FOR THE LASTING PRIDE A campaign to protect Tagbilaran City’s underwater kingdom Goals for your survey– pre campaign

15 Designing Your Survey Goals for your survey– pre campaign 6. Ground Truth – test your assumptions about community, strategy, and ToC "99% of community members know about the importance of keeping rats off of the island”… what does this tell you about your ToC?

16 Designing Your Survey DESIGNING YOUR QUESTIONNAIRE

17 Designing Your Survey A Note of Caution… Developing and implementing quantitative surveys is a lot of work bias questions ask questions of the wrong people, or at least not enough of the “right” ones, of your critical target audience(s) ask too many questions make mistakes in choosing your samples -  your baseline and post-campaign samples are not comparable make mistakes during data entry  you enter the wrong data into the wrong variable and end up with meaningless results

18 Designing Your Survey Biased Questions The most important challenge of survey questions is that they can be BIASED: Biased questions mean: they make respondents tend to pick one response over another because of the way the question is worded or the way the response options are offered.

19 Designing Your Survey Biased Questions Biased questions: Try to educate Assume what they ask (are leading) Are double-barreled, asking 2 questions at once Are ambiguous or confusing

20 Designing Your Survey Key Elements of Good Survey Questions?  Unbiased – not leading or pushing respondents towards a specific answer  Tied to Objectives  Time bound  Clear  Not Judgemental

21 Designing Your Survey Writing good survey questions Other important criteria for good survey questions: Include a relevant time frame Use filter questions as needed Don’t overlap (e.g. age ranges) Use appropriate language for the respondents

22 Designing Your Survey Activity 3: Writing the Best possible Survey Questions Purpose: To Practice writing “good” survey questions and appropriate response choices Instructions: (Turn to pages ) 1.Work in pairs 2.Read through each of the survey questions assigned to you and the corresponding responses found 3.For each of the question, do the following: a.Identify what makes this as poorly written questions

23 Designing Your Survey Survey Questionnaire #1

24 Designing Your Survey Survey Questionnaire #2

25 Designing Your Survey Survey Questionnaire #3

26 Designing Your Survey Survey Questionnaire #5

27 Designing Your Survey Survey Questionnaire #6

28 Designing Your Survey Survey Questionnaire #7

29 Designing Your Survey Survey Questionnaire #8

30 Designing Your Survey Survey Questionnaire #9

31 Designing Your Survey Survey Questionnaire #10

32 Designing Your Survey Survey Questionnaire #11

33 Designing Your Survey YOUR QUESTIONS?

34 Designing Your Survey 1.Read and review the lesson content especially on Biased and Unbiased questions (pages ) 2. Read Lesson Content pages Write in a piece of paper 2-3 questions (per participant) that you would want to be clarified or know more. READING ASSIGNMENTS FOR TOMORROW

35 Designing Your Survey REQUIREMENTS: Identified Preliminary Target Audience Preliminary SMART Objectives DESIGNING YOUR QUESTIONNAIRE

36 Designing Your Survey How Many QUESTIONNAIRES?  ONLY ONE QUESTIONNAIRE (Pre and Post KAP Survey) for all target audiences  Some questions though may not be applicable to some audience  Create a system where some questions are answered by certain target audience while others are not.

37 Designing Your Survey How Many QUESTIONS should I ask?  Factors that can affect your survey are:  the number of respondents  length of questions  who will administer  Data entry  Data analysis and writing  Cost  Include questions that will answer only the six goals of survey

38 Designing Your Survey Where do the survey questions come from? KEY CHARACTERISTICS 1.Date 2.Enumerator Name 3.Form # 4.Geographic Area/ Sampling location 5.Respondent Information: Age, gender, education level, income sources, religion? 6.Knowledge Question 7.Attitude Question 8.Interpersonal Communication Question 9.Behavior Change Question 10.Stages of Behavior Change 11.Barrier Removal (As Appropriate) 12.Exposure to Materials

39 Designing Your Survey SMART Examples

40 Designing Your Survey Behavior Change and Questionnaire Surveys Statement from stage of behavior change question in survey Stage of behavior change it should correspond to I have never considered using rat traps in my fishing boat. Validation stage I am considering using rat traps in my fishing boat but haven’t decided yet. Pre-contemplation stage I intend to use rat traps in my fishing boat in the near future, but I haven’t talked to anyone about it yet. Maintenance stage I have talked to someone about my intention to use rat traps in my fishing boat. Action Stage I used rat traps in my fishing boat at least once.Contemplation stage I always use rat traps in my fishing boat.Preparation stage

41 Designing Your Survey Sample Behavior Change Questions  Clearly define the “desired behavior change” of your campaign  Make one question each along the Stages of Behavior (Behavior Change continuum)  Example:..\..\EXAMPLE behavior change questions.docx..\..\EXAMPLE behavior change questions.docx

42 Designing Your Survey Exposure Questions  Make at least 8-10 Examples of potential materials that you will be producing in your campaign and ask the participants if they have see  See example..\..\EXAMPLE media exposure questions.docx..\..\EXAMPLE media exposure questions.docx

43 Designing Your Survey TOP TIPS 1.Complete each interview in about 30 minutes or less 2.The categories should also be “exhaustive;” that is, there is no other possible response that a respondent wants to give but cannot. 3.Be sure that your questions directly relate to what it is you are studying, especially for those questions intended to measure SMART objectives.

44 Designing Your Survey Top Tips….. 4. Be very aware of the fact that people in your target communities may not have the same vocabulary as you. 5. If you want to force people to choose from a short list of possible responses, use the closed-ended format. 6. If there are some questions that you feel might be sensitive you should put these at the end of the survey so that respondents don’t get upset at the beginning.

45 Designing Your Survey Top Tips 7. Have one person do the first translation from the language the survey was drafted in into the second language. Then, have a different person translate the survey back into the original language (back translation). 8. “Administer the survey to a small number of respondents (five is usually enough) similar to your target population to make sure that categories are clear and complete”.

46 Designing Your Survey Key Elements of Good Survey Questions?  Unbiased – not leading or pushing respondents towards a specific answer  Tied to Objectives  Time bound  Clear  Not Judgemental

47 Designing Your Survey Writing good survey questions Other important criteria for good survey questions: Include a relevant time frame Use filter questions as needed Don’t overlap (e.g. age ranges) Use appropriate language for the respondents

48 Designing Your Survey Writing good survey questions FILTER QUESTIONS In the past 6 months, have you talked to anyone about benefits to the community of a well enforced sanctuary? YES NO If you have, please tell me all of the people with whom you have talked to about this. [ ] Have not talked to anyone [ ] Talked to spouse/partner [ ] Talked to parents, or in-laws [ ] Talked to your children aged 16 or older [ ] Talked to your children aged 15 or younger [ ] Talked to friend or neighbour [ ] Talked to village elder or local authority [ ] Talked to National government environmental officials [ ] Talked to Non-government environment officials [ ] Talked to Local government environmental officials [ ] Talked to a Local XXXX Fisher or Gleaner [ ] Other ________________

49 Designing Your Survey Writing good survey questions  RESPONSE QUESTIONS TO CATEGORIES SHOULD NOT OVERLAP  PROVIDE ACCURATE TRANSLATION  CLOSE VERSUS OPEN-ENDED QUESTIONS

50 Designing Your Survey ORDER OF QUESTIONS  Use transitions between modules – use to switch from one topic to another  Provide Accurate translation  Review and Pretest your Questionnaire

51 Designing Your Survey Criteria for a Good Survey Questionnaire Uses the Rare template as a starting point, and asks the socioeconomic and demographic questions exactly as are in template. There is a clear link to the campaign’s objectives so that the data can help in campaign design, test assumptions and be used to measuring its impact across all stages of the Theory of Change. Questions cover most important factors such as: — Socioeconomic and Demographic profiles — Stages of behavior change question — SMART objectives measurement — Exposure to Campaign Activities & Messages

52 Designing Your Survey Criteria …. Questions are clear, unbiased and appropriate level of language and vocabulary for the target audiences. Interpersonal communication, behavior change, and exposure to Pride activity questions are time-bound, if appropriate, so you can measure change, normally this means they state a time of less than 1 year. Questionnaire did not touch upon socially sensitive issues which introduces bias. Questionnaire takes not more than 30 minutes to administer to a respondent. Questionnaire was pretested.

53 Designing Your Survey INDIVIDUAL/ GROUP WORKTIME Drafting your Survey Questions  Your Draft Questions must follow the template provided to you earlier  Add: Behavior Change Questions Media Exposure Questions

54 Designing Your Survey Conducting the Survey

55 Designing Your Survey Guide to Individual/Group Work ( 60 MINUTES) – 10:00-11:00 1.Review the Serena Island Research Plan (tables 5&6) and pages in your packet 2.Using your preliminary Results Chains and preliminary SMART Objectives begin to develop your site's survey questions. Take the time to develop survey questions 3.Incorporate the newly developed survey questions into the Quantitative Data Collection section of your site's Research Plan. (Section 5.4 & 5.5) ( 30 MINUTES) 11:00-11:30 1.Share with another CF within your group 2.Consult/ Share with your PPM on your Survey Questions

56 Designing Your Survey Other Considerations 1.Filter Questions 2.Response Categories for Questions Should not Overlap 3.Closed versus Open-ended Questions 4.Order of questions 5.Provide accurate translation 6.Review and pretest your questionnaire

57 Designing Your Survey Selecting Your Sample 1.Representative sample 2.Sample size 3.Identify your random sampling procedure

58 Designing Your Survey ASSIGNMENT  ASSIGNMENT #7, Draft Quantitative Survey (ComX 5337)  Points: 30 points  DATE ASSIGNED: 21 September 2012  DATE DUE: 24 September, Monday, 9:00PM  What to Submit: o Draft Survey Questions for K, A, IC, BC, Stage of Behavior Questions and Media Exposure

59 Designing Your Survey Grading CriteriaPoints Tied to SMART objectives Although there are a lot of questions that you might be interested in asking your respondents, the goal of your questionnaire survey is to understand your impact and exposure in relation to your objectives. Ensuring that the survey question is clearly tied to your objectives will make understanding your impact transparent. 10 Unbiased The questions are not leading or pushing the respondent towards a specific answer. 5 Follows logical order The survey should be organized so that simple and non-sensitive questions come first with any sensitive questions coming last. Also, survey should test knowledge of topics before describing what those topics are. Questions should flow in a logical order. 5 Clear If the questions are complex or unclear, your responses will be as well! Simple questions that are not overly complicated by jargon will make responses to your survey precise and accurate. 5 Not judgmental or emotional Overly emotional survey questions will make respondents feel uncomfortable and may answer in a way that doesn’t accurately reflect how they feel. 5 Total 30 Penalty for late submission (-1.5)

60 Designing Your Survey Revisiting Your Research Plan Template SECTION 5.0 QUANTITATIVE DATA COLLECTION 5.1 KAP Survey 5.2 Sampling Design 5.3 Survey Planning Table 5.4 Draft Survey Questions 5.5 Draft Stage-of-Behavior Questions

61 Designing Your Survey By the end of this lesson you will be able to: Define the 6 goals of the questionnaire survey. Differentiate between a biased and an unbiased survey question. Develop survey questions from preliminary objectives.

62 Designing Your Survey Design a survey that: — has appropriate sample sizes and strategy — measures the impact of your Pride campaign on SMART objectives for each target audience — measures exposure to your Pride campaign activities


Download ppt "Designing Your Survey. By the end of this lesson you will be able to: Define the 6 goals of the questionnaire survey. Differentiate between a biased and."

Similar presentations


Ads by Google