1. S URVEY D ESIGNS Chapter 12 Lauri Cabral Sean Lafontaine

2. O UTLINE – SURVEY RESEARCH What is it? What types are there? What are the features? Types / Features / Design and Construction What are the ethical considerations? What are the steps involved? How do you evaluate and analyze them? What are their strengths & limitations?

3. W HAT IS S URVEY R ESEARCH ? A form of quantitative research in which researchers administer a survey to a sample or to the entire population to describe attitudes, opinions, behaviours, or characteristics of the population Collecting quantitative data using questionnaires, researchers statistically analyze the data to describe trends about responses to questions and to test research questions and/or hypotheses Similar to correlational designs because surveys do not explain cause and effect as experimental research does

4. T RANSLATION....

5. S URVEYS Surveys are effective and efficient instruments used by researchers to gather information on trends, attitudes, and characteristics of a population

6. W HEN D O YOU USE S URVEY R ESEARCH ? To describe trends E.g. trends about levels of bullying in schools To identify individual opinions about policy issues E.g. Your opinion on Bill 115 To evaluate the level of success of programs E.g. The success of the breakfast program To identify important beliefs and attitudes of a population E.g. student beliefs about what constitutes bullying

7. 2 TYPES OF S URVEY D ESIGNS Cross Sectional  Collects data at one point in time  Measures:  current attitudes, beliefs, opinions, or behaviours of one or more groups OR  community needs OR  evaluate programs Longitudinal Trend Studies : examine change within a population over time E.g. Level of bullying in certain schools Cohort Studies : a sub- group of a population that has a common defining characteristic E.g. Parents level of education Panel Studies : the same people are examined over time E.g. The 7 Up Project

8. T YPES OF CROSS - SECTIONAL AND LONGITUDINAL SURVEY DESIGNS ( FIGURE 12.1) Time of Data Collection LongitudinalCross-Section TrendsPanelCohort Attitudes and Practices Group Comparisons Program Evaluation National Assessment Community Needs

9. K EY CHARACTERISTICS OF SURVEY RESEARCH 1. Sampling from a population 2. Collecting data through questionnaire or interviews 3. Designing instruments for data collection 4. Obtaining a high response rate

10. SAMPLING FROM A POPULATION Population Target Population (or sampling frame) Sample

11. T O MAXIMIZE SURVEY RESULTS  Ensure a good target population  Select a large enough sample  Have clear, unambiguous questions and response options  Use rigorous administrative procedures to achieve a large return rate.  RANDOM SAMPLING is the most rigorous form of sampling.... Exception: census studies

12. C OLLECTING DATA Q UESTIONNAIRES AND INTERVIEWS 1. Interview Survey A form on which the researcher records answers supplied by the participant. Questions are asked from an interview guide Researcher listens for answers or observes behaviours and records responses The investigator uses a structured or semi-structured interview consisting mostly of closed-ended questions, provides response options to interviewees and records their responses.

13. 2. T YPES OF QUESTIONNAIRE SURVEYS 1. Mailed Questionnaires: 2. Web based questionnaires: 3. One on one interviews: 4. Focus group interviews 5. Telephone interviews Youth Bullying Survey

14. 3.I NSTRUMENT DESIGN Options: 1. Consider using an existing survey instrument 2. Consider modifying an existing tool 3. When neither 1 nor 2 are options, design your own instrument

15. D ESIGNING AN INSTRUMENT FOR DATA COLLECTION 1. Write different types of questions  Personal, Attitudinal and Behavioural questions  Sensitive questions 2. Use strategies for good question construction  Closed questions  Open-ended questions  Semi-structured questions 3. Perform a pilot test of the questions

16. Q UESTION CONSTRUCTION

17. Must be clear and unambiguous Sensitive to gender, class and cultural needs of participants Ensure question has a clear response and that the participant will be able to answer

18. COMMON CONSTRUCTION PROBLEMS Question is unclear Multiple questions within a single question Question is too wordy Question is negatively worded Question includes jargon There are overlapping responses Unbalanced response options A question includes overly technical language Not all questions are applicable to all participants

19. P ILOT TESTING A test run of your survey to ensure it does what it was designed to do Always pilot test your survey! E.g. Neighbourhood Aesthetics Observational Survey

20. R ESPONSE RATE Researchers seek a high response rate that is not biased A high response rate creates a stronger claim in generalizing results Interviews obtain higher response rates To encourage high response in questionnaires, certain methods are used: Pre-notify participants that they will receive a questionnaire Use good follow-up procedures Study a problem of interest to the population being studied Use a brief instrument Consider the use of incentives

21. R ESPONSE BIAS Occurs when the responses do not accurately reflect the views of the sample and the population Wave analysis is a procedure to check for response bias. Investigators group returns by intervals and check to see if the answers to a few select questions change from the first week to the final week of the study E.g. Failing a test in the middle of a month may change a student’s attitudes toward the teacher

22. C ONSTRUCTING A MAILED QUESTIONNAIRE The Cover letter: Highlights importance of participant and value of response Purpose of the study, including informed consent Assurances of confidentiality Sponsorship Completion time and returns

23. T IPS FOR QUESTIONNAIRE CONSTRUCTION Short length Begins with straightforward demographic or personal questions that encourage commitment Use a variety of closed ended questions Some open-ended items to encourage elaboration Pleasing layout Closing instructions thanking the participant

24. D ATA ANALYSIS OF A QUESTIONNAIRE ( F IG.12.7) 1. Identify response rate and response bias 2. Descriptively analyse the data to identify general trends 3. Write the report presenting the descriptive results and/or use advanced statistics

25. P OTENTIAL ETHICAL ISSUES Survey research may be exempt from a detailed review by review boards (unless it addresses sensitive topics or minor populations) Incentives can be used, but should not be large Do not overstate the benefits of participating Do not put interviewee safety at risk or be deceptive, misleading or inaccurate. Confidentiality of respondents must be maintained. Links between answers and participants should be made with an ID number that is only known by the researcher. Be careful when reporting a small subset of results that could disclose identity When the project concludes, you are responsible for destruction of detailed/personal survey instrument information

26. S TEPS IN SURVEY RESEARCH Write the Report Analyse the Data Administer the Instrument Develop or Locate an Instrument Determine Survey Design and Data Collection Procedures Identify the Population, target population and Sample If so, identify your research questions or hypotheses Is a survey the best design to use?

27. E XAMPLES Neighbourhood Aesthetics Observational Survey Personality Test www.personalitytest.net/ipip/ipipneo1.htm www.prevnet.ca/BullyingFacts/BullyingSt atistics/tabid/122/Default.aspx

28. T HANK Y OU !

29. D ISCUSSION  Strengths?  Weaknesses?  What do you think?  Questions?