1. Quantitative Research Methods Survey Research Ebrahim Mohseni - PhD Candidate at U of MD - Director of Pars Advanced Research Scholars

2. Background (1)  In 1916 “Literary Digest” accurately predicted the election of US President Woodrow Wilson Using Postal Survey.  Literary Digest was able to accurately predict the next 4 presidential winners as well.  In 1936, however, failed to predict the winner despite their large sample (2.3 million). Sample bias (sample was composed of affluent Americans who tended to be Republicans)

3. Background (2)  During the same presidential elections, George Gallup surveyed a much smaller sample, using scientific methods, and was able to predict the victory of FDR.  This was the start of using scientific methods to evaluate the situation on the ground and predict future events.

4. Background (3)  Sociopolitical survey research has its roots in the “scientific approach”, seeking to understand the truth (reality) trough objective techniques.

5. The Scientific Approach  Ibn al-Haytham (965–1039 A.D.) is recongnized as the first person who emphasized on the scientific approach: “Truth is sought for its own sake. And those who are engaged upon the quest for anything for its own sake are not interested in other things. Finding the truth is difficult, and the road to it is rough."

6. Elements of the Scientific Approach  Define the question  Conduct preliminary research  Form a hypothesis  Perform experiment and collect data  Analyze data  Interpret the data

7. Elements of Survey Research  Define the question  Conduct preliminary research  Form a hypothesis (Focus Groups) Dependent Vs. Independent variables Questionnaire design  Pre-test  Identify your target population  Identify the most suitable (Practical and effective) sampling technique  Collect data  Analyze data (Objective)  Interpret the data (Subjective) Focus Groups

8. Formulating a Hypothesis (1) Observe  Study the problem  Think about your dependent and independent variables  Formulate your own understanding of the relationship between your dependent and independent variables

9. Formulating a Hypothesis (2)  Read past literature (Lit. review) Evaluate their objectiveness Understand their mode of analysis  Philosophical vs. Mathematical Uncover their independent and dependent variables. Develop your own interpretation of their conclusions.

10. Formulating a Hypothesis (3)  Focus Groups (FG) Conduct intensive direct group interviews with individuals who are in your target population. Uncover group consensus Uncover group disagreements Uncover the relationship between different opinions and mind frames Note: The FG results are not representative.

11. Questionnaire Design (Questions -1)  Is more of an Art than a precise science:  Focus Every Question should only and only focus on a single, very specific issue or topic and be as precise as possible.  When and how do you usually go to work?  What time do you usually leave home for work?  What means of transportation do you usually use to go to work from your home?

12. Questionnaire Design (Questions -2)  Brevity The questions should be as short and to the point as possible.  When was the last time that you went to the doctor for a physical examination on your own or because you had to?  When was the last time you had a physical examination? Questionnaire Design (Questions -1)

13. Questionnaire Design (Questions -3)  Clarity The meaning of the questions must be completely clear to all respondents. ALL respondents must have an identical understanding of the questions.  Do you drink?  Do you drink alcoholic beverages?

14. Questionnaire Design (Questions -4)  State the criteria of evaluation Never assume the criteria to be common knowledge or that it is obvious.  How important is it for stores to carry a large variety of different snacks?  How important is it to you that the store you generally shop at carry a large variety of different snacks?

15. Questionnaire Design (Questions -5)  Avoid giving examples Examples skew the mind toward the given example and not what seeks to elucidate.  Do you watch international news channels such as CNN or BBC?  Do you watch news channels that are internationally broadcasted?

16. Questionnaire Design (Questions -6)  Avoid Overgeneralization Make your scale of evaluation as precise as possible.  Do you participate in national elections?  Did you participate in the previous presidential elections?

17. Questionnaire Design (Questions -7)  Avoid Overemphasis Avoid adjectives or labels that are not conventionally accepted.  Would you favor increasing taxes to cope with the current fiscal crises?  Would you favor increasing taxes to cope with the current fiscal problems?

18. Questionnaire Design (Questions -8)  Avoid Ambiguity Nothing should be ambiguous in a question.  Do your parents work fulltime?  Does your father work more than 30 hours a week?  How about your mother?

19. Questionnaire Design (Questions -9)  Avoid double barrel questions Do you regularly take vitamins to avoid getting sick? Do you regularly take vitamins? Why or why not?

20. Questionnaire Design (Questions -10)  Avoid leading question. Questions should not have any assumptions.  Don’t you see some danger in the new policy?  Do you see any danger in the new policy?

21. Questionnaire Design (Questions -11)  Avoid loaded questions Would you support attacking Iran to prevent it from destroying israel?

22. Questionnaire Design (Questions -12)  Relevant (one size never fits all) The same question cannot usually be used to measure attitudes toward an identical topic across different cultural or sociopolitical landscapes. Hence, the questions must be relevant to the target population. To measure religiosity:  How often do you go to the mosque during a regular month?  How often do you perform your prayers during a regular day?

23. Questionnaire Design (Questions -13)  Vocabulary Limit your vocabulary to words that could easily be understood. For a national survey:  In US: 8 th grade vocabulary  In Iran: 5 th grade vocabulary

24. Questionnaire Design (Questions -14)  Avoid non-questions Do you still beat your wife?

25. Questionnaire Design (Response options - 1)  There are four general types of response options: Pure Open-ended Pre-coded open-ended Close ended Hybrid

26. Questionnaire Design (Response options - 2)  Pure Open-ended: No response options are read. Responses are recorded verbatim.  Best when response options are not known  Difficult to analyze Requires coding

27. Questionnaire Design (Response options - 3)  Pre-coded Open-ended: No response options are read. Responses are placed under pre-determined categories.  Best when response options are inherently leading.  Best when looking for “top of the mind” responses.  Difficult to execute Requires coding

28. Questionnaire Design (Response options - 4)  Closed ended questions: Responses are read and the respondents must select from the list of provided options.  Must be Mutually exclusive  Must be mutually exhaustive (comprehensive)

29. Questionnaire Design (Pre-test)  Once a questionnaire is designed, it must be pre-tested (piloted) to make sure it is ready for the field: Nuclear weapons vs. Atomic Bomb

30. Sampling  Identify your target population Filter questions  Probability sampling Vs. Non-probability sampling SRS – Uniform Probability

31. Face to Face (FTF)  More intimate respondent – interviewer conversation (source of comfort AND interviewer related bias)  Lengthier interview  Longer time before data is collected  Very costly

32. Computer Assisted Telephone Interviewing (CATI)  Best for short and non-intrusive interviews  Better supervision  Sampling problems Landline penetration Mobile phone vs. Landline

33. Postal and Internet  Best for long and intrusive questions  Sampling problems

34. Margin of Error