Presentation on theme: "1 Survey Research UAPP702: Research Design for Urban & Public Policy Class Notes Based on Earl Babbie, The Practice of Social Research Danilo Yanich Center."— Presentation transcript:
1 Survey Research UAPP702: Research Design for Urban & Public Policy Class Notes Based on Earl Babbie, The Practice of Social Research Danilo Yanich Center for Community Research & Service School of Public Policy & Administration University of Delaware
2 Topics appropriate for survey research Used for exploratory, descriptive & explanatory purposes Probably best method to collect original data for describing a population too large to observe directly
3 Guidelines for Asking Questions Choose appropriate question form: two options Open-ended Open-ended--R is asked to provide own answer to question Closed-ended Closed-ended--R is asked to select an answer from among a list provided by researcher Closed-ended questions require the categories of answers to be exhaustive and mutually exclusive
4 Guidelines for Asking Questions, p.2 Make items clear Babbie uses examples of "proposed peace plan"...Which peace plan? And, employment "last week"...Monday through Friday or Sunday through Saturday as the Census Bureau defines week?
5 Guidelines for Asking Questions, p.3 Avoid double-barreled questions Example: Example: "Did you walk to work or carry your lunch?" …..Yes or No Regardless of the answer you get from the respondent, you don’t know what information you have.
6 Double-barreled to the max Question on 1986 Harris poll re: American public opinion about Libya If Libya now increases terrorist acts against the U.S. and we keep inflicting more damage on Libya, then inevitably it will all end in the U.S. going to war and finally invading that country which would be wrong. Respondents were given opportunity to answer “Agree”, “Disagree” or “Not sure”
7 Double-barreled to the max, p.2 Notice the elements contained in the complex statement Will Libya increase its terrorist acts against the U.S.? Will the U.S. inflict more damage on Libya? Will the U.S. inevitably or otherwise go to war with Libya? Would the U.S. invade Libya? Would that be right or wrong? These elements offer the possibility of numerous points of view—far more than the three alternatives offered to the R’s
8 Guidelines for Asking Questions, p.4 Respondents must be competent to answer Babbie uses example of asking students to indicate the percentage of fees to be used for a long list of activities, about which the respondents have little knowledge Respondents must be willing to answer There may be a risk to the respondent in answering the question Reluctance may be due to nature of the information sought, but good science and human subjects requirements dictate caution. Example: Example: asking a staff member of a non-profit agency her/his opinion about the leadership qualities of the administrator Using a technique that guarantees anonymity or confidentiality greatly increases R’s willingness to answer questions
9 Guidelines for Asking Questions, p.5 Questions should be relevant to the Respondent fictitious Babbie uses example of fictitious person, Tom Sakumoto 9% of R's said they were familiar with him. Shows R’s tendency to be helpful to researcher Short items are best Avoid negative items Possibility of misinterpretation is great Babbie uses example of asking R to say if they agree/disagree with statement that "The U.S. should not recognize Cuba". Often, R’s will “read over” the word “NOT”
10 Guidelines for Asking Questions, p.6 Avoid biased items and terms Questions should not encourage a particular response or discourage another. Be wary of "social desirability" of answers Attitudes re: government spending for programs in the General Social Survey, 1989 More support Less support Assistance to the poorWelfare reform Halting rising crime rateLaw enforcement Dealing with drug addiction Drug rehabilitation Solving problems of big cities Assistance to big cities Improving conditions of blacks Assistance to blacks Protecting social security Social security
Surveys “gone wild” National Park Survey National Park Survey 11 NRA Survey 1NRA Survey 1 NRA Survey 2NRA Survey 2 NRA Survey 3NRA Survey 3
12 Questionnaire Construction General questionnaire format should be uncluttered Squeezed-together questionnaires are disastrous Formats for respondents should be clean and clear
13 Questionnaire Construction, p.2 Contingency question A question that only applies to the R’s who answered a previous question in a particular “qualifying” manner Make sure that the directions for answering the subsequent questions are clear for the respondent
14 Contingency question, example Have you ever been abducted by aliens? [ ] Yes [ ] No (go to ques. # 4) If yes If yes: Did they let you steer the ship? [ ] Yes [ ] No If yes If yes: How fast did you go? [ ] Warp speed [ ] Weenie speed
15 Questionnaire Construction, p.3 Matrix questions Advantages Advantages Uses space efficiently, R's will probably complete it faster Format may increase comparability of responses to different questions Disadvantages Disadvantages May encourage researcher to use matrix format when another form would be better Can foster a response-set among R's in which they may develop a pattern
16 Matrix question, example Contact with criminal justice system Have you ever… YesNoRef/DK Been a defendant in a criminal case127 Been a witness in a criminal case127 Reported a crime to the police127 Been the victim of a crime127
17 Matrix question, example 2 Using the A, B, C, D, F grading system that we learned in school, please rate the importance of the following services in this hotel. Hotel service ABCDFRef/DK Dining services 543217 Laundry service 543217 Room service 543217
18 Questionnaire Construction, p.4 Ordering questions in a questionnaire The appearance of one question can affect answers given to later ones As remedy, do not make the questions random, that will look chaotic to the R and confuse the issue Safest solution is sensitivity to the problem
19 Questionnaire Construction, p.5 Questionnaire Instructions COMPLETE Must be clear, concise and COMPLETE Pre-testing the questionnaire ALWAYS PRE-TEST!!!! ALWAYS PRE-TEST!!!!
20 Comparing different survey methods Choosing among mail questionnaire, personal interview and telephone survey Factors influencing coverage and information gathered Mailed questionnaire Personal interview Telephone survey Lowest relative cost132 Highest % return312 Highest accuracy of information213 Largest sample coverage313 Completeness, including sensitive material312 Overall reliability & validity213 Time required to gather information321 Ease of gathering information132 Total number of rankings—1,2,32,2,45,1,21,5,1 Note: 1=most favorable ranking, 2=intermediate ranking, 3=least favorable ranking Source: Delbert Miller. Handbook of Social Research Design and Measurement, 5 th Edition,1991, p. 168
21 Strengths of survey research Useful in describing large populations Are flexible in that you can ask many questions regarding your topic Standardized questionnaires have strength regarding measurement generally Survey researcher must ask the same question of all subjects and… Impute the same intent to all respondents giving a particular response
22 Weaknesses of survey research Standardized questionnaire items often represent the least common denominator in assessing people’s attitudes, etc. Does not deal well with the context of social life Can be inflexible…cannot change the survey instrument if field conditions change
23 Weaknesses of survey research, p.2 Subject to artificiality A person giving a conservative answer to a questionnaire does not necessarily mean the person is conservative Artificiality has two aspects Topic of study may not be amenable to measurement through a questionnaire Studying the topic may affect it… Asking someone whether they think the governor should be impeached when R may have given no thought to it until asked for the opinion
24 Ch 14: Quantifying data, Coding System to translate the responses to questions to numeric expressions that the computer can process Two basic approaches to coding process Begin with developed coding scheme Begin with developed coding scheme Babbie uses occupation as example...but must use scheme that is appropriate for research question. Generate codes from data that you collected Generate codes from data that you collected.
25 Using a developed coding scheme, example Occupation The question to the R would be: What is your occupation? The question to the R would be: What is your occupation? The R’s response is then placed in one of several categories that you have already identified The R’s response is then placed in one of several categories that you have already identified
26 Using a developed coding scheme, example, p.2 What is your occupation? Let’s say that the answer was “nurse” Let’s say that the answer was “nurse” Scheme 1: by type Scheme 1: by type Professional, managerial, clerical, semi-skilled, etc. In this scheme, the R’s occupation would be placed in the “professional” category Scheme 2: by sector of the economy Scheme 2: by sector of the economy Manufacturing, health, education, commerce, etc. In this scheme, the R’s occupation would be placed in the “health” category
27 Using a developed coding scheme, example, p.3 Note: record the response of the R verbatim Place the response into one of your pre-determined categories during the data manipulation phase Use “recode” facility in SPSS Use “recode” facility in SPSS Remember, you can always “aggregate” data Cannot “disaggregate” data Cannot “disaggregate” data
28 Using a developed coding scheme, example, p.4 What is your occupation? Verbatim responses Verbatim responses R1: “nurse” R2: “sell shoes” R3: “build cars” R4: “manager at Wendy’s” R5: “physician” R6: “computer programmer” R7: “retired” R8: “soldier” TypeSector professionalhealth salescommercial semi-skilled laborcommercial managerialcommercial professionalhealth professionalinformation tech retired military
29 Generating codes from data collected, example Student responses to “biggest problem facing college today” Responses can be coded as “academic” or “non-academic” AcademicNon-academic Tuition is too high x Not enough parking spaces x Faculty don’t know what they’re doing x Advisors are never available x Not enough classes offered x Cockroaches in the dorms x Too many requirements x Cafeteria food is infected x Books cost too much x Not enough financial aid x
30 Codebook construction End product of the coding process is the conversion of data items into numerical codes These codes represent attributes composing variables which.... Are assigned locations within a data file Location means the specific column of the data file where, for example, the responses for “gender” would occur Location means the specific column of the data file where, for example, the responses for “gender” would occur A codebook is a document that describes the locations and lists the assignment of codes to the attributes composing those variables It is the fundamental document of the research process It is the fundamental document of the research process
31 Codebook serves two functions Primary guide for the coding process Guide to locating variables in the data file during analysis
32 Example of Coding Instructions Public Attitudes: Crime, Drugs & Public Services Enterprise Community/Wilmington/Statewide Survey Variable Name (column location) Value label 1. ID# (1-3) Continuous (001 through n) 2. Area (4) 1= Westside, 2= West Center, 3= Delaware Avenue 4= Southwest, 5= Eastside, 6= Northeast 7= Northwest, 8= New Castle, 9= Kent, 10= Sussex 3. Defendant in criminal case (5) 1=Yes, 2=No, 7=Ref/DK, 8=NA 4. Witness in a criminal case (6) 1=Yes, 2=No, 7=Ref/DK, 8=NA 5. Report crime to police (7) 1=Yes, 2=No, 7=Ref/DK, 8=NA 6. Victim of crime (8) 1=Yes, 2=No, 7=Ref/DK, 8=NA
33 Data Cleaning...a fundamental activity The process of detecting and correcting coding errors Two types Possible-code cleaning--for any given variable there are only a specified set of codes possible Possible-code cleaning--for any given variable there are only a specified set of codes possible Example, gender--computer program would "beep" when an erroneous code is entered and refuse the code Family Court experience with gender codes Contingency Cleaning--process of checking only those cases that should have data on a particular variable do in fact have such data Contingency Cleaning--process of checking only those cases that should have data on a particular variable do in fact have such data
Your consent to our cookies if you continue to use this website.