PowerPoint presentation to accompany Research Design Explained 6th edition ; ©2007 Mark Mitchell & Janina Jolley Chapter 8 Survey Research

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley Overview l Questions to ask before doing survey research l The advantages and disadvantages of different survey instruments l Planning your survey l Administering your survey l Analyzing your data

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley Questions to Ask Before Doing Survey Research l Do you have a clear hypothesis? l Do your questions focus on that hypothesis? l Will participants answers provide accurate answers to your questions?** l To whom will your results apply?**

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley Answers May Be Inaccurate Because: l People don’t know the causes of their own behavior l People’s memories are inaccurate l People are not good at predicting their future behavior l People are not telling you what they know because of –Social desirability bias –Demand characteristics –Response sets

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley Generalizing Your Results l Do you know what your population is? l How can you get a representative sample of that population? l Will nonresponse bias prevent you from generalizing your results to that population?

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley Advantages and Disadvantages of Survey Research:Conclusions l Easy way to get a lot of information l However, that information: –Will not have internal validity –May not have construct validity because of self-report problems – May not have external validity because of poor sampling or because of nonresponse bias –May not answer research question because survey questions weren’t focused on hypotheses

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley The Advantages and Disadvantages of Different Survey Instruments l Written Instruments * l Interviews*

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley Written Instruments l Self-administered questionnaires –Cheap, easy to distribute to large sample--but nonresponse bias is big problem –Anonymous which may promote honest responses l Investigator-administered questionnaires –Higher response rates –May hurt sense of anonymity and thus decrease honesty of responses l Note: A highly refined version of the investigator- administered questionnaire is the psychological test**

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley Psychological tests l May be part or all of a questionnaire l Even if you don’t use questions from tests, you can try to use strategies that test-makers use**

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley Things Questionnaire Writers Can Learn from Test Makers l Pretest questions and measure l Standardize how measure is administered l Be objective l Be aware of response sets l Seek evidence for measure’s reliability and validity

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley Interviews l May be worth added expense if –It increases response rate –Need ability to clarify questions, follow up on responses l May not be worth added expense if construct validity is harmed by –Interviewer bias –Participant trying to impress interviewer l Telephone interviews may be ideal solution**

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley Advantages of Telephone Interviews l Cheaper than face-to-face interviews l Can have high relatively high response rate l Interviewer bias controlled by –Lack of body cues –Monitoring/taping interviewer’s calls

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley Planning a Survey l Deciding on a research question l Choosing the format of your questions** l Choosing the format of your interview--if you use an interview** l Editing your questions** l Sequencing your questions** l Refining your survey instrument** l Choosing a sampling strategy**

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley Choosing the Format of Your Questions l Fixed alternative –Yes/No l Reliable l Not powerful –Likert l Open-ended –May not be properly answered –May be difficult to score

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley Choosing the Format of Your Interview l Unstructured –Interviewer bias is a serious problem –Data may not be hard to analyze l Semi-structured –Follow-up questions allowed –Probably best for pilot studies l Structured –Standardized, reducing interviewer bias

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley Editing Questions: Nine Mistakes to Avoid 1. Avoid leading questions 2. Avoid questions that invite the social desirability bias 3. Avoid double- barreled questions 4. Avoid long questions 5. Avoid negations 6. Avoid irrelevant questions 7. Avoid poorly worded response options 8. Avoid big words 9. Avoid ambiguous words & phrases

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley Sequencing Questions l To boost response rate, put innocuous questions first, personal questions last l To avoid wasting time, qualify early l To increase accuracy, keep similar questions together l To boost response rate, put demographic questions last

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley Putting the Final Touches on Your Survey Instrument l Professional appearance l Proof reading l Pilot testing l Practice coding responses--may lead to refining questionnaire so that it is easier to code responses

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley Choosing a Sampling Strategy l Random sampling –Proportionate stratified random sampling l Convenience sampling –Quota sampling l Conclusions –Only random sampling will be representative –Nonresponse bias may ruin your sample

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley Administering the Survey l Informed consent l Clear instructions l Debriefing l Confidentiality

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley Analyzing Survey Data l Summarizing data l Inferential statistics

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley Summarizing Data l Interval or ratio data –Mean –Correlation coefficients –Tables of means l Nominal data –Frequencies, percentages –Phi coefficient –Tables of frequencies

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley Using Inferential Statistics l Interval or ratio data –Looking at relationships between pairs of variables l If have two groups, could use t-test between means l If not, should use test to see whether the correlation between two variables was significant l Be aware that if you do numerous statistical tests, you may be setting yourself up for a Type 1 error –To look at more than two variables at once, you can do l ANOVA l Multivariate analysis of variance, multiple regression, factor analysis, and other sophisticated tests

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley Using Inferential Statistics l Nominal data –Chi-Square test –Be aware that if you do numerous statistical tests, you may be setting yourself up for Type 1 errors

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley Concluding Remarks l Survey research is the most used research method l Survey research is the most misused research method l You know how to use rather than abuse survey research

PowerPoint presentation to accompany Research Design Explained, 6th edition ; ©2007 Mark Mitchell & Janina Jolley The End