We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byNorman Hamer
Modified about 1 year ago
© 2009 by The McGraw-Hill Companies, Inc. Research Methods in Psychology Survey Research
© 2009 by The McGraw-Hill Companies, Inc. Survey Research Survey results describe opinions, attitudes, preferences allow predictions about behavior Survey research uses questionnaires predetermined set of questions a sample represents a population
© 2009 by The McGraw-Hill Companies, Inc. Survey Research, continued Surveys can be limited, specific in scope more global in their goals Are surveys always biased? Don’t assume bias just because a specific organization or company has sponsored the survey Examine the survey procedures and analyses
© 2009 by The McGraw-Hill Companies, Inc. Correlational Research Assess relationships among naturally occurring variables for example: attitudes, preferences, personality traits, feelings, age, sex Correlation coefficients strength and direction of predictive relationship between two variables – negativenopositive relationship
© 2009 by The McGraw-Hill Companies, Inc. Correlational Research, continued Do these relationships indicate positive or negative correlations? As the number of years in which individuals smoke cigarettes increases, the likelihood of lung cancer increases As the frequency of participating in volunteer activities increases, occasions of depressed mood decrease As arousal level increases, the likelihood of retaliation following an offense increases
© 2009 by The McGraw-Hill Companies, Inc. Obtaining a Sample Researchers are not interested simply in the responses of those who complete a survey They seek to describe the larger population from which the survey was drawn Careful selection of a sample allows researchers to generalize findings from the sample to the population
© 2009 by The McGraw-Hill Companies, Inc. Basic Terms of Sampling Population set of all cases of interest examples: current students at your school current residents of your city citizens of the United States Sampling Frame list of the members of a population example: registrar’s list of registered students
© 2009 by The McGraw-Hill Companies, Inc. Basic Terms of Sampling, continued Sample subset of the population used to represent the entire population example: students in this class as a sample of all students at this school (or this city) Element each member of the population
© 2009 by The McGraw-Hill Companies, Inc. Goal of Sampling sample should represent the population characteristics of participants in sample should be similar to those of the entire population example: Which sample represents a population that is 30% freshmen, 30% sophomore, 20% junior, 20% senior? Sample ASample B 30 freshmen, 30 sophomores,60 freshmen, 60 sophomores, 20 juniors, 20 seniors40 juniors, 40 seniors
© 2009 by The McGraw-Hill Companies, Inc. Biased Samples A biased sample occurs when characteristics of the sample differ systematically from those of the population samples can overrepresent or underrepresent a segment of a population samples made up of psychology students overrepresent college students and underrepresent people not in college most research underrepresents individuals from diverse cultures
© 2009 by The McGraw-Hill Companies, Inc. Biased Samples Two sources Selection bias occurs when a researcher’s procedures for selecting a sample result in one or more segments of the population being under- or overrepresented Response-rate bias occurs when individuals selected for the initial sample do not complete and return the survey lack of interest, worried about privacy, don’t have time
© 2009 by The McGraw-Hill Companies, Inc. Approaches to Sampling “Sampling” refers to procedures used to obtain a sample Two basic approaches: nonprobability sampling probability sampling
© 2009 by The McGraw-Hill Companies, Inc. Approaches to Sampling, continued Nonprobability sampling no guarantee each member of population has an equal chance to be in sample “convenience sampling” researcher selects individuals who are available and willing to respond to the survey example: magazine surveys, call-in radio surveys
© 2009 by The McGraw-Hill Companies, Inc. Approaches to Sampling, continued Probability sampling all members of population have an equal chance of being selected for the survey “simple random sample” Two common methods: choose randomly from the sampling frame of the population random-digit dialing
© 2009 by The McGraw-Hill Companies, Inc. Approaches to Sampling, continued Probability sampling Stratified random sample: divide population into subpopulations, called strata Draw random samples from the strata best to select samples proportional to the strata size Stratified random sampling increases the likelihood the sample will represent the population
© 2009 by The McGraw-Hill Companies, Inc. Survey Methods Four methods for obtaining survey data: mail surveys personal interviews telephone interviews Internet surveys Each method has advantages and disadvantages Choose method based on research question
© 2009 by The McGraw-Hill Companies, Inc. Survey Methods, continued Mail surveys quick, convenient, self-administered, best for highly personal or embarrassing topics problem of response rate: people selected for sample don’t complete and return the survey final sample may be biased—not representative of the population little control over how people respond to the questions
© 2009 by The McGraw-Hill Companies, Inc. Survey Methods, continued Personal interviews researchers gain more control over how survey is administered interviewers can seek clarification of answers, ask questions potential problem: interviewer bias occurs when interviewers record only selected portions of answers or changes wording of questions and answers. interviews are costly; interviewers must be highly motivated, carefully trained, supervised
© 2009 by The McGraw-Hill Companies, Inc. Survey Methods, continued Telephone interviews complete brief surveys efficiently and with greater access to population random-digit dialing to select random samples supervise interviewers easily Problems selection bias: no phone or multiple phone numbers response-rate bias: willingness to answer questions on phone interviewer bias: changes in survey questions and responses
© 2009 by The McGraw-Hill Companies, Inc. Survey Methods, continued Internet surveys efficient, low-cost, potential for very large samples samples can be very diverse and access typically underrepresented samples problems: selection bias: access to Internet response-rate bias: willingness to respond lack of control over research environment
© 2009 by The McGraw-Hill Companies, Inc. Survey Methods, continued Ways to increase response rate questionnaire has a “personal touch” use name, not “Dear student” responding requires minimal effort topic of survey is interesting to respondents respondents identify with organization or sponsor of survey
© 2009 by The McGraw-Hill Companies, Inc. Survey Research Designs “Research design” a plan for conducting a research project choose research method best suited for answering a particular question Three types of survey designs cross-sectional design successive independent samples design longitudinal design
© 2009 by The McGraw-Hill Companies, Inc. Survey Research Designs, continued Cross-sectional survey design select sample from one or more populations at one time choose population of interest use probability sampling or convenience sampling probability sampling leads to a more representative sample respondents complete a survey
© 2009 by The McGraw-Hill Companies, Inc. Survey Research Designs, continued Cross-sectional survey design Survey responses are used to describe population (descriptive statistics) make predictions for the population (correlations) at that one point in time If samples are drawn from different populations, compare the populations cannot assess change over time with cross- sectional designs
© 2009 by The McGraw-Hill Companies, Inc. Survey Research Designs, continued Successive independent samples design a series of cross-sectional designs over time a different sample from the population completes the survey each time each sample is selected from the same population responses from each sample are used to describe the population at each point in time
© 2009 by The McGraw-Hill Companies, Inc. Survey Research Designs, continued Successive independent samples design compare survey responses from each sample to see how the population changes over time cannot determine whether individuals change over time Problem of noncomparable samples If different populations are sampled each time, responses may differ because of true changes over time or because different populations were sampled
© 2009 by The McGraw-Hill Companies, Inc. Survey Research Designs, continued Longitudinal survey design same sample of individuals completes the survey at different points in time can assess how individuals change over time responses from the sample are generalized to describe changes over time in the population
© 2009 by The McGraw-Hill Companies, Inc. Survey Research Designs, continued Longitudinal survey design: Problems Attrition: people drop out of the study sample no longer represents population from which it was selected Reactivity: respondents may strive to be consistent over time or become sensitized to the topic longitudinal surveys can’t tell why people change over time (only correlations)
© 2009 by The McGraw-Hill Companies, Inc. Measures in Survey Research Questionnaires most frequently used to collect survey data measure different types of variables demographic variables using checklists preferences and attitudes self-report scales respond using rating scales (assume interval level of measurement)
© 2009 by The McGraw-Hill Companies, Inc. Measures in Survey Research, continued All measures must be reliable and valid Reliability refers to consistency of measurement Test-retest reliability administer measure two times to same sample individuals’ scores should be consistent over time a high correlation between the two sets of scores indicates good test-retest reliability (r >.80) individuals’ scores need not be identical each time, only same place in the distribution of scores
© 2009 by The McGraw-Hill Companies, Inc. Measures in Survey Research, continued How to improve reliability? more items greater variability among individuals on the factor being measured testing situation free of distractions clear instructions A measure can be reliable but not valid
© 2009 by The McGraw-Hill Companies, Inc. Measures in Survey Research, continued Validity refers to the truthfulness of a measure A valid measure assesses what it is intended to measure Construct validity refers to whether an instrument measures the theoretical construct it was designed to measure
© 2009 by The McGraw-Hill Companies, Inc. Measures in Survey Research, continued Example of construct validity: Intelligence: Do these questions from a common measure of intelligence assess a person’s intelligence in a valid manner? comprehension: “Why would people use a secret ballot?” vocabulary: “What does dilatory mean?” similarities: “How are a telephone and a radio alike?”
© 2009 by The McGraw-Hill Companies, Inc. Measures in Survey Research, continued Establishing construct validity: convergent validity extent to which two measures of the same construct are correlated (go together) discriminant validity extent to which two measures of different constructs are not correlated (do not go together)
© 2009 by The McGraw-Hill Companies, Inc. Measures in Survey Research, continued Construct validity example new measure of self-esteem Which constructs should show convergent validity with self-esteem measure and which would show discriminant validity? Constructs: depression, well-being, intelligence, extraversion, age, sensation- seeking, social anxiety, life satisfaction, grade point average, reading comprehension, artistic ability
© 2009 by The McGraw-Hill Companies, Inc. Measures in Survey Research, continued Correlations demonstrating construct validity and reliability are shown in a correlation matrix: (1)(2)(3)(4) (1) new self-esteem, Time (2) new self-esteem, Time (3) measure of positive affect (4) measure of artistic ability test-retest reliability convergent validity discriminant validity
© 2009 by The McGraw-Hill Companies, Inc. Constructing a Questionnaire Best choice for selecting a measure use measure already shown to be reliable and valid in previous research If no suitable measure is found create a questionnaire or measure Creating a reliable and valid questionnaire is not easy
© 2009 by The McGraw-Hill Companies, Inc. Constructing a Questionnaire, continued Important first steps Decide what information should be sought Decide what type of questionnaire should be used (e.g., self-administered?) Write a first draft of the questionnaire Have experts review questionnaire and then revise it based on their suggestions Pretest the questionnaire using sample and conditions similar to the planned administration of the survey Review results and edit the questionnaire
© 2009 by The McGraw-Hill Companies, Inc. Constructing a Questionnaire, continued Next steps: Establish reliability and validity Reliability Test and retest questionnaire using sample and conditions similar to planned survey. Validity Convergent: Administer questionnaire with measures of theoretically related constructs Discriminant: Administer questionnaire with measures of theoretically unrelated constructs
© 2009 by The McGraw-Hill Companies, Inc. Constructing a Questionnaire, continued Guidelines for Writing Survey Questions Choose how participants will respond free-response (open-ended) greater flexibility in responses difficult to code closed-response (multiple-choice, true-false) responses are quick and easy easy to score may not accurately describe individuals’ responses Use simple, familiar vocabulary; keep questions short
© 2009 by The McGraw-Hill Companies, Inc. Constructing a Questionnaire, continued Guidelines for Writing Survey Questions Write clear and specific questions avoid double-barreled questions place conditional phrases at the beginning of sentences avoid leading questions avoid loaded (emotion-laded) questions
© 2009 by The McGraw-Hill Companies, Inc. Constructing a Questionnaire, continued Ordering of questions self-administered questionnaires most interesting questions first personal and telephone interviews demographic questions first use funnel questions start with general questions and move to more specific questions on a given topic use filter questions direct respondents to appropriate questions
© 2009 by The McGraw-Hill Companies, Inc. Thinking Critically About Survey Research Reported vs. Actual Behavior Survey responses may not be truthful Reactivity not truthful because the information is being recorded Social desirability not truthful because responding as they “should”
© 2009 by The McGraw-Hill Companies, Inc. Thinking Critically About Survey Research, continued Measuring social desirability sample items from Marlowe-Crowne (1964) Social Desirability Scale What is the socially desirable response? 1. No matter who I’m talking to, I’m always a good listenerT F 2.I like to gossip at timesT F 3.I’m always willing to admit it when I make a mistakeT F 4.I have almost never felt the urge to tell someone offT F
© 2009 by The McGraw-Hill Companies, Inc. Thinking Critically About Survey Research, continued Guidelines for social desirability Accept people’s responses as truthful unless there’s reason to suspect otherwise Because actual behavior doesn’t always match questionnaire responses, use a multimethod approach to answering research questions
© 2009 by The McGraw-Hill Companies, Inc. Thinking Critically About Survey Research, continued Correlation and causality “correlation does not imply causality” example: correlation between being socially active (A) and life satisfaction (B) three possible causal relationships A causes B B causes A variable C causes both A and B (e.g., number of friends) = spurious relationship
© 2009 by The McGraw-Hill Companies, Inc. Thinking Critically About Survey Research, continued Path analysis statistical procedure to tease apart complex correlational relationships among variables Mediators variables used to explain a correlation between two variables Moderators variables that affect direction or strength of correlation between two variables
© 2009 by The McGraw-Hill Companies, Inc. Thinking Critically About Survey Research, continued Example of path analysis Evans et al. (2005) observed a positive correlation between measures of poverty and measures of psychological distress among children Poverty → Psychological distress This is called a direct relationship or path a
© 2009 by The McGraw-Hill Companies, Inc. Thinking Critically About Survey Research, continued Path analysis example Evans et al. sought to explain why this relationship exists. level of chaos in the home as possible mediator chaotic living environment measured using concepts such as unpredictability, confusion, lack of structure, noise, overcrowding, and poor-quality housing
© 2009 by The McGraw-Hill Companies, Inc. Thinking Critically About Survey Research, continued Path analysis example Evans et al. observed two important correlations Measures of poverty were positively correlated with greater chaos in the home (Path b) Greater chaos in the home was positively correlated with psychological distress (Path c) These two correlations represent the indirect relationship between poverty and psychological distress
© 2009 by The McGraw-Hill Companies, Inc. Thinking Critically About Survey Research, continued Path analysis example Diagram of direct and indirect relationships Chaos path bpath c PovertyPsychological path adistress “Chaos” mediates the relationship between poverty and psychological distress
© 2009 by The McGraw-Hill Companies, Inc. Thinking Critically About Survey Research, continued Path analysis example What if the relationships between poverty, chaos, and psychological distress are not observed for all children? A moderator variable may affect these relationships moderators affect the direction and strength of relationships
© 2009 by The McGraw-Hill Companies, Inc. Thinking Critically About Survey Research, continued Path analysis example Possible moderators: sex of the child the relationships between poverty, chaos, and psychological distress may exist for boys, but not girls population density the relationships between poverty, chaos, and psychological distress may exist in urban areas, but not rural areas personality features of children the relationships may exist for low-resilient children, but not high-resilient children
© 2009 by The McGraw-Hill Companies, Inc. Thinking Critically About Survey Research, continued Path analysis helps us to understand relationships among variables but these relationships are still correlational cannot make definitive causal statements other untested variables are related to children’s psychological distress
Experimental concepts. TheoryTheory. A theory can be defined as a "general principle proposed to explain how a number of separate facts are related." In.
Surveys and Questionnaires Editor: Stephen Murray 2008.
1 Psychological Practical (Year 2) PS2001 Introduction Dr. John Beech.
Research Methods in Crime and Justice Chapter 10 Survey/Interview Research Methods.
Quantitative methods Questionnaire Design. 2 Stage 1: Research aims Stage 2: Literature Stage 3: Research design Stage 4: Instrumentation Stage 5: Piloting.
Copyright © 2010, 2007, 2004 Pearson Education, Inc. All Rights Reserved. Lecture Slides Elementary Statistics Eleventh Edition and the Triola.
Sampling Design & Procedure. Population The aggregate of the all the elements, sharing some common set of characteristics that comprises the universe.
QNT Overview Introductions Admin Syllabus Review material Learning teams Next assignment.
To gain an overall understanding of the survey cycle. To understand the basics of survey design and question development. To consider the advantages and.
How to design a statistical study How to collect data by doing an observational study, performing an experiment, using a simulation, or using a survey.
Research Process, Research Design and Questionnaires.
RESEARCH DESIGN Vivek Patkar
A.MANN ADAPTED FROM HAMILTON AP STATS Chapter 5 Producing Data.
Copyright 2007 McGraw-Hill Pty Ltd PPTs t/a Marketing Research 2e by Hair, Lukas, Bush and Ortinau Slides prepared by Judy Rex 7-1 Chapter Seven Descriptive.
Probability and Statistics Representation of Data Measures of Center for Data Simple Analysis of Data.
Quality Tools and Techniques in the School and Classroom.
What is Primary Research and How Do I Do It? Source: esource/559/1/ esource/559/1/
A2 Sport Psychology. January 2010 D1 - Sita needs to improve her running times to ensure selection for a major competition. (a)(i) Describe one psychological.
IB Psychology Research Mr. Hoffman. To be able to get new findings. To be able to critically analyze existing findings and find their flaws. Psychologists.
Data Collection and Aggregation 2/8/2014 Data Collection and Aggregation 1 Presented by AmeriCorps Program Staff and JBS International.
PLANNING THE AUDIT Individual audits must be properly planned to ensure: Appropriate and sufficient evidence is obtained to support the auditors opinion;
OVERVIEW OF SAMPLE SURVEYS Mehdi Nassirpour,Ph.D. Illinois Department of Transportation This presentation was part of the Applied Sampling Workshop at.
ICAA5151B GATHER DATA TO IDENTIFY BUSINESS REQUIREMENTS.
Principles of Survey Design and Management Mel Kollander
CBR 303: A Hands-On Approach to Developing Survey Tools in Community-Based Research.
QUESTIONNAIRE DESIGN. 1. What should be asked? 2. How should questions be phrased? 3. In what sequence should the questions be arranged? 4. What questionnaire.
Educational Research Chapter 22 Evaluating a Research Report Gay, Mills, and Airasian.
The Lexile Framework An Introduction for Educators Thomas Schnick and Mark Knickelbine Forward by A.J. Stenner An overview created by the Clair E. Gale.
A scientific social research method that involves Selecting a random sample of people to answer some questions Designing a standardized questionnaire.
CBR 302 A Hands-On Approach to Qualitative Methods and Analysis.
© 2016 SlidePlayer.com Inc. All rights reserved.