1. Survey Designs EDUC 640- Dr. William M. Bauer
Chapter 13
Survey Designs
EDUC 640- Dr. William M. Bauer l

2. Key Ideas History of Survey Research Types of Survey Designs
Key Characteristics of Designs
Constructing Questionnaires
Conducting Interviews
Steps in the Survey Research Process
Criteria for Evaluating Survey Research l

3. A Brief History of Survey Research
1817 International Survey of Education Systems
1890 Stanley Hall survey of children
1907 the Pittsburgh Survey examined social problems l

4. A Brief History of Survey Research
WWI and WWI sampling techniques improved
Applications began to vary from social science to media
Universities established social science research centers
Private organizations such as Gallup, Rand Corporation, and Roper formed l

5. Types of Survey Designs
Cross-sectional longitudinal
Longitudinal
trend
cohort
panel l

6. Types of Cross-sectional Longitudinal Survey Designs
Time of Data Collection
Study Over Time
Study at One Point in Time
Longitudinal
Cross-sectional
Trends in
the same
population
over time
Changes
in a
sub-population
group identified
by a common
characteristic
over time
Changes
in the
same
people
over time
Attitudes
and
Practices
Community
Needs
Program
Evaluation
Group
Comparisons
National
Assessment
Trend
Cohort
Panel l

7. Key Characteristics of Survey Designs
Sampling from a population
Collecting data through questionnaires or interviews
Designing instruments for data collection
Obtaining a high response rate
Designing and using a mailed questionnaire
Conducting an interview survey l

8. Population, Target Population, Sampling Frame, Sample
The Population
is the group of individuals having one characteristic that distinguishes them from other groups.
The Target Population or Sampling Frame
is the actual list of sampling units from which the sample is selected.
The Sample
Is the group of participants in a study selected
from the target population from which the
researcher generalizes to the target population. l

9. Collecting Data Through Questionnaires
Mailed questionnaires
Electronic questionnaires
One-on-one interviews
Focus group interviews
Telephone interviews l

10. Forms of Data Collection in Survey Research
Who Completes or Records the Data?
Participant
Researcher
Mailed
Questionnaire
Electronic
Questionnaire
One on
One
To a
Group
Over
Telephone
Individual
Interview
Focus Group
Interview
Telephone
Interview l

11. Designing Instruments For Data Collection: Types of Questions
personal
attitudinal
behavioral
sensitive
scale
open-ended l

12. Question Construction Problems
Question is unclear
Multiple questions
Question is wordy
Question is negatively worded
Overlapping responses l

13. Question Construction Problems
Unbalanced response options
Mismatch between questions and answers
Question includes overly technical language
Not all questions are applicable to all the participants l

14. Pilot Testing Test on a small number of individuals in the sample
Ask for written feedback on the questions
Revise the survey based on the written comments
Exclude the pilot participants from the final sample for the study l

15. Obtaining a High Response Rate
Mail out original questionnaire with cover letter
Follow it in two weeks with a second mailing of questionnaire to those who have not responded
Follow again in two weeks with a postcard to non-respondents reminding them to complete the questionnaire l

16. Three Phase Survey Administration Procedure
Step 1:
First Mailing
of Survey
Step 2:
Second Mailing
of Survey
Step 3:
Postcard
Mailing
2 Weeks
2 Weeks
2 Weeks
Start
6 Weeks
Time l

17. Planning and Designing a Mailed Questionnaire
Write a cover letter to invite the participants to complete the questionnaire
Form and construct the questionnaire
Identify what statistical procedures will be used to analyze data from the mailed questionnaire l

18. Conducting an Interview Survey
The interviewer should maintain a neutral stance during the interview
Train the interviewers prior to the interview
Take good notes of responses or use a tape recorder
For telephone interviews, develop a telephone interview guide prior to the interview l

19. Steps in Conducting Survey Research
Decide if a survey is the best deign to use
Identify the research questions or hypotheses
Identify the population, the sampling frame and the sample
Determine the survey design and data collection procedures l

20. Steps in Conducting Survey Research
Develop or locate an instrument
Administer the instrument
Analyze the data to address the research questions or hypotheses
Write the report l

21. Criteria For Evaluating Survey Research
Was the target population or sampling frame clearly specified?
Was the sampling procedure specified? If a random sample was not used, were modifications from it explained?
Was the sample clearly identified and the basis on which it was chosen specified? l

22. Criteria For Evaluating Survey Research
Did the type of survey match the questions or hypotheses advanced by the author?
Was it clear whether a questionnaire or an interview survey comprised the form of data collection? Did the researcher identify the basis for selecting or developing the instrument? l

23. Criteria For Evaluating Survey Research
Was information reported on reliability and validity of scores from past use of the questionnaire or interview?
Did the author mention the dates on which the questionnaire or interview was administered?
Were the administration procedures clearly identified? l

24. Criteria For Evaluating Survey Research
If a questionnaire was used, were the procedures for obtaining responses identified? Were follow-up procedures used to ensure a high response rate? l

25. Criteria For Evaluating Survey Research
If an interview was used, were the field procedures for collecting this information specified? Were demographic characteristics, previous experience, training, and monitoring discussed for the interviewer(s)?
Did the data analysis match the research questions or hypothesis to be answered in the study? l

26. Criteria For Evaluating Survey Research
Did the researcher check on the reliability and validity of scores from the data collection in the study?
Was the study written scientifically and ethically? l

27. Applying What you Have Learned: A Survey Study
Review the article and look for the following:
The research problem and use of quantitative research
Use of the literature
The purpose statement and research hypothesis
Types and procedures of data collection
Types and procedures of data analysis and interpretation
The overall report structure l