1. Survey Research Chapter 17: How To Design And Evaluate Research In Education James Blackwood AED 615 – Fall Semester 2006

2. Survey Research Topics 1. What Is A Survey? 2. Why Are Surveys Conducted? 3. Types of Surveys 4. Correlational Research 5. Steps In Survey Research 6. Nonresponse 7. Problems With The Instrument 8. Evaluating Threats To Internal Validity 9. Data Analysis 10. Journal Article Employing Survey Research

3. What Is A Survey 1. Gathering of a sample of data or opinions considered to be representative of a whole group or population. 2. Information is collected primarily by asking questions. 3. Information is collected from a sample (or portion) of a population.

4. Why Are Surveys Conducted? 1. Surveys are primarily used to describe characteristics of a population. 2. Researchers use surveys to examine the distribution of the identified characteristics within the population. 3. The description of the population as a whole is inferred by the results obtained from the sample.

5. Useful Surveys (Arlene Fink, 1995) 1. Specific, measurable objectives 2. Sound research design 3. Sound choice of population or sample 4. Reliable and valid 5. Appropriate analysis 6. Accurate reporting of results

6. Types of Surveys There are two major types of surveys.. Cross-Sectional Surveys & Longitudinal Surveys

7. Cross-Sectional Surveys Information collected from a sample of a predetermined population Information collected from a sample of a predetermined population Information is collected at approximately one point in time Information is collected at approximately one point in time If an entire population is surveyed, it is described as being a census (Like the US Census that is conducted every 10 years). If an entire population is surveyed, it is described as being a census (Like the US Census that is conducted every 10 years).

8. Longitudinal Surveys Information is collected at different points in time to study changes that occur over a time period Trend Study – samples a population whose members may change over the course of the study Trend Study – samples a population whose members may change over the course of the study Cohort Study - samples a population whose members do not change over the course of the study Cohort Study - samples a population whose members do not change over the course of the study Panel Study – survey the same sample of a population at different times during the course of the study Panel Study – survey the same sample of a population at different times during the course of the study

9. Correlational Research Correlation of the results of two separate surveys to determine if a relationship exists Scores or results of surveys are examined in this method by calculating correlation coefficients or by preparing contingency tables

10. Steps In Survey Research 1. Define the problem 2. Identify the population 3. Choose the type of instrument to collect the data 4. Design, construct, pilot and refine the instrument 5. Select a representative sample 6. Administer the survey 7. Analyze, interpret, and communicate your findings

11. Nonresponse Nonresponse refers to members of a sample who do not respond to the instrument Total Nonresponse – not responding to the entire survey/instrument Item Nonresponse – not responding to some of the questions within a survey/instrument

12. Reducing Nonresponse Examples of methods to reduce nonresponse… 1. Pretesting an instrument 2. Training interviewers 3. Providing incentives for participation Contacting non-respondents using with an alternate form of the instrument used for testing for nonresponse error

13. Problems With The Instrument Many issues can affect the validity of an instrument Many issues can affect the validity of an instrument 1. Venue/Time (location or time that the instrument is presented) 2. Question preparation (leading questions) 3. External issues 4. Characteristics of the data collector

14. Evaluating Threats To Internal Validity Four main threats to internal validity Four main threats to internal validity 1. Mortailty – removing data from lost members of a study 2. Location threat – data collection in locations that would affect responses 3. Instrumentation – defects in the instrument itself 4. Instrument decay – interviewers get tired or are rushed

15. Data Analysis 1. Summarize responses to draw conclusions from the results 2. Size of sample & percentage of returns reported 3. Percentage of total sample responding for each item reported 4. Percentage of respondents who chose each alternative for each question reported

16. Journal Article Example of Survey Research Public Perception of Extension (1996) – JOE 34(4) Paul D. Warner, James A. Christenson, Don A. Dillman, & Priscilla Salant This article addressed the issues of how people's perception/awareness and use of Extension changed over a 13- year period. Telephone surveys of the U.S. population in 1982 and again in 1995 were utilized as the instrument in this study.

17. Thanks For Listening!