Survey design

What is a survey?? Asking questions – questionnaires Finding out things about people Simple things – lots of people What things? What people?

Why do a survey? Cheap (!) Quick (!) Need data on a population

Surveys What people think, feel and do and are.… Self-reported beliefs, knowledge, opinions (questionnaire) What they are like (field tests) Quantitative methodology numbers / statistics Cross-sectional (a snapshot) differences and relationships NOT causes

Surveys can be:- Self-report questionnaires Interviewer-administered questionnaires Field tests e.g. physical, psychological, behavioural

Main requirements of surveys Representative sample generalisability (to the population) sub-group analyses reduce sampling bias Meaningful results validity of questions, scales, tests accurate descriptions, differences and relationships High response rate reduce response bias

Surveys Find a few people……. Ask a few questions……. Get my tutor to do the statistics….. NO!

So you want to do a survey…? PROS quick results relatively cheap can use relatively simple measures high levels of statistical power can involve simple statistics

So you want to do a survey…? CONS substantial planning needed sampling can get complicated time consuming data cleaning / entry need large numbers cannot identify cause and effect can involve complicated statistics complicated logistics

Value for money Rigour Right research question(s) Right size Internal validity External validity Reliability Appropriate design Competent team Feasibility Time Personnel Access Ethics Not too big Not too small Data is NEEDED Costs are justified

So, lets design a survey

Survey planning State the research questions Define your population Define your main dependent variables (DV) Define your independent variables (IV) Choose questions, scales, tests and analyses What will be your sample? Design sampling methodology Estimate sample size

Research questions (permissible) What is the prevalence of….? How tall / overweight / active / fit are..? Is there a difference between….? What factors are related to….? What factors predict…?

Research questions (not permissible) What causes x…..? What effect does x have…..?

Establish your aim Establish your research question(s)

Data collection Use questionnaire / data sheets that make coding and data collection easy Use simple questions For field tests:- Ensure test accuracy Calibrate equipment Standardise test administration Record data systematically and safely

For each research question: What variables must you measure? How will you measure them? Hw will you analyze the data?

Populations Do you need population data What is a population? A group of people Carefully defined Inclusion / exclusion criteria What is your population?

Considerations Access to population Nature of the population (ethical implications) Existence of sampling frame Lists, directories, addresses, tel. numbers Access to sampling frame Access to sample

Define your population

Sampling A way of representing a large population through the study of a smaller number selected from that population Saves time and money BUT - introduces error

The sample must REPRESENT the population as closely as possible....… E.g. Age, sex, socio-demographic profile, ethnicity, environment, Need proportionate numbers of all sub- groups

Sampling Type of people? Number of people? Number of people within each type?

Individuals Population country, city, university, school

Simple random sampling Everyone in the target population has an equal chance of being selected

Population - country, city, university, school Do you have a SAMPLING FRAME? Strata age gender SEP

Population country, city, university, school Clusters cities, suburbs departments, classes

Population country, city, university, school Clusters cities suburbs departments classes Strata age gender SEP Gets complicated…!!

Sources of error Systematic error (bias) e.g. high non-response, interviewer or instrument variation, incomplete sampling frame Response bias Subjects giving socially-desirable answers (lying???!!!) Random error Because your tests are carried out on a sample, not the whole population

Major tasks Identify relevant strata within the population do they influence the DV? Locate the sampling frame Does it contain strata information? Calculate proportionate numbers to sample within each stratum Select the sample

How will you choose your sample?

How many subjects do I need...?? No. of subjects VARIES with: No. of sub-groups Level of sampling error Variability of outcome measures Target differences / relationships Statistical power required Significance level You need to also consider: Multiple outcome measures Logistics Cost Use of results No. of subjects DOES NOT vary with : Size of population

How many participants will you need? [Or – what is the largest number of participants you can cope with?]

Response rate No. of respondents divided by number asked to respond Non-response bias Sample potentially being biased because non- respondents are fundamentally different to respondents Non-participation study Finding out whether non-respondents ARE fundamentally different from respondents