1. ST3004: Research Methods Data Collection and Sampling

2. Research Question
Literature Review
Research Design
Data Collection
Data Analysis
Write Report
Present

3. The Research Onion
Source: © Mark Saunders, Philip Lewis and Adrian Thornhill 2008

4. Sources of Data Secondary Data Primary Data
Information gathered from sources already existing
Examples: Company records or archives, government publications, industry analyses offered by the media, websites
Primary Data
Information obtained first hand by the researcher on the variables of interest
Only to be done if secondary data is not available or not adequate
Examples: Interviews, surveys, focus groups, panels…

5. Secondary Data - Advantages
Easily accessible, relatively inexpensive, and quickly obtained.
Useful to:
Identify the problem
Better define the problem
Develop an approach to the problem
Formulate an appropriate research design
Answer research questions and test hypotheses
Interpret primary data with more insight

6. Secondary Data - Disadvantages
Data originally collected for other purpose so:
Many not be sufficiently relevant/accurate
Secondary data may be out of date
The accuracy/ methods may be unreliable

7. Secondary Data Sources
Economic and Social Research Institute (ersi.ie)
Global Market Information Database (gmid.euromonitor.com)
Central Statistics Office (cso.is)
Enterprise Ireland (enterprise-ireland.com)

9. General Rule Start with Secondary data
Examination of available secondary data is a prerequisite to the collection of primary data
Proceed to primary data only when the Secondary data sources have been exhausted or yield marginal returns

10. Criteria for Evaluating Secondary data
Does the data set contain the information you require to answer your research question(s) and meet your objectives?
Do the measures used match those you require?
Does the data set cover the population that is the subject of your research?
Does the data set cover the geographical area that is the subject of your research?

11. Criteria for Evaluating Secondary data
Can data about the population that is the subject of your research be separated from unwanted data?
Are the data for the right time period or sufficiently up to date?
Are data available for all the variables you require to answer your research question(s) and meet your objectives?

12. Primary Data Sources Observation Interviews Experiment Focus Groups
Surveys

13. Observation
Observational techniques are methods by which an individual or individuals gather firsthand data on programs, processes, or behaviours being studied.
Observation allows the researcher to learn about issues the participants may be unaware of or that they are unwilling or unable to discuss candidly in an interview or focus group

14. Observation Structured Participant
Recording prespecified behavioural patterns of people, objects and events in a systematic manner.
Quantitative in nature
Different types
Personal observation (e.g., mystery shopper, behavioral observation)
Electronic observation (e.g., scanner data, people meter, eye tracking)
Participant
Researcher becomes member of group/organisation
Qualitative in nature
Useful in business normally in conjunction with other methods

15. Use observation when...
The behaviour or object you are studying occurs in a public or accessible location.
Observation will produce more accurate findings than simply asking people.
Observation is often done in conjunction with a normal survey by the interviewer noting attributes

16. Observation - Advantages
Provide direct information about behaviour of individuals and groups
Permit researcher to enter into and understand situation/context
Provide good opportunities for identifying unanticipated outcomes
Exist in natural, unstructured, and flexible setting

17. Observation - Disadvantages
Expensive and time consuming
Need well-qualified, highly trained observers; may need to be content experts
May affect behaviour of participants
Selective perception of observer may distort data
Behaviour or set of behaviours observed may be atypical

18. Interviews
Begins with the assumption that the participants’ perspectives are meaningful, knowable, and can be made explicit, and that their perspectives affect the success of the project.
Can be
structured interviews, in which a carefully worded questionnaire is administered, or
In depth interviews, in which the interviewer does not follow a rigid form.

19. Use Interviews when… interpersonal contact is important
opportunities for followup of interesting comments are desired.
the situation involves complex subject matter, detailed information, high-status respondents, or highly sensitive subject matter.

20. Interviews – Advantages
Usually yield richest data, details, new insights
Permit face-to-face contact with respondents
Provide opportunity to explore topics in depth
Allow interviewer to experience the affective as well as cognitive aspects of responses
Allow interviewer to explain or help clarify questions, increasing the likelihood of useful responses
Allow interviewer to be flexible in administering interview to particular individuals or in particular circumstances

21. Interviews - Disadvantages
Expensive and time-consuming
Need well-qualified, highly trained interviewers
Interviewee may distort information through recall error, selective perceptions, desire to please interviewer
Flexibility can result in inconsistencies across interviews
Volume of information very large; may be difficult to transcribe and reduce data

22. Focus Groups
Focus groups are a gathering of 8-12 people who share some characteristics relevant to the evaluation.
Focus groups combine elements of both interviewing and participant observation.
The hallmark of focus groups is the explicit use of the group interaction to generate data and insights that would be unlikely to emerge otherwise.

23. Use Focus Groups when…
interaction of respondents may stimulate a richer response or new and valuable thought.
subject matter is not so sensitive that respondents will temper responses or withhold information.
the topic is such that most respondents can say all that is relevant or all that they know in less than 10 minutes
an acceptable number of target respondents can be assembled in one location.

24. Focus Groups – Advantages
Useful to obtain detailed information about personal and group feelings, perceptions and opinions
Can save time and money compared to individual interviews
Can provide a broader range of information
They offer the opportunity to seek clarification
they provide useful material eg. quotes for public relations publication and presentations

25. Focus Groups –Disadvantages
there can be disagreements and irrelevant discussion which distract from the main focus
they can be hard to control and manage
they can be to tricky to analyse
It can be difficult to encourage a range of people to participate
some participants may find a focus group situation intimidating or off-putting; participants may feel under pressure to agree with the dominant view
as they are self-selecting, they may not be representative of non-users.

26. Surveys
Surveys are a very popular form of data collection, especially when gathering information from large groups, where standardization is important.
Surveys can be constructed in many ways, but they always consist of two components: questions and responses.
Responses can be
open ended, i.e., allow respondents to answer in a free flowing narrative form, or
close-ended, in which respondents are asked to select from a range of predetermined answers.

27. Use Surveys when…
information is to be collected from a large number of people or when answers are needed to a clearly defined set of questions.
you need to obtain information on a wide range of topics when indepth probing of responses is not necessary,

28. Surveys - Advantages and Disadvantages
Good for gathering descriptive data
Can cover a wide range of topics
Are relatively inexpensive to use
Can be analysed using a variety of existing software
Disadvantages:
Self-report may lead to biased reporting
Data may provide a general picture but lack depth
May not provide adequate information on context

29. Experiment
Data collection method in which one or more independent variables are manipulated in order to measure their effect on a dependent variable, while controlling for external variables in order to test a hypothesis
E.g. pharmaceutical research - administer the new drug to one group of subjects, and not to the other, while monitoring them both.
Cause and effect relationship is established by
Manipulation of independent variable
Controlling for outside factors

30. Use experiments if you want to…
You want to clearly determine cause and effect
You need to try to eliminate placebo effects

31. Experiments - Advantages
A high level of control - The control over the irrelevant variables is higher as compared to other research types or methods.
Easy Determination of Cause and Effect Relationship (clear-cut conclusions)
Due to the control set up by experimenter and the strict conditions experiments can be repeated and results can be checked again

32. Experiments - Disdvantages
Often just not possible to control outside events
Creates artificial situations
Subject to human error
The degree to which results can be generalized all over situations and real world applications is limited. 
Ethics!

33. Sampling
A sample is a subset of a population that is used to represent the entire group as a whole.

34. Why Sampling?...
When sampling is used, it is possible to generate findings that are representative of the whole population rather than collecting the data for the whole population.
Cheaper to collect some data rather than lots.
Easier to make sure that sample is random (if trying a census, non response causes bias- you can at least chase 20 non-responding sample individuals rather than 200 census).
Often just not possible to do census.

35. However...
Have to think carefully about how to choose, and methods of analysis
Not 100% certain (but ‘certainty’ is just comfort - in practice data errors mean a census is not 100% certain)

36. Sampling ... Many methods exist for picking a sample
If there is possible bias, it has to be described and accounted for in your methodology section

37. Population, Sample and Cases
Case or element
Population
Sample

38. Sampling Techniques Probability or representative sampling;
Non-probability or judgemental sampling.

39. Probability Sampling
Any method of sampling that utilizes some form of random selection.
The chance, or probability, of each case being selected from the population is known and is usually equal for all cases.
This means that it is possible to answer research questions and to achieve objectives that require you to estimate statistically the characteristics of the population from the sample.
Consequently, probability sampling is often associated with survey and experimental research strategies

40. Non-Probability Sampling
Non-random so the probability of each case being selected from the total population is not known.
It is impossible to answer research questions or to address objectives that require you to make statistical inferences about the characteristics of the population.
You may still be able to generalise from non-probability samples about the population, but not on statistical grounds.

41. What kind of Sample? Sample Do you need to generalise? No Yes
Probability Sampling No
Non-Probability Sampling

42. Deciding Factors
Do you want the sample to be representative of a larger population?
In your analysis do you want to use probability-based statistical methods in analysis?
Does a convenient sampling frame exist?
What size do you want your sample to be?
Are you able to afford the time and money to carry out the sample collection?

43. Steps in Probability Sampling
Determine the sampling frame
Determine the sampling design
Determine the appropriate sample size
Execute the sampling process

44. Sampling Frame
The sampling frame for any probability sample is a complete list of all the cases in the population from which your sample will be drawn.
Examples of existing sampling frames:
Voting register
Membership List
Companies Database

45. Sampling Frame Created/Distilled from lists
New company start-ups (County Enterprise Boards)
Potential Problems with Sampling Frames:
Out of date/incomplete (voting register)
Unit may be inappropriate (household not individuals)
May contain ineligible elements (teetotallers)

46. Sampling Frame
Within probability sampling, by defining the sampling frame you are defining the population about which you want to generalise.
This means that if your sampling frame is a list of all customers of an organisation, strictly speaking you can only generalise, that is apply the findings based upon your sample, to that population.

47. Sample Size
Usually assumed that ‘more is better’ (large samples reflect more accurately the characteristics of the population than small samples)
The standard error of the mean decreases as the samples size n increases but this may depend on the variability of the population sampled
Even a large sample using a poor sampling design could give you less information than a smaller, more carefully designed sample

49. Sample Size - minimum
The larger the absolute size of a sample, the more closely its distribution will be to the normal distribution and thus the more robust it will be.
A sample size of 30 or more will usually result in a sampling distribution for the mean that is very close to a normal distribution.
For this reason, a minimum number of 30 for statistical analyses provides a useful rule of thumb.
Where the population in the category is less than 30 you should normally collect data from all cases in that category.

50. Probability Sampling Techniques
Simple Random
Systematic
Stratified Random
Cluster
Multi-stage

51. Simple Random Sampling
Based on the use of random numbers
Each unit of the population is given the same probability of independent selection
Ideally requires a sample size of over a few hundred
Only suitable for geographically dispersed cases if you do not require face-to-face contact when collecting your data.

52. Simple Random Sampling
In each case:
The chances of obtaining an unrepresentative sample are small
This chance decreases as the sample size increases
This chance can be calculated (sampling error)

53. Systematic Sampling
Sampling fraction = Actual sample size/total population

54. Systematic Sampling
Systematic selection of cases from sampling frame (every nth name on database/list)
Regular intervals with random starting point
It is probability-based because you can determine the probability of each unit being in the sample
BUT can be subject to selection bias - need to ensure that the lists do not contain periodic patterns.
Works equally well for a small or large number of cases.

55. Stratified Random Sampling
Divide the population into two or more relevant and significant strata based on one or a number of attributes.
A random sample (simple or systematic) is then drawn from each of the strata.
E.g. , you may wish to stratify a sample of an organisation’s employees by both department and salary grade.

56. Cluster Sampling
Divide the population into discrete groups (clusters) prior to sampling ( e.g., you could group your data by type of manufacturing firm or geographical area.
Your sampling frame is now the complete list of clusters rather than of individual cases within the population.

57. Cluster Sampling
You then select a few clusters, normally using simple random sampling. Data are then collected from every case within the selected clusters (this is also an example of multi-stage sampling)

59. Non-Probability Sampling Techniques
Quota Sampling
Judgemental/Purposeful Sampling
Snowball Sampling

60. Sample Size
For all non-probability sampling techniques, other than for quota samples (later) there are no rules.
Your sample size is dependent on your research question(s) and objectives

61. Quota Sampling
Common in market data research, especially interview surveys
Quota calculated often based on census data
Interviewer selects people to reach quota
E.g. Commonly opinion polls of c. 1,000 of the voting population are targeted

62. Purposive / Judgement Extreme case/deviant sampling
The most subjective method based on judgement derived from previous experience
Extreme case/deviant sampling
Studying the outliers that diverge from the norm as regards a particular phenomenon, issue, or trend. 
By studying the deviant cases, researchers can often gain a better understanding of the more regular patterns of behaviour.
E.g., research into the relationship between study habits and high academic achievement, should purposively sample students considered high achievers.

63. Purposive / Judgement Heterogeneous/maximum variation sampling
Selected to provide a diverse range of cases relevant to a particular phenomenon or event.
The purpose of this kind of sample design is to provide as much insight as possible into the event or phenomenon under examination.
E.g. , when conducting a street poll about an issue, a researcher would want to ensure that he or she speaks with as many different kinds of people as possible in order to construct a robust view of the issue from the public's perspective.

64. Purposive / Judgement Homogeneous sampling
Selected for having a shared characteristic or set of characteristics.
E.g., a team of researchers wanted to understand of what significance having red hair is to red-heads, so they asked red- heads about this. This is a homogenous sample created on the basis of hair colour.

65. Purposive / Judgement Critical case sampling
Just one case is chosen for study because the researcher expects that studying it will reveal insights that can be applied to other like cases.
E.g. If you wanted to study sexuality and gender identity develop among secondary school students, you could select what is considered to be an average secondary school in terms of population and family income, so that your findings from this case could be more generally applicable.

66. Purposive / Judgement Typical case sampling
Useful when a researcher wants to study a phenomenon or trend as it relates to what are considered "typical" or "average" members of the effected population.
E.g., if you want to study how a type of educational curriculum affects the average student, then you could choose to focus on average members of a student population.

67. Snowball Sampling
Used when the members of a population are difficult to locate.
The researcher collects data on the few members of the target population he or she can locate, then asks those individuals to provide information needed to locate other members of that population whom they know.

68. Self-Selection Sampling
Wanted!!!
Research participants
5 Euro Voucher reward

69. Convenience Sampling