Presentation on theme: "Course Content Introduction to the Research Process"— Presentation transcript:
1 Course Content Introduction to the Research Process Identification of the Research ProblemDevelopment of the Research Question or HypothesisFormulation of the Research MethodsAnalysis and Interpretation of the Collected DataWriting the Research Report
2 The Scientific Method Develop the problem Develop a theoretical solution to the problemFormulate the hypothesis or questionFormulate the research plan (methods)Collect and analyze the dataInterpret the results and form conclusionsRefine the theoryDeveloping the problemWhy do you want to observe?Formulating the hypothesis or questionWhom & what do you want to observe?Hypothesis – expected resultBased on theoretical construct, results of previous studies, or past experiences or observationsHypothesis should be testable, not a value judgment or unobservable abstract phenomenonGathering the dataHow and when do you want to observe?Decide on proper methods to acquire data (very difficult)Reliability and validity of measuring instrumentsEmployment of controlsObjectivity and precision of data gathering processMaximize internal and external validityExternal validity – generalizability of resultsInternal validity – extent to which results can be attributed to treatments used (EVs)Gather data (easy part)Analyzing and interpreting the resultsMost formidable stepTypically use inductive reasoning (synthesize data from his/her study with results of other studies) to contribute to development or substantiation of a theoryReview study from pp in Baumgartner & Strong
3 Formulation of the Research Methods Selecting the Appropriate DesignSelecting the SubjectsSelecting Measurement Methods & TechniquesSelecting Instrumentation
4 Formulation of the Research Methods Developing Procedures & ProtocolUsing a Pilot StudySelecting the Appropriate Analysis TechniquesDeveloping a Timeline & BudgetCollecting the DataWe will talk about each of these in more detail as we go through the semester. The purpose of this lecture is to simply provide an overview of the steps that need to be taken. Although, we will discuss selection of subjects in much detail tonight. The other three topics will be discussed in more detail later.
6 DefinitionsPopulation – group of things (people) having one or more common characteristicsSample – representative subgroup of the larger populationUsed to estimate something about a population (generalize)Must be similar to population on characteristic being investigatedSample should be large & randomly drawnRandom samplingensures representativeness for generalizabilityshows researcher was unbiased as to selectionequalizes characteristics among groups at the beginningLarge sampleensures reliabilityallows attrition
8 Non-Probability Sampling Sampling MethodsProbability SamplingSimple random samplingStratified random samplingSystematic samplingCluster (area) samplingMultistage samplingNon-Probability SamplingDeliberate (quota) samplingConvenience samplingPurposive sampling
9 Simple Random Sampling Equal probabilityTechniquesFishbowl (with replacement & w/o replacement)Table of random numbersAdvantageMost representative groupDisadvantageDifficult to identify every member of a populationSometimes difficult or impossible to identify every member of the population.Inferences biased to some degree away from the characteristics of those missing portions of the sample.
10 Stratified Random Sampling TechniqueDivide population into various strataRandomly sample within each strataSample from each strata should be proportionalAdvantageBetter in achieving representativeness on control variableDisadvantageDifficult to pick appropriate strataDifficult to ID every member in populationDivide population into various strata (subgroups) based on some characteristic that is important to studySelect a set # from each strataNumber of subjects from each strata should represent the proportion of that subgroup in the total population
11 Systematic Sampling Technique Advantage Disadvantage Use “system” to select sample (e.g., every 5th item in alphabetized list, every 10th name in phone book)AdvantageQuick, efficient, saves time and energyDisadvantageNot entirely bias free; each item does not have equal chance to be selectedSystem for selecting subjects may introduce systematic errorCannot generalize beyond pop actually sampledA sample obtained by determining the sample interval, selecting a random starting point between 1 and k, and then selecting every kth elementNon-randomShopping Mall example – location in mall may still bias sample1936 electionEX of error:Use of phone book – those who don’t have phonesIntramural basketballPass by store in mall – all stores in this area are stores for women’s clothing
12 Cluster (Area) Sampling Randomly select groups (cluster) – all members of groups are subjectsAppropriate whenyou can’t obtain a list of the members of the populationhave little knowledge of pop characteristicsPop is scattered over large geographic areaEXAMPLEYou want 1000 teachers out of population of 4600 teachers in 46 schools in a city (100 teachers per school). Problem is that a true random sample would have you all over the place. Randomly choose 12 schools (clusters) and sample all 100 teachers in each school selected.Randomly select football teams from DII.Bias in the selection of clusters will tend to be magnified in its effect on the outcome of the study.
13 Cluster (Area) Sampling AdvantageMore practical, less costlyConclusions should be stated in terms of cluster (sample unit – school)Sample size is # of clustersEXAMPLEYou want 1000 teachers out of population of 4600 teachers in 46 schools in a city (100 teachers per school). Problem is that a true random sample would have you all over the place. Randomly choose 12 schools (clusters) and sample all 100 teachers in each school selected.Randomly select football teams from DII.Bias in the selection of clusters will tend to be magnified in its effect on the outcome of the study.
14 Multistage Sampling Stage 1 Stage 2 randomly sample clusters (schools) randomly sample individuals from the schools selected
15 Non-Probability Sampling Sampling MethodsProbability SamplingSimple random samplingStratified random samplingSystematic samplingCluster (area) samplingMultistage samplingNon-Probability SamplingDeliberate (quota) samplingConvenience samplingPurposive sampling
16 Deliberate (Quota) Sampling Similar to stratified random samplingTechniqueQuotas set using some characteristic of the population thought to be relevantSubjects selected non-randomly to meet quotas (usu. convenience sampling)Disadvantageselection biasCannot set quotas for all characteristics important to studySample is biased; researcher is free to select subjects as he or she sees fit.Difficult to set quotas for all of the demographic and other characteristics that may be relevant to the outcome of the study (views toward abortion may be affected by gender, parental status, religion, economic status, education, political affiliation, race, family history, and many other factorsonly use when research advantages are superior to statistical and public relations aspects of bias-free selection
17 Convenience Sampling “Take them where you find them” - nonrandom Intact classes, volunteers, survey respondents (low return), a typical group, a typical personDisadvantage: Selection biasUse post hoc analysis to show groups were equal at the startMust describe in detail the characteristics of the people participatingShow groups were equal at the start (post hoc analysis) – better than nothing, but not as good as random sampling; still don’t know if groups differ on some unmeasured characteristic
18 Purposive Sampling Purposive sampling (criterion-based sampling) Establish criteria necessary for being included in study and find sample to meet criteriaSolution: ScreeningUse random sampling to obtain a representative sample of larger population and then those subjects that are not members of the desired population are screened or filtered outEX: want to study smokers but can’t identify all smokers
19 Sample Size Critical factor is whether sample is representative Necessary sample size depends on population sizeRecommendations:Use tables from books30 per groupDescriptive studies – 10-20% of populationNo more than 50% of populationStatistical powerAttritionWhen selecting sample size, considerselection process (random or not) – most important for generalizabilitytypes of variables studiedhow data are collectedstatistical procedurespublic relationsExperimental studies use less subjects than descriptive because attrition usually less; nonresponse in questionnaire studies important
20 Other Sampling Considerations Random assignmentSampling of treatments (experimental research)Use post hoc analysis to show groups were equal at the startSince random sampling is often impossible, sample must be selected on some theoretical basisBe careful with generalizations
21 When Selecting Subjects … Are subjects with special characteristics necessary for your research? (age, gender, trained/untrained, expert/novice, size, etc.)Can you obtain the necessary permission and cooperation from the subjects?Can you find enough subjects?Interaction among selection of subjects, treatments, and measures is essential for experimental studies.Examples of interactions to be considered:Subjects with high level of fitness will not respond to moderate training program.Subjects with high physical fitness will have small range of scores (homogeneous) on VO2 max, and will have poor correlation between VO2max and endurance performance.Does not mean that there is no relationship, but rather you have restricted range of performance so much that correlation can not be exhibited.We will now discuss various techniques used to select samples of subjects.
22 Reporting Subjects State how many subjects were selected Describe how the subjects were selectedDiscuss whether any subjects were lost during the study and whyExplain why the subjects were selectedDescribe subject characteristics that are pertinent to study – be very specificIdentify procedures taken to protect the subjectsIdentify procedures taken to protect the subjectsThis may vary from study to studyIt is usually sufficient to state that informed consent was obtained, but where there were substantial risks to the subjects, the precautionary steps should be explained