Course Content Introduction to the Research Process Identification of the Research Problem Development of the Research Question or Hypothesis Formulation of the Research Methods Analysis and Interpretation of the Collected Data Writing the Research Report

The Scientific Method Develop the problem Develop a theoretical solution to the problem Formulate the hypothesis or question Formulate the research plan (methods) Collect and analyze the data Interpret the results and form conclusions Refine the theory Developing the problem Why do you want to observe? Formulating the hypothesis or question Whom & what do you want to observe? Hypothesis – expected result Based on theoretical construct, results of previous studies, or past experiences or observations Hypothesis should be testable, not a value judgment or unobservable abstract phenomenon Gathering the data How and when do you want to observe? Decide on proper methods to acquire data (very difficult) Reliability and validity of measuring instruments Employment of controls Objectivity and precision of data gathering process Maximize internal and external validity External validity – generalizability of results Internal validity – extent to which results can be attributed to treatments used (EVs) Gather data (easy part) Analyzing and interpreting the results Most formidable step Typically use inductive reasoning (synthesize data from his/her study with results of other studies) to contribute to development or substantiation of a theory Review study from pp. 9-10 in Baumgartner & Strong

Formulation of the Research Methods Selecting the Appropriate Design Selecting the Subjects Selecting Measurement Methods & Techniques Selecting Instrumentation

Formulation of the Research Methods Developing Procedures & Protocol Using a Pilot Study Selecting the Appropriate Analysis Techniques Developing a Timeline & Budget Collecting the Data We will talk about each of these in more detail as we go through the semester. The purpose of this lecture is to simply provide an overview of the steps that need to be taken. Although, we will discuss selection of subjects in much detail tonight. The other three topics will be discussed in more detail later.

Sampling Procedures

Definitions Population – group of things (people) having one or more common characteristics Sample – representative subgroup of the larger population Used to estimate something about a population (generalize) Must be similar to population on characteristic being investigated Sample should be large & randomly drawn Random sampling ensures representativeness for generalizability shows researcher was unbiased as to selection equalizes characteristics among groups at the beginning Large sample ensures reliability allows attrition

Representative

Non-Probability Sampling Sampling Methods Probability Sampling Simple random sampling Stratified random sampling Systematic sampling Cluster (area) sampling Multistage sampling Non-Probability Sampling Deliberate (quota) sampling Convenience sampling Purposive sampling

Simple Random Sampling Equal probability Techniques Fishbowl (with replacement & w/o replacement) Table of random numbers Advantage Most representative group Disadvantage Difficult to identify every member of a population Sometimes difficult or impossible to identify every member of the population. Inferences biased to some degree away from the characteristics of those missing portions of the sample.

Stratified Random Sampling Technique Divide population into various strata Randomly sample within each strata Sample from each strata should be proportional Advantage Better in achieving representativeness on control variable Disadvantage Difficult to pick appropriate strata Difficult to ID every member in population Divide population into various strata (subgroups) based on some characteristic that is important to study Select a set # from each strata Number of subjects from each strata should represent the proportion of that subgroup in the total population

Systematic Sampling Technique Advantage Disadvantage Use “system” to select sample (e.g., every 5th item in alphabetized list, every 10th name in phone book) Advantage Quick, efficient, saves time and energy Disadvantage Not entirely bias free; each item does not have equal chance to be selected System for selecting subjects may introduce systematic error Cannot generalize beyond pop actually sampled A sample obtained by determining the sample interval, selecting a random starting point between 1 and k, and then selecting every kth element Non-random Shopping Mall example – location in mall may still bias sample 1936 election EX of error: Use of phone book – those who don’t have phones Intramural basketball Pass by store in mall – all stores in this area are stores for women’s clothing

Cluster (Area) Sampling Randomly select groups (cluster) – all members of groups are subjects Appropriate when you can’t obtain a list of the members of the population have little knowledge of pop characteristics Pop is scattered over large geographic area EXAMPLE You want 1000 teachers out of population of 4600 teachers in 46 schools in a city (100 teachers per school). Problem is that a true random sample would have you all over the place. Randomly choose 12 schools (clusters) and sample all 100 teachers in each school selected. Randomly select football teams from DII. Bias in the selection of clusters will tend to be magnified in its effect on the outcome of the study.

Cluster (Area) Sampling Advantage More practical, less costly Conclusions should be stated in terms of cluster (sample unit – school) Sample size is # of clusters EXAMPLE You want 1000 teachers out of population of 4600 teachers in 46 schools in a city (100 teachers per school). Problem is that a true random sample would have you all over the place. Randomly choose 12 schools (clusters) and sample all 100 teachers in each school selected. Randomly select football teams from DII. Bias in the selection of clusters will tend to be magnified in its effect on the outcome of the study.

Multistage Sampling Stage 1 Stage 2 randomly sample clusters (schools) randomly sample individuals from the schools selected

Non-Probability Sampling Sampling Methods Probability Sampling Simple random sampling Stratified random sampling Systematic sampling Cluster (area) sampling Multistage sampling Non-Probability Sampling Deliberate (quota) sampling Convenience sampling Purposive sampling

Deliberate (Quota) Sampling Similar to stratified random sampling Technique Quotas set using some characteristic of the population thought to be relevant Subjects selected non-randomly to meet quotas (usu. convenience sampling) Disadvantage selection bias Cannot set quotas for all characteristics important to study Sample is biased; researcher is free to select subjects as he or she sees fit. Difficult to set quotas for all of the demographic and other characteristics that may be relevant to the outcome of the study (views toward abortion may be affected by gender, parental status, religion, economic status, education, political affiliation, race, family history, and many other factors only use when research advantages are superior to statistical and public relations aspects of bias-free selection

Convenience Sampling “Take them where you find them” - nonrandom Intact classes, volunteers, survey respondents (low return), a typical group, a typical person Disadvantage: Selection bias Use post hoc analysis to show groups were equal at the start Must describe in detail the characteristics of the people participating Show groups were equal at the start (post hoc analysis) – better than nothing, but not as good as random sampling; still don’t know if groups differ on some unmeasured characteristic

Purposive Sampling Purposive sampling (criterion-based sampling) Establish criteria necessary for being included in study and find sample to meet criteria Solution: Screening Use random sampling to obtain a representative sample of larger population and then those subjects that are not members of the desired population are screened or filtered out EX: want to study smokers but can’t identify all smokers

Sample Size Critical factor is whether sample is representative Necessary sample size depends on population size Recommendations: Use tables from books 30 per group Descriptive studies – 10-20% of population No more than 50% of population Statistical power Attrition When selecting sample size, consider selection process (random or not) – most important for generalizability types of variables studied how data are collected statistical procedures public relations Experimental studies use less subjects than descriptive because attrition usually less; nonresponse in questionnaire studies important

Other Sampling Considerations Random assignment Sampling of treatments (experimental research) Use post hoc analysis to show groups were equal at the start Since random sampling is often impossible, sample must be selected on some theoretical basis Be careful with generalizations

When Selecting Subjects … Are subjects with special characteristics necessary for your research? (age, gender, trained/untrained, expert/novice, size, etc.) Can you obtain the necessary permission and cooperation from the subjects? Can you find enough subjects? Interaction among selection of subjects, treatments, and measures is essential for experimental studies. Examples of interactions to be considered: Subjects with high level of fitness will not respond to moderate training program. Subjects with high physical fitness will have small range of scores (homogeneous) on VO2 max, and will have poor correlation between VO2max and endurance performance. Does not mean that there is no relationship, but rather you have restricted range of performance so much that correlation can not be exhibited. We will now discuss various techniques used to select samples of subjects.

Reporting Subjects State how many subjects were selected Describe how the subjects were selected Discuss whether any subjects were lost during the study and why Explain why the subjects were selected Describe subject characteristics that are pertinent to study – be very specific Identify procedures taken to protect the subjects Identify procedures taken to protect the subjects This may vary from study to study It is usually sufficient to state that informed consent was obtained, but where there were substantial risks to the subjects, the precautionary steps should be explained