© 2006 by The McGraw-Hill Companies, Inc. All rights reserved. 1 Chapter 7 Sampling, Significance Levels, and Hypothesis Testing Three scientific traditions critical to experimental research –Sampling –Significance levels –Hypothesis testing

© 2006 by The McGraw-Hill Companies, Inc. All rights reserved. 2 Population and Sample Population – all units (people or things) possessing the attributes and characteristics of interest Sample -- subset of a population Sampling frame -- subset of units that have a chance to become part of the sample Researchers study the sample to make generalizations back to the population

© 2006 by The McGraw-Hill Companies, Inc. All rights reserved. 3 Defining the Population Choose the dimensions or characteristics meaningful to the hypothesis or research question Must be at least one common characteristic among all members of a population Must develop procedure to ensure representative sampling

© 2006 by The McGraw-Hill Companies, Inc. All rights reserved. 4 Addressing Generalizability Extent to which conclusions developed from data collected from sample can be extended to its population Sample is representative to the degree that all units had same chance for being selected Representative sampling eliminates selection bias –Characteristics of population should appear to the same degree in sample Representativeness can only be assured through random sampling

© 2006 by The McGraw-Hill Companies, Inc. All rights reserved. 5 Probability Sampling The probability of any unit being included in the sample is known and equal When probability for selection is equal, selection is random Also known as random sampling Sampling error will always occur

© 2006 by The McGraw-Hill Companies, Inc. All rights reserved. 6 Types of Probability Sampling Simple random sampling –Simplest and quickest Systematic sampling –If used on a randomly ordered frame, results in truly random sample Stratified random sampling –Random sampling within all subgroups Cluster sampling –Random sampling within known clusters

© 2006 by The McGraw-Hill Companies, Inc. All rights reserved. 7 Nonprobability Sampling Does not rely on random selection Weakens sample-to-population representativeness Used when other techniques will not result in an adequate or appropriate sample Used when researchers desire participants with special experiences or abilities

© 2006 by The McGraw-Hill Companies, Inc. All rights reserved. 8 Nonprobability Sampling Techniques Convenience sample Volunteer sample Inclusion/exclusion sample Snowball sample Network sample Purposive sample Quota sample

© 2006 by The McGraw-Hill Companies, Inc. All rights reserved. 9 Sample Size Number of people/units for whom you need to collect data Determined prior to selecting sample Less than the number you ask to participate The larger the sample relative to the population, the less error or bias

© 2006 by The McGraw-Hill Companies, Inc. All rights reserved. 10 Comparisons of Sample Size to Population Population Size Sample Size Population Size Sample Size , , ,000384

© 2006 by The McGraw-Hill Companies, Inc. All rights reserved. 11 Significance Levels The researcher sets the significance level, or p, for each statistical test The degree of error the researcher finds acceptable in a statistical test An estimate of what would happen if the study were actually repeated many times Generally.05 is accepted level

© 2006 by The McGraw-Hill Companies, Inc. All rights reserved. 12 Significance Levels.05 significance level = 5 out of 100 findings that appear to be valid will be due to chance Also known as the alpha level or p If p >.05, the finding is nonsignificant If p is .05, the finding is significant or real

© 2006 by The McGraw-Hill Companies, Inc. All rights reserved. 13 Hypothesis Testing Hypothesis states the expected relationship or difference between two or more variables Alternative hypothesis presented in report Null is statistically tested –Act of decision making based on the significance level –Decision based on comparison between p set before study to p produced by statistical test

© 2006 by The McGraw-Hill Companies, Inc. All rights reserved. 14 Hypothesis Testing Belief in the null hypothesis continues until there is sufficient evidence to the contrary If p for statistical test exceeds significance level, null is retained (p >.05) If p for statistical test is .05 then alternative hypothesis is accepted

© 2006 by The McGraw-Hill Companies, Inc. All rights reserved. 15 Error in Hypothesis Testing In reality, the null hypothesis is true In reality, the null hypothesis is false Use level of significance to reject null Type I error – Null is rejected even though it is true Decision 1 – Null is rejected when it is false Use level of significance to retain the null Decision 2 – Null is retained when it is true Type II error – Null is retained even though it is false