14 Statistical Testing of Differences and Relationships

Evaluating Differences and Changes “Our overall customer satisfaction score increased from 92 percent 3 months ago to 93.5 percent today.”  Did customer satisfaction really increase? Should we celebrate? “In a recent product concept test, 19.8 percent of those surveyed said they were very likely to buy the new product they evaluated.”  Is this good? Is it better than the results we got last year for a similar product? What do these results suggest in terms of whether to introduce the new product?

These are the common questions in marketing and marketing research. Although considered boring by some, statistical hypothesis testing is important because it helps researchers get closer* to answers to these questions. (Remark: “closer” because certainty is never achieved in answering these questions in marketing research.)

Statistical Significance Statistical significance: A difference that is large enough that is not likely to have occurred because of chance or sampling error.

Things to keep in mind on significance tests Random samples are assumed. Big data does not mean “good” data. Don’t overrely on significance testing.

Hypothesis Testing Hypothesis: Assumption or theory that a researcher or manager makes about some characteristic of the population under study. The marketing researcher is often faced with the question of whether research results are different enough from the norm that some element of the firm’s marketing strategy should be changed.

Considering the following situations: The results of a tracking survey show that awareness of a product is lower than it was in a similar survey conducted six months ago. Are these results significantly lower? Are the results sufficiently lower to call for a change in advertising strategy?

Considering the following situations: A product manager believes that the average purchaser of his product is 35 years of age. A survey is conducted to test this hypothesis and the survey shows that the average purchaser of the product is 38.5 years of age. Is the survey result different enough from the product manager’s belief to cause him to conclude that his belief is incorrect?

All of these questions can be evaluated with some kind of statistical test. In hypothesis testing, the researcher determines whether a hypothesis concerning some characteristic of the population is likely to be true, given the evidence. A statistical hypothesis test allows us to calculate the probability of observing a particular result if the stated hypothesis is actually true.

Steps in Hypothesis Testing Step 1: Stating the Hypothesis Hypotheses are stated using 2 basic forms: – The null hypothesis H 0 (the hypothesis of status quo) is the hypothesis that is tested against its compliment – The alternative hypothesis H a (the research hypothesis of interest)

Steps in Hypothesis Testing Step 1: Stating the Hypothesis – Ex: Suppose the manager of Burger City believes that his operational procedures will guarantee that the average customer will wait 2 minutes in the drive-in window line. He conducts research, based on the observation of 1,000 customers at randomly selected stores at randomly selected times. The average customer observed in this study spends 2.4 minutes in the drive-in window line.

Steps in Hypothesis Testing Step 1: Stating the Hypothesis – The null hypothesis and the alternative hypothesis might be stated as follows: The null hypothesis H 0 : Mean waiting time = 2 minutes The alternative hypothesis H a : Mean waiting time ≠ 2 minutes

Steps in Hypothesis Testing Step 2: Choosing the Appropriate Statistical Test The analyst must choose the appropriate statistical test, given the characteristics of situation under investigation.

Commonly Used Statistical Hypothesis Tests Independent vs. Related Samples Independent samples: Samples in which measurement of a variable in one population has no effect on measurement of the variable in the other. – Ex. Men and women were interviewed in a particular survey regarding their frequency of eating out, there is no way that man’s response could affect/change the way woman would respond to a question in the survey.

Commonly Used Statistical Hypothesis Tests Independent vs. Related Samples Related samples: Samples in which measurement of a variable in one population may influence measurement of the variable in the other. – Ex. The researcher needed to determine the effect of a new advertising campaign on consumer awareness of a particular brand. To do this, the researcher might survey a random sample of consumers before introducing the new campaign and then survey the same sample of consumers 90 days after the new campaign was introduced. These samples are not independent. The measurement of awareness 90 days after the start of the campaign may be affected by the first measurement.

Commonly Used Statistical Hypothesis Tests Degrees of Freedom Degree of freedom: Number of observations in a statistical problem that are free to vary. Many statistical tests require the researcher to specify degrees of freedom in order to find the critical value of the test statistic from the table for that statistic. The number of degree of freedom (d.f.) is equal to the number of observations minus the number of assumptions/constraints necessary to calculate a statistic.

Goodness of Fit Chi-Square Test Test of the goodness of fit between the observed distribution and the expected distribution of a variable. The test is applied when you have two categorical variables from a single population. It is used to determine whether there is a significant association between the two variables.

Hypotheses about One Mean Z Test Hypothesis test used for a single mean if the sample is large enough (n ≥ 30)and drawn at random. t Test Hypothesis test used for a single mean if the sample is too small (n < 30) to use the Z test.

Analysis of Variance (ANOVA) Analysis of variance (ANOVA): Test for the differences among the means of several independent groups (3 or more sample means).

Steps in Hypothesis Testing Step 3: Developing a Decision Rule Decision rule: rule or standard used to determine whether to reject or fail to reject the null hypothesis. The significance level ( ) is critical in the process of choosing between the null and alternative hypotheses. The level of significance---.10,.05, or.01, for example---is the probability that is considered too low to justify acceptance of the null hypothesis.

Steps in Hypothesis Testing Step 3: Developing a Decision Rule Consider a situation in which the researcher has decided that she wants to test a hypothesis at the.05 level of confidence. This means that she will reject the null hypothesis if the test indicates that the probability of occurrence of the observed result because of the chance or sampling error is less than 5 percent. Rejection of the null hypothesis is equivalent to supporting the alternative hypothesis, but statistically, we can only state that the null hypothesis is not true.

Steps in Hypothesis Testing Step 4: Calculating the Value of the Test Statistic The researcher does the following: – Uses appropriate formula to calculate the value of statistic for the test chosen. – Compares the value just calculated to the critical value of the statistic (from appropriate table), based on the decision rule chosen. – Based on the comparison, determines to either reject or fail to reject the null hypothesis H 0.

Steps in Hypothesis Testing Step 5: Stating the Conclusion The conclusion summarizes the results of the test. It should be stated from the perspective of the original research question.

Types of Errors in Hypothesis Testing Type I error ( error): Rejection of the null hypothesis when, in fact, it is true. – The researcher may reach this incorrect conclusion because the observed difference between the sample and population values is due to sampling error. – The probability of committing a type I error is referred to as the alpha ( ) level. Conversely, 1 - is the probability of marking a correct decision by not rejecting the null hypothesis when, in fact, it is true.

Types of Errors in Hypothesis Testing Type II error ( β error): Failure to reject the null hypothesis when, in fact, it is false. – A type II error is referred to as a beta (β) error. – The value 1- β reflects the probability of marking a correct decision in rejecting the null hypothesis when, in fact, it is false.

Types of Errors in Hypothesis Testing The level of is set by the researcher, after consulting with his/her client, considering the resources available for the project, and considering the implication of marking type I and type II errors. However, the estimation of β is more complicated and it is beyond the scope of our discussion. Actual State of the Null Hypothesis Fail to Reject H 0 Reject H 0 H 0 is trueCorrect (1 - )Type I error ( ) H 0 is falseType II error (β)Correct (1 - β)

One-Tailed vs. Two-Tailed Test Tests are either one-tailed or two-tailed. The decision as to which to use depends on the nature of the situation and what the researcher is trying to demonstrate. – Ex. When the quality control department of a fast- food org. receives a shipment of chicken breasts from one of its vendors and needs to determine whether the product meets specifications in regard to fat content, a one-tailed test is appropriate. The shipment will be rejected if it does not meet minimum specifications.

One-Tailed vs. Two-Tailed Test – Ex. On the other hand, the managers of the meat company that supplies the product should run two- tailed tests to determine two factors. First, they must make sure that the product meets the min. specifications of their customer before they ship it. Second, they want to determine whether the product exceeds specifications because this can be costly to them. If they are consistently providing a product that exceeds the level of quality they have contracted to provide, their costs may be unnecessarily high.