Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall Conducting Research and Utilizing Tests to Measure Performance Chapter 10

Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall Please Note:

Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall Opening Vignette Omaha Steaks International

Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall Research Serves Direct Marketers Fact-Finding Information Gathering Problem-Solving Decision-Making

Need for Marketing Research Managers must have good, accurate, timely information with which to make decisions. Marketing research helps to gather the needed information. The results of research can be quantitative and/or qualitative. Valid research measures results … not opinions. Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

Problem Structure How much advertising is needed? How will the direct marketing mix be selected? How will our resources be utilized? Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

The Nature of Research Surveys vs. Experiments: A survey looks at things the way they are to measure product preferences or determine future buying intentions An experiment or test measures the effect of change Which one is used more widely by direct marketers? Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

Databased Research and Analysis “My mind’s made up. Don’t confuse me with facts.” A characteristic of database-driven and directed marketing is measurement and accountability for actions. Decisions are based on facts, not opinions. Direct marketers build databases from facts, relying not so much on responses derived from survey, but more on conclusions derived from experimentation. Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

Products and Services Media Offers/Propositions Copy Platforms Creative Formats Timing/Seasonality What Do Direct Marketers Test? Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

TEST THE BIG THINGS Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

Sourcing & Collecting Information  Secondary Data  Primary Data Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

Secondary Data Sources The Organization’s Internal Records Government Sources: Federal, State & Local U.S. Department of Commerce - Bureau of the Census U.D. Department of Labor - Bureau of Labor Statistics U.S. Department of Agriculture Other U.S. Government Sources: President’s Office, Congress, Treasury Department, Interior Department, and Health and Human Services Department State and Local Governments: economic surveys, license registrations, and tax records Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

Secondary Data Sources Trade, Technical, Professional and Business Associations Private Research Organizations Foundations, Universities and Other Nonprofits Libraries, Public and Private Advertising Media Financial Institutions and Utilities Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

 Behavior  Intentions  Knowledge  Socioeconomic Status  Attitudes and Opinions  Motivations  Psychological Traits Primary data, collected via survey, can yield information about: Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

PRIMARY DATA COLLECTION METHODS Survey MethodAdvantagesDisadvantages Personal Interview More complete and accurate sample, more complete information, greater flexibility in structuring questions, high response rate Cost, possibility of interview bias, and need for extensive interviewer supervision and control Telephone Interview Economy, speed, representative sampling, minimal nonresponse, simple callbacks, ability to make interview coincide with other activities Limited availability of information at time of contact, excludes those without phone service, increasing public resistance to phone calls, inability to use questions requiring demonstration or visualization. Survey Ability to reach hard-to-reach consumer segments, speed, economy, respondent choice in when to reply Representative sample issues, higher degrees of nonresponse bias Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

PRIMARY DATA COLLECTION METHODS Survey MethodAdvantagesDisadvantages Mail Questionnaire Great versatility, low cost, no interviewer bias, no field staff needed, respondent choice in when to reply, respondent confidentiality High rate of nonresponse, need for follow-up, response bias factors On-line Surveys Ability to reach potential respondents at most opportune time, speed and convenience Similar to surveys - Representative sample issues, higher degrees of nonresponse bias Observation Removes respondent biasMore costly and time consuming than some of the other methods Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

How to Design An Experiment  Control  Randomization  Statistically-valid sample size Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

How to Track Responses Key Codes Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

Response rate & break-even analysis Control vs. experimental packages Direct marketers test or experiment with different offers and campaign themes to determine which one generates the greatest response rate Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

Promotion Cost = Break-even Number of Sales Unit Profit per Sale Break-even analysis Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

Samples & Estimations Random sample designs Simple Random Samples Systematic Random Samples Stratified Random Samples Cluster Samples Replicated Samples Sequential Samples Determination of sample size Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

Using Statistical Evaluation to Determine the Sample Size Confidence level - the number of standard deviations from the mean in a normal distribution (for example a 95% confidence levels corresponds to 1.96 standard deviations from the mean) Limit of error - the number of percentage points by which the researcher can miscalculate the actual response rate Expected (actual) response rate - the number of positive responses expressed as a percentage of the total Sample size - the number of observations in an experiment Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

Measurement of Differences Hypothesis testing Types of errors in hypothesis testing Statistical evaluation of differences Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

Hypothesis Testing Hypotheses are typically stated in negative terms; that is, a null hypothesis (H 0 ) versus an alternative hypothesis (H a ) in a form such as the following: H 0 : Direct mail response from the test promotion is at or below direct mail response from the control promotion. H a : Direct mail response from the test promotion is above direct mad response from the control promotion. The null hypothesis, then, states that direct mail response will not be better than the control. Our measurement sets out to disprove this null hypothesis. Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

Types of Error in Hypothesis Testing Type One: Results when the decision-maker rejects the null hypothesis even though it is, in fact, true... i.e., taking an action when one shouldn't Type Two: Results when the decision-maker accepts the null hypothesis when, in fact, not true... i.e., not taking an action when one should. Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

Assume that a sample has been properly selected and is of an adequate size. Assume further that an experiment has been designed and implemented in a valid manner. It now remains for the direct marketer to be able to recognize the difference in the response rate from a test and that from a control, with some degree of confidence and within an acceptable limit of error. Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

TestControlTotals Response A C A + C Non-response B D B + D Total mailed A + B C + D A + B + C + D = N The statistic  2 is computed as follows:  2 = N [  (A x D) - (C x B)  - N/2] 2 (A+B) x (C+D) x (A+C) x (B+D) Here is a sample calculation: TestControlTotals Response Non-response Total mailed  2 = 2,000 x [  180, ,000  - 1,000] 2 1,000 x 1,000 x 300 x 1,700  2 = which is significant at the 99++% level since it exceeds the critical value in the X 2 table for one degree of freedom for a significance level of 0.001, given as Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

STRUCTURING and EVALUATING AN EXPERIMENT State the hypothesis Develop, by a priori analysis, the assumptions required and compute the appropriate sample size Structure and perform the experiment Develop, by a posteriori analysis, statistics for judging hypothesis validity Make the decision Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

CHAPTER CASE THE AMERICAN HEART ASSOCIATION Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall