Chapter Nine Primary Data Collection: Experimentation and Test Marketing Chapter Nine

Chapter Nine Objectives Understand the nature of experiments Gain insight into the requirements for proving causation Learn about the experimental setting Examine experimental validity Compare types of experimental designs Gain insight into test marketing Learn the limitations of experimentation in marketing research Chapter Nine

What Is an Experiment? Independent Variable: Dependent Variable: Price, Packaging, Distribution, Product, Advertising Dependent Variable: Sales, Customer Satisfaction Treatment: Independent variable manipulated to measure its effect on the dependent variable Extraneous Factors: Things you do not control, such as the weather Chapter Nine

Demonstrating Causation Determine whether a change in one variable likely caused an observed change in another Causal relationships must show three things: Concomitant variation Appropriate time order of occurrence Elimination of other possible causal factors Chapter Nine

Demonstrating Causation 1 . Concomitant Variation: A statistical relationship between variables 2. Appropriate Time Order of Occurrence: Change in an independent variable occurred before an observed change in the dependent variable 3. Elimination of Other Possible Causal Factors: No other independent variables are causing the change Chapter Nine

Experimental Setting Laboratory: Field: Experiments conducted in a controlled setting Field: Tests conducted outside the laboratory in an actual environment, such as a marketplace Chapter Nine

Experimental Validity Internal Validity: The extent to which competing explanations for the experimental results observed can be ruled out. External Validity: The extent to which causal relationships measured in an experiment can be generalized to outside persons, settings, and times. Chapter Nine

Experimental Notation “X” = Independent Variable: Exposure of an individual or group to experimental treatment Variable is something the researcher can change Goal is to test if the change in the independent variable will cause a change in the dependent variable “O” = Dependent Variable: A variable the researcher cannot change directly Test to see if changing the independent variable will result in changes to the dependent variable. The dependent variable is “dependent” upon what the researcher does with the independent variable Chapter Nine

Extraneous Variables History: Maturation: Instrument Variation: Intervention, between beginning and end of experiment, of outside variables that might change the dependent variable Maturation: Changes in subjects occurring during the experiment not related to the experiment but which might affect subjects’ response to the treatment factor Instrument Variation: Changes in measurement instruments (e.g., interviews or observers) that might affect measurements Chapter Nine

Extraneous Variables Selection Bias: Mortality: Testing Effect: Systematic differences between the test group and the control group due to a biased selection process Mortality: Loss of test units / subjects during the course of an experiment Testing Effect: An effect that is a by-product of the research process Regression to the Mean: Tendency of subjects with extreme behaviour to move toward the average for that behaviour during the course of the experiment Chapter Nine

Controlling Extraneous Variables Randomization: The random assignment of subjects to treatment conditions to ensure equal representation of subject characteristics Holding constant the value or level of extraneous variables throughout the course of an experiment Use of experimental design to control extraneous causal factors Adjusting for effects of extraneous variables by adjusting the value or the dependant variable for each treatment condition Physical Control: Design Control: Statistical Control: Chapter Nine

Experimental Treatment and Effects Experimental Design: A test in which the researcher has control over and manipulates one or more independent variables Treatment Variable: The independent variable that is manipulated in an experiment Experimental Effect: The effect of the treatment variable on the dependent variable Chapter Nine

Experimental Terms Test Group Control Group Experimental Effect Group is exposed to manipulation (change) of independent variable Control Group Group in which the independent variable is not changed Group is used for comparison Experimental Effect Effect of the treatment variable on the dependent variable Chapter Nine

Selected Experimental Designs O = The Measurement of the Dependent Variable X = The Manipulation / Change of Independent Variable E = Experimental Effect: Change in Dependent Variable due to Change in the Independent Variable One-Shot Case Study Design X O1 Change the independent variable, then measure the change in the dependent variable to see if there was, in fact, a change in the dependent variable that the researcher might conclude resulted from the change in the independent variable Chapter Nine

Selected Experimental Designs O = The Measurement of the Dependent Variable X = The Manipulation / Change of Independent Variable E = Experimental Effect: Change in Dependent Variable due to Change in the Independent Variable One-Group Pre-Test–Post-test Design O1 X O2 Same as one-shot design except measure the dependent variable before the change in the independent variables. The researcher is establishing a benchmark from which to gauge the change. Chapter Nine

Selected Experimental Designs O = The Measurement of the Dependent Variable X = The Manipulation / Change of Independent Variable E = Experimental Effect: Change in Dependent Variable due to Change in the Independent Variable Before and After with Control Group Design Experimental Group: O1 X O2 Control Group: O3 O4 A true experimental design that involves random assignment of subjects or test units to experimental and control groups and pre- and post-measurements of both groups. Chapter Nine

Selected Experimental Designs O = The Measurement of the Dependent Variable X = The Manipulation / Change of Independent Variable E = Experimental Effect: Change in Dependent Variable due to Change in the Independent Variable After-Only with Control Group Design Experimental Group: X O1 Control Group: O2 Subjects in the experiment are randomly assigned to experiment and control groups. No pre-measurements of the dependent variable are taken. Chapter Nine

Quasi-Experiments Interrupted Time-Series: Multiple Time-Series: Research in which repeated measurement of an effect “interrupts” previous data patterns Multiple Time-Series: Interrupted time-series design with a control group Chapter Nine

Test Markets: Types Standard Scanner Controlled Simulated (STM) Chapter Nine

Test Markets: Costs Direct Costs: Indirect Costs: Advertising expenses Syndicated research Coupons, sampling, POP materials Trade allowances Indirect Costs: Diversion of activity from existing products Possible negative impact of test failure Possible negative trade reactions Competition becomes aware of new product idea Chapter Nine

Six Steps in a Test Market Study 1. Define the Objective: What do you hope to learn? What are the characteristics of the people / products of interest? 2. Select a Basic Approach: Simulated, controlled, scanner, or standard test? 3. Develop Detailed Test Procedures: Generate a full marketing plan Determine positioning approach Chapter Nine

Six Steps in a Test Market Study 4. Select the Test Market: Market should not be over tested Should have little media spillover Demographics should be similar to your target population Market should be large enough to provide useful results Distribution and other patterns should be similar to the nation 5. Execute the Plan: Determine length of time of test Finalize the parties involved 6. Analyze the Results: Use qualitative and quantitative techniques when possible Chapter Nine

Limitations of Experimental Research High Cost: Is the research affordable? Will the research be beneficial and help solve problems? Has a cost and benefit analysis been done? Security Issues: Particularly critical with field experiments The competition might be “tipped-off” Are the data and findings secure? Implementation Problems: People who unwittingly get caught in the experiment Outside factors unnaturally affecting the experiment Participants who intentionally try to skew the results Chapter Nine