1. Chapter Nine Primary Data Collection: Experimentation and
Test Marketing
Chapter Nine

2. Chapter Nine Objectives
Understand the nature of experiments
Gain insight into the requirements for proving causation
Learn about the experimental setting
Examine experimental validity
Compare types of experimental designs
Gain insight into test marketing
Learn the limitations of experimentation in marketing research
Chapter Nine

3. What Is an Experiment? Independent Variable: Dependent Variable:
Price, Packaging, Distribution, Product, Advertising
Dependent Variable:
Sales, Customer Satisfaction
Treatment:
Independent variable manipulated to measure its effect on the dependent variable
Extraneous Factors:
Things you do not control, such as the weather
Chapter Nine

4. Demonstrating Causation
Determine whether a change in one variable likely caused an observed change in another
Causal relationships must show three things:
Concomitant variation
Appropriate time order of occurrence
Elimination of other possible causal factors
Chapter Nine

5. Demonstrating Causation 1 . Concomitant Variation:
A statistical relationship between variables
2. Appropriate Time Order of Occurrence:
Change in an independent variable occurred before an observed change in the dependent variable
3. Elimination of Other Possible Causal Factors:
No other independent variables are causing the change
Chapter Nine

6. Experimental Setting Laboratory: Field:
Experiments conducted in a controlled setting
Field:
Tests conducted outside the laboratory in an actual environment, such as a marketplace
Chapter Nine

7. Experimental Validity
Internal Validity:
The extent to which competing explanations for the experimental results observed can be ruled out.
External Validity:
The extent to which causal relationships measured in an experiment can be generalized to outside persons, settings, and times.
Chapter Nine

8. Experimental Notation
“X” = Independent Variable:
Exposure of an individual or group to experimental treatment
Variable is something the researcher can change
Goal is to test if the change in the independent variable will cause a change in the dependent variable
“O” = Dependent Variable:
A variable the researcher cannot change directly
Test to see if changing the independent variable will result in changes to the dependent variable.
The dependent variable is “dependent” upon what the researcher does with the independent variable
Chapter Nine

9. Extraneous Variables History: Maturation: Instrument Variation:
Intervention, between beginning and end of experiment, of outside variables that might change the dependent variable
Maturation:
Changes in subjects occurring during the experiment not related to the experiment but which might affect subjects’ response to the treatment factor
Instrument Variation:
Changes in measurement instruments (e.g., interviews or observers) that might affect measurements
Chapter Nine

10. Extraneous Variables Selection Bias: Mortality: Testing Effect:
Systematic differences between the test group and the control group due to a biased selection process
Mortality:
Loss of test units / subjects during the course of an experiment
Testing Effect:
An effect that is a by-product of the research process
Regression to the Mean:
Tendency of subjects with extreme behaviour to move toward the average for that behaviour during the course of the experiment
Chapter Nine

11. Controlling Extraneous Variables
Randomization:
The random assignment of subjects to treatment conditions to ensure equal representation of subject characteristics
Holding constant the value or level of extraneous variables throughout the course of an experiment
Use of experimental design to control extraneous causal factors
Adjusting for effects of extraneous variables by adjusting the value or the dependant variable for each treatment condition
Physical Control:
Design Control:
Statistical Control:
Chapter Nine

12. Experimental Treatment and Effects
Experimental Design:
A test in which the researcher has control over and manipulates one or more independent variables
Treatment Variable:
The independent variable that is manipulated in an experiment
Experimental Effect:
The effect of the treatment variable on the dependent variable
Chapter Nine

13. Experimental Terms Test Group Control Group Experimental Effect
Group is exposed to manipulation (change) of independent variable
Control Group
Group in which the independent variable is not changed
Group is used for comparison
Experimental Effect
Effect of the treatment variable on the dependent variable
Chapter Nine

14. Selected Experimental Designs
O = The Measurement of the Dependent Variable
X = The Manipulation / Change of Independent Variable
E = Experimental Effect: Change in Dependent Variable due to Change in the Independent Variable
One-Shot Case Study Design
X O1
Change the independent variable, then measure the change in the dependent variable to see if there was, in fact, a change in the dependent variable that the researcher might conclude resulted from the change in the independent variable
Chapter Nine

15. Selected Experimental Designs
O = The Measurement of the Dependent Variable
X = The Manipulation / Change of Independent Variable
E = Experimental Effect: Change in Dependent Variable due to Change in the Independent Variable
One-Group Pre-Test–Post-test Design
O1 X O2
Same as one-shot design except measure the dependent variable before the change in the independent variables. The researcher is establishing a benchmark from which to gauge the change.
Chapter Nine

16. Selected Experimental Designs
O = The Measurement of the Dependent Variable
X = The Manipulation / Change of Independent Variable
E = Experimental Effect: Change in Dependent Variable due to Change in the Independent Variable
Before and After with Control Group Design
Experimental Group: O1 X O2
Control Group: O3 O4
A true experimental design that involves random assignment of subjects or test units to experimental and control groups and pre- and post-measurements of both groups.
Chapter Nine

17. Selected Experimental Designs
O = The Measurement of the Dependent Variable
X = The Manipulation / Change of Independent Variable
E = Experimental Effect: Change in Dependent Variable due to Change in the Independent Variable
After-Only with Control Group Design
Experimental Group: X O1
Control Group: O2
Subjects in the experiment are randomly assigned to experiment and control groups. No pre-measurements of the dependent variable are taken.
Chapter Nine

18. Quasi-Experiments Interrupted Time-Series: Multiple Time-Series:
Research in which repeated measurement of an effect “interrupts” previous data patterns
Multiple Time-Series:
Interrupted time-series design with a control group
Chapter Nine

19. Test Markets: Types Standard Scanner Controlled Simulated (STM)
Chapter Nine

20. Test Markets: Costs Direct Costs: Indirect Costs: Advertising expenses
Syndicated research
Coupons, sampling, POP materials
Trade allowances
Indirect Costs:
Diversion of activity from existing products
Possible negative impact of test failure
Possible negative trade reactions
Competition becomes aware of new product idea
Chapter Nine

21. Six Steps in a Test Market Study
1. Define the Objective:
What do you hope to learn?
What are the characteristics of the people / products of interest?
2. Select a Basic Approach:
Simulated, controlled, scanner, or standard test?
3. Develop Detailed Test Procedures:
Generate a full marketing plan
Determine positioning approach
Chapter Nine

22. Six Steps in a Test Market Study
4. Select the Test Market:
Market should not be over tested
Should have little media spillover
Demographics should be similar to your target population
Market should be large enough to provide useful results
Distribution and other patterns should be similar to the nation
5. Execute the Plan:
Determine length of time of test
Finalize the parties involved
6. Analyze the Results:
Use qualitative and quantitative techniques when possible
Chapter Nine

23. Limitations of Experimental Research
High Cost:
Is the research affordable?
Will the research be beneficial and help solve problems?
Has a cost and benefit analysis been done?
Security Issues:
Particularly critical with field experiments
The competition might be “tipped-off”
Are the data and findings secure?
Implementation Problems:
People who unwittingly get caught in the experiment
Outside factors unnaturally affecting the experiment
Participants who intentionally try to skew the results
Chapter Nine