1. Joseph G Pigeon Villanova University
Complex Experimental Design and Simple Data Analysis: A Pharmaceutical Example
Joseph G Pigeon
Villanova University

2. Introduction
Designs with restricted randomization have multiple error measures
Pharmaceutical example where the split plot structure is even more complex
Whole plot structure in two dimensions
Correlation structure in two dimensions
Caveats
Limited understanding of the biology involved
No originality of statistical methods claimed

3. Split Plot Designs Originated in agricultural experiments where
Levels of some factors are applied to whole plots
Levels of other factors are applied to sub plots
Separate randomizations to whole plots and sub plots
Two types of experimental units
Two types of error measures
Correlation among the observations

4. Split Plot Designs Also common in industrial experiments when
Complete randomization does not occur
Some factor levels may be impractical, inconvenient or too costly to change
This restriction on randomization results in some whole plot factors and some sub plot factors
Data analysis needs to account for this restricted randomization or split plot structure

5. Split Plot Example Consider a paper manufacturer who wants to study
Effects of 3 pulp preparation methods
Effects of 4 temperatures
Response is tensile strength
Pilot plant is capable of 12 runs per day
One replicate on each of three days

6. Split Plot Example

7. Split Plot Example
Initially, we might consider this to be a 4 x 3 factorial in a randomized block design
If true, then the order of experimentation within a block should have been completely randomized
However, this was not feasible; data were not collected this way
The multiplication sign in 4 x 3 had to be replaced with an x.

8. Split Plot Example Experiment was conducted as follows:
A batch of pulp was produced by one of the three methods
The batch was divided into four samples
Each sample was cooked at one of the four temperatures
Split plot design with
Pulp preparation method as whole plot treatment
Temperature as sub (split) plot treatment

9. Split Plot Example

10. Split Plot Example
Subplot error is less than whole plot error (typical)

11. Split Plot Example Lessons
We must carefully consider how the data were collected and incorporate all randomization restrictions into the analysis
Whole plot effects measured against whole plot error
Sub plot effects measured against sub plot error

12. Description of Example – MQPA Assay
Multivalent Q-PCR based Potency Assay
Used to assign potencies (independently) to each of five reassortants of a pentavalent vaccine
Relies on the quantitation of viral nucleic acid generated in 24 hours
Two major components
Biological component (infection of the standard and sample viruses)
Biochemical component (quantitative PCR reaction where PCR = Polymerase Chain Reaction)

13. Polymerase Chain Reaction (PCR)

14. Description of Example- Biological Component
Vero cell maintenance and set up
Serial dilution of known standard and unknown sample are incubated with trypsin
Infected in 4 replicate wells of Vero cell monolayers seeded in a 96 well plate
Infection proceeds for 24 hours and then halted with the addition of a detergent and storage at –70C

15. Description of Example- Biochemical Component
Lysate is thawed and diluted
Preparation of a “master mix”
Preparation of Q-PCR plate (master mix + diluted lysates)
Configuration of the Q-PCR detection system
Potency is determined by parallel line analysis of standard and test samples
Specific interest is on optimization of the PCR portion of the assay

16. PCR Optimization Design
Discussions with Biologists identified 13 factors
8 factors associated with preparation of master mix
5 factors associated with configuration of PCR detection system (instrument)
Discussions with Biologists identified 3 responses
Lowest cycle time (range: 1 – 40)
Least variability between replicates
Valid amplification plot (range: 0 – 4)
Completion of experiments and analysis immediately!

17. PCR Optimization Design

18. PCR Optimization Design Considerations
Interactions not expected to exist
Experiments performed in a 96 well plate
Each plate can accommodate at most 15 master mix combinations
12 run PB deign for 8 factors
The exponent 2 ^ (5-1) would not copy from word. I left two parentheses to mark the spot.

19. PCR Optimization Design Considerations
Time constraints imply at most 16 plates (instrument settings)
25-1 fractional factorial for 5 factors (5 = 1234)
Concern about using only 12 of 28 combinations
Half of the plates use a 12 run PB design (123 = 45 = +1)
Half of the plates use the foldover PB design (123 = 45 = 1)

20. Plackett-Burman Design
Factors: Replicates: Design: 12
Runs: Center pts (total): 0
Data Matrix (randomized)
Run A B C D E F G H

21. Half Fraction Design Factors: 5 Base Design: 5, 16 Resolution: V
Runs: Replicates: Fraction: 1/2
Blocks: none Center pts (total):
Design Generators: E = ABCD
Row StdOrder RunOrder A B C D E

22. PCR Optimization Design Layout
Each  represents a 12 run PB design
16 × 12 = 192 observations

23. PCR Optimization Design Layout
Master Mix 1 2  11 12 13 14 23 24 X
Plate  15 16

24. PCR Optimization Design Layout
Master Mix 1 2  11 12 13 14 23 24 X
Plate  15 16
Whole plot structure in two dimensions

25. PCR Optimization Results
Biologists provided this summary of the 21 runs with an amplification plot rating of 4

26. PCR Optimization Results
plate Count mm Count mm1 Count mm2 Count mm3 Count mm4 Count
N= N= N= N= N= N=
mm5 Count mm6 Count mm7 Count mm8 Count instr1 Count N=
N= N= N= N=
instr2 Count instr3 Count instr4 Count instr5 Count
N= N= N= N=

27. PCR Optimization Analysis Log
mm7 = 1; instr4 = –1

28. PCR Optimization Results
plate Count mm Count mm1 Count mm2 Count mm3 Count mm4 Count
N= N= N= N=
N= N=
mm5 Count mm6 Count mm7 Count mm8 Count instr1 Count
N= N= N= N= N=
instr2 Count instr3 Count instr4 Count instr5 Count
N= N= N= N=

29. PCR Optimization Analysis Log
mm7 = 1; instr4 = –1
mm3 = 1; mm7 = 1; mm8 = –1; instr3 = 1

30. PCR Optimization Results
Fractional Factorial Fit: ctgm
Estimated Effects and Coefficients for ctgm (coded units)
Term Effect Coef SE Coef T P
Constant
instr
instr
instr
instr
instr
instr1*instr
instr1*instr
instr1*instr
instr1*instr
instr2*instr
instr2*instr
instr2*instr
instr3*instr
instr3*instr
instr4*instr

31. PCR Optimization Results

32. PCR Optimization Results

33. PCR Optimization Results

34. PCR Optimization Results

35. PCR Optimization Analysis Log
mm7 = 1; instr4 = -1
mm3 = 1; mm7 = 1; mm8 = -1; instr3 = 1
Instr3 = 1; instr2 and instr5 should have opposite signs?

36. PCR Optimization Results

37. PCR Optimization Results

38. PCR Optimization Results
Estimated Effects and Coefficients for ctgm (coded units)
Term Effect Coef SE Coef T P
Constant mm mm mm mm mm mm mm mm

39. PCR Optimization Results

40. PCR Optimization Results

41. PCR Optimization Results

42. PCR Optimization Analysis Log
mm7 = 1; instr4 = – 1
mm3 = 1; mm7 = 1; mm8 = –1; instr3 = 1
instr3 = 1; instr2 and instr5 should have opposite signs?
mm3 = 1; mm7 = 1; mm8 = –1

43. PCR Optimization Results
Row plate mm ct1 ct2 ct3 ct4 ctgm well1 well2
Row amprating mm1 mm2 mm3 mm4 mm5 mm6 mm7 mm8 instr1 instr2
Row instr3 instr4 instr5

44. PCR Optimization Results

45. PCR Optimization Results

46. PCR Optimization Summary
No complex models – all simple analyses
5 factors were found to be significant (mm3, mm7, mm8, instr3 and instr4)
These factors were further studied using response surface experiments
Scientists seem quite happy with the results of the PCR optimization experiments

47. Concluding Remarks
Many industrial experiments do have a split or strip plot structure which means multiple and possibly complex error measures
Arises from the conduct of an experiment and/or any restrictions on the randomization
We need to incorporate these considerations into a proper analysis and interpretation of experimental data

48. Concluding Remarks
Experimental designs with balance, symmetry and orthogonality permit simple but effective graphical analyses (even with some missing data)
Much can be learned from simple analyses following suitable experimental design
All models are wrong, but some models are useful
All models are wrong, but some models are more wrong than others