1. Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals School of IT, The University of Sydney 1 Factorial Designs Week 9 Lecture 2

2. Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals School of IT, The University of Sydney 2 Agenda Basic factorial design concepts Main and interaction effect Factorial design in computer system performance analysis

3. Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals School of IT, The University of Sydney 3 What are factorial designs Two or more independent variables are manipulated in a single experiment They are referred to as factors The major purpose of the research is to explore their effects jointly Factorial design produce efficient experiments, each observation supplies information about all of the factors

4. Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals School of IT, The University of Sydney 4 A simple example Investigate an education program with a variety of variations to find out the best combination –Amount of time receiving instruction 1 hour per week vs. 4 hour per week –Settings In-class vs. pull out 2 X 2 factorial design –Number of numbers tells how many factors –Number values tell how many levels –The result of multiplying tells how many treatment groups that we have in a factorial design

5. Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals School of IT, The University of Sydney 5 Null outcome None of the treatment has any effect Main effect –is an outcome that is a consistent difference between levels of a factor. Interaction effect –An interaction effect exists when differences on one factor depend on the level you are on another factor.

6. Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals School of IT, The University of Sydney 6 Main effects Main effect of time Main effect of setting Main effects on both

7. Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals School of IT, The University of Sydney 7 Interaction effect An interaction effect exists when differences on one factor depend on the level of another factor How do we know if there is an interaction in a factorial design? –Statistical analysis will report all main effects and interactions. –If you can not talk about effect on one factor without mentioning the other factor –Spot an interaction in the graphs – whenever there are lines that are not parallel there is an interaction present!

8. Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals School of IT, The University of Sydney 8 Interaction effect Interaction as a difference in magnitude of response Interaction as a difference in direction of response

9. Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals School of IT, The University of Sydney 9 Factorial design variations A 2 X 3 example study the effect of different treatment combinations for cocaine abuse. –Factor 1: treatment psychotherapy behavior modification –Factor 2: inpatient day treatment outpatient –Dependent variable severity of illness rating

10. Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals School of IT, The University of Sydney 10 Factorial designs in computer system performance analysis Personal workstation design –Processor: 68000, Z80, 8086 –Memory size: 512K 2M or 8M bytes –Number of disks: one, two or three –Workload: Secretarial, managerial or scientific –User education: high school, college, post- graduate level Dependent variable –Throughput, response time

11. Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals School of IT, The University of Sydney 11 2 2 factorial design Two factors, each at two levels Example: workstation design –Factor 1: memory size –Factor 2: cache size –DV: performance in MIPS Cache size Memory size 4M byte8M byte 1K1545 2K2575

12. Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals School of IT, The University of Sydney 12 Quantify effects We want to learn which factor contribute more to the performance. –Define two variable –The regression model

13. Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals School of IT, The University of Sydney 13 Quantifying results (cont) Resolving those coefficients We get How do you read this?

14. Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals School of IT, The University of Sydney 14 Quantify effects by sign table Sign table method IABABy 1 115 11 45 11 25 111175 160804020Total 4020105Total/4

15. Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals School of IT, The University of Sydney 15 2 K factorial design K factors, each at two level 2 K experiments 2 3 design example –In designing a personal workstation, the three factors needed to be studied are: cache size, memory size and number of processors FactorLevel -1Level 1 Memory size4Mbytes16Mbytes Catch size1Kbytes2Kbytes Number of processors 12 Cache size (Kbytes) 4 Mbytes16 Mbytes 1212 114462258 210503486

16. Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals School of IT, The University of Sydney 16 2 k factorial design with replication r replications of 2 k experiments –2 K r observations –Allows estimation of experimental errors –2 2 3 design example The memory-cache experiments were repeated three times each. The result is shown below Cache sizeMemory size 4M8M 1 K15, 18, 1245, 48,51 2K25, 28, 1975,75,81

17. Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals School of IT, The University of Sydney 17 Full and fractional factorial design Full factorial design –Study all combinations –Can find effect of all factors –May try 2 K factorial design first Fractional (incomplete) factorial design –Leave some treatment groups empty –Less information –May not get all interactions –No problem if interaction is negligible

18. Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals School of IT, The University of Sydney 18 2 k-p Fractional factorial design Large number of factors –Large number of experiments –Full factorial design too expensive –Use a fractional factorial design 2 k-p design allows analyzing k factors with only 2 k-p experiments. –2 k-1 design requires only half as many experiments –2 k-2 design requires only one quarter of the experiments

19. Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals School of IT, The University of Sydney 19 Example: 2 7-4 Design Exp No.ABCCEFG 1 111 21 11 3 1 1 1 411 1 5 11 1 61 1 1 7 11 1 81111111 Study 7 factors with only 8 experiments When quantify the effects, just calculate the main effects Will be able to eliminate some factors in further study.

20. Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals School of IT, The University of Sydney 20 Quantify model IABCCEFG 1 111 20 11 1135 11 1 17 111 1 42 1 11 136 111 1 50 111 1 45 1111111182 31710135109431473Total 39.6212.624.3713.625.370.1255.870.37Total/8

21. Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals School of IT, The University of Sydney 21 Preparing the sign table Choose k - p factors and prepare a complete sign table for a full factorial design with k-p factors Of the 2 k-p –k +p -1 column on the right, choose p columns and mark them with the p factors that were not chosen in step 1.