IMD09120: Collaborative Media Brian Davison 2010/11 Research design IMD09120: Collaborative Media Brian Davison 2010/11

Research design Recap Variables Types of design Hypotheses Sampling Your designs

Recap: tests you have done (1) Can the number of social networking accounts predict the number of contacts a person has? Do female students have significantly more social networking contacts than male students? Does face-to-face differ significantly from WebCT chat as a channel of communication in terms of satisfaction? Do students from different schools differ significantly in the number of social networking contacts they have?

Recap: tests you have done (2) Do students differ significantly by age in the number of social networking contacts they have? Does the quality of video and/or group size have a significant effect on satisfaction in mediated group communication? Is the preference for Facebook statistically significant? Is preferred social networking site related to gender?

Recap: tests you have used Correlation Independent t-test Paired t-test ANOVA 2

Variables Things you can measure Height, weight, age Number of car journeys Gender, nationality Attitude, belief, satisfaction

Types of variable Continuous Discrete Categorical Examples Infinitely divisible Discrete Values come in defined ordered steps Categorical Values are categories with no ordering Examples Height Number of car journeys Gender Nationality Observed symptoms of disease Attitude Age Belief Weight Satisfaction

Dependence Independent variable Dependent variable Examples Manipulated by the investigator Dependent variable Measured for effect Examples Does face-to-face differ significantly from WebCT chat as a channel of communication in terms of satisfaction? Is preferred social networking site related to gender?

Confounding variable A factor that affects the dependent variable and which is not included in the design What confounding variables might there be in the F2F-chat experiment?

Research designs Correlational Experimental and quasi-experimental Looking at how one variable changes in relation to another eg smoking and incidence of cancer Experimental and quasi-experimental Looking for effects of one variable on another IV and DV

Correlation designs Collect data about both variables Measure the extent to which their distributions match Causation cannot be inferred from a correlation Which is correlational: Can the number of social networking accounts predict the number of contacts a person has? Do female students have significantly more social networking contacts than male students? Does face-to-face differ significantly from WebCT chat as a channel of communication in terms of satisfaction?

Experimental designs Researcher manipulates IV Subjects are randomly assigned to test conditions Which is experimental: Can the number of social networking accounts predict the number of contacts a person has? Do female students have significantly more social networking contacts than male students? Does face-to-face differ significantly from WebCT chat as a channel of communication in terms of satisfaction?

Quasi-experimental designs Some variables cannot be manipulated eg gender, occupation Therefore random allocation to conditions is impossible Which is quasi-experimental: Do students differ significantly by age in the number of social networking contacts they have? Does the quality of video and/or group size have a significant effect on satisfaction in mediated group communication? Is the preference for Facebook statistically significant?

Between or within subjects Between-subjects design Different subjects randomly assigned to different conditions Within-subjects design Same subjects used in all conditions

Within-subjects Also called related or repeated measures Pro: Con: Using same subjects helps to eliminate confounding variables Fewer participants needed Con: Subjects may perform differently on second test – ordering effects Subjects more likely to guess the purpose of the study – demand effects Not possible for quasi-experimental studies Ordering effects can be eliminated by counterbalancing

Between-subjects Also called independent or unrelated May be significant variation within each group Pro: No ordering effects Demand effects are less likely Con: More participants required Less control over confounding variables

What kind of design? Do female students have significantly more social networking contacts than male students? Does the quality of video and/or group size have a significant effect on satisfaction in mediated group communication? Does face-to-face differ significantly from WebCT chat as a channel of communication in terms of satisfaction?

Population Pop. mean

Sampling error Population Sample Pop. mean Spl. mean

What rule connects this series of numbers? 10, 20, 30 Confirmation bias

Testing hypotheses Null hypothesis Alternative hypothesis There is no relationship / difference between two variables ie. All observed differences are the result of sampling error Probability values in tests = probability of the observed result if the null hypothesis is true Alternative hypothesis There is a relationship / difference between two variables Specific to the study you are doing

Logic of hypothesis testing Formulate a hypothesis Measure the variables and examine their relationship (using samples) Calculate the probability of the result if the null hypothesis is true If the calculated probability is small enough, reject the null hypothesis

Two types of test Is x significantly larger / smaller than expected? Is x significantly different from expected?

How many tails? Do female students have significantly more social networking contacts than male students? Does face-to-face differ significantly from webCT chat as a channel of communication in terms of satisfaction? Do students from different schools differ significantly in the number of social networking contacts they have?

Short break

Effect size Small effect Large effect A large effect size is easier to detect than a small one The power of a test is a measure of its ability to detect an effect More participants are required to detect a small effect size

Cohen’s power primer Tabulates required sample size according to Confidence interval Statistical test Effect size Values for 95% confidence Cohen (1992) Test Small Medium Large Pearson’s r 783 85 28 T-test 393 64 26

ANOVA sample sizes for 95% confidence Groups Small Medium Large 2 393 64 26 3 322 52 21 4 274 45 18 5 240 39 16 6 215 35 14 7 195 32 13

2 sample sizes for 95% confidence df Small Medium Large 1 785 87 26 2 964 107 39 3 1,090 121 44 4 1,194 133 48 5 1,293 143 51 6 1,362 151 54 More precise figures can be calculated using G*Power

How many variables do you have? Are you looking for differences between conditions, or relationships among variables? Do you have a between-participants or a within-participants design? Do you want a regression equation, or simply the strength of a relationship? Are you interested in a regression equation, or exploring clusters of correlations? Do you have more than one DV? Do you have more than one IV? Do you want to look for differences between conditions while controlling for the effects of another variable? Independent t-test Related t-test Pearson’s Product Moment Correlation Coefficient Linear regression Multiple regression Factor analysis One-way ANOVA ANOVA for multiple IVs MANOVA Analysis of covariance Two More than two Differences Relationships Between Within Strength Regression Correlation Yes No Dancey & Reidy (2007) Statistics without maths for psychologists. Prentice Hall (p. 157)

How many variables do you have? Are you looking for differences between conditions, or relationships among variables? Do you have a between-participants or a within-participants design? Do you want a regression equation, or simply the strength of a relationship? Are you interested in a regression equation, or exploring clusters of correlations? Do you have more than one DV? Do you have more than one IV? Do you want to look for differences between conditions while controlling for the effects of another variable? Independent t-test Related t-test Pearson’s Product Moment Correlation Coefficient Linear regression Multiple regression Factor analysis One-way ANOVA ANOVA for multiple IVs MANOVA Analysis of covariance Two More than two Differences Relationships Between Within Strength Regression Correlation Yes No Dancey & Reidy (2007) Statistics without maths for psychologists. Prentice Hall (p. 157)

Your turn Design a study to test whether caffeine affects level of anxiety.

Your turn again Design a study to test whether mathematical ability is related to musical ability

One more time Design a study to test whether there is a relationship between doing sport and intention to vote

Coursework: variables and hypotheses What are you trying to improve? eg group identity, social presence, group effectiveness This is your DV What are you changing? eg additional features, different layout, modified presentation Your IV is WebCT version: may be old/new, or +/- combined factors What do you expect? Eg adding feature X leads to reduced social loafing This is your hypothesis Null hypothesis is “there is no difference”

Coursework: research design Levels of IV How to measure DV Experimental groups Within subjects or between subjects Eliminating confounding variables and unwanted effects Quality of system – create an “as-is” Powerpoint Ordering effects Demand effects Researcher bias

Statistical test Method 1 Method 2 Use one method to check the other What question are you asking? Which practical example matches? Method 2 Follow the flowchart Use one method to check the other