1. IMD09120: Collaborative Media Brian Davison 2010/11
Research design
IMD09120: Collaborative Media
Brian Davison 2010/11

2. Research design Recap Variables Types of design Hypotheses Sampling
Your designs

3. Recap: tests you have done (1)
Can the number of social networking accounts predict the number of contacts a person has?
Do female students have significantly more social networking contacts than male students?
Does face-to-face differ significantly from WebCT chat as a channel of communication in terms of satisfaction?
Do students from different schools differ significantly in the number of social networking contacts they have?

4. Recap: tests you have done (2)
Do students differ significantly by age in the number of social networking contacts they have?
Does the quality of video and/or group size have a significant effect on satisfaction in mediated group communication?
Is the preference for Facebook statistically significant?
Is preferred social networking site related to gender?

5. Recap: tests you have used
Correlation
Independent t-test
Paired t-test
ANOVA 2

6. Variables Things you can measure Height, weight, age
Number of car journeys
Gender, nationality
Attitude, belief, satisfaction

7. Types of variable Continuous Discrete Categorical Examples
Infinitely divisible
Discrete
Values come in defined ordered steps
Categorical
Values are categories with no ordering
Examples
Height
Number of car journeys
Gender
Nationality
Observed symptoms of disease
Attitude
Age
Belief
Weight
Satisfaction

8. Dependence Independent variable Dependent variable Examples
Manipulated by the investigator
Dependent variable
Measured for effect
Examples
Does face-to-face differ significantly from WebCT chat as a channel of communication in terms of satisfaction?
Is preferred social networking site related to gender?

9. Confounding variable
A factor that affects the dependent variable and which is not included in the design
What confounding variables might there be in the F2F-chat experiment?

10. Research designs Correlational Experimental and quasi-experimental
Looking at how one variable changes in relation to another
eg smoking and incidence of cancer
Experimental and quasi-experimental
Looking for effects of one variable on another
IV and DV

11. Correlation designs Collect data about both variables
Measure the extent to which their distributions match
Causation cannot be inferred from a correlation
Which is correlational:
Can the number of social networking accounts predict the number of contacts a person has?
Do female students have significantly more social networking contacts than male students?
Does face-to-face differ significantly from WebCT chat as a channel of communication in terms of satisfaction?

12. Experimental designs Researcher manipulates IV
Subjects are randomly assigned to test conditions
Which is experimental:
Can the number of social networking accounts predict the number of contacts a person has?
Do female students have significantly more social networking contacts than male students?
Does face-to-face differ significantly from WebCT chat as a channel of communication in terms of satisfaction?

13. Quasi-experimental designs
Some variables cannot be manipulated
eg gender, occupation
Therefore random allocation to conditions is impossible
Which is quasi-experimental:
Do students differ significantly by age in the number of social networking contacts they have?
Does the quality of video and/or group size have a significant effect on satisfaction in mediated group communication?
Is the preference for Facebook statistically significant?

14. Between or within subjects
Between-subjects design
Different subjects randomly assigned to different conditions
Within-subjects design
Same subjects used in all conditions

15. Within-subjects Also called related or repeated measures Pro: Con:
Using same subjects helps to eliminate confounding variables
Fewer participants needed
Con:
Subjects may perform differently on second test – ordering effects
Subjects more likely to guess the purpose of the study – demand effects
Not possible for quasi-experimental studies
Ordering effects can be eliminated by counterbalancing

16. Between-subjects Also called independent or unrelated
May be significant variation within each group
Pro:
No ordering effects
Demand effects are less likely
Con:
More participants required
Less control over confounding variables

17. What kind of design?
Do female students have significantly more social networking contacts than male students?
Does the quality of video and/or group size have a significant effect on satisfaction in mediated group communication?
Does face-to-face differ significantly from WebCT chat as a channel of communication in terms of satisfaction?

18. Population
Pop. mean

19. Sampling error
Population
Sample
Pop. mean
Spl. mean

20. What rule connects this series of numbers?
10, 20, 30
Confirmation bias

21. Testing hypotheses Null hypothesis Alternative hypothesis
There is no relationship / difference between two variables
ie. All observed differences are the result of sampling error
Probability values in tests = probability of the observed result if the null hypothesis is true
Alternative hypothesis
There is a relationship / difference between two variables
Specific to the study you are doing

22. Logic of hypothesis testing
Formulate a hypothesis
Measure the variables and examine their relationship (using samples)
Calculate the probability of the result if the null hypothesis is true
If the calculated probability is small enough, reject the null hypothesis

23. Two types of test Is x significantly larger / smaller than expected?
Is x significantly different from expected?

24. How many tails?
Do female students have significantly more social networking contacts than male students?
Does face-to-face differ significantly from webCT chat as a channel of communication in terms of satisfaction?
Do students from different schools differ significantly in the number of social networking contacts they have?

25. Short break

26. Effect size Small effect Large effect
A large effect size is easier to detect than a small one
The power of a test is a measure of its ability to detect an effect
More participants are required to detect a small effect size

27. Cohen’s power primer Tabulates required sample size according to
Confidence interval
Statistical test
Effect size
Values for 95% confidence
Cohen (1992)
Test
Small
Medium
Large
Pearson’s r
783 85 28
T-test
393 64 26

28. ANOVA sample sizes for 95% confidence
Groups
Small
Medium
Large 2
393 64 26 3
322 52 21 4
274 45 18 5
240 39 16 6
215 35 14 7
195 32 13

29. 2 sample sizes for 95% confidencedf
Small
Medium
Large 1
785 87 26 2
964
107 39 3
1,090
121 44 4
1,194
133 48 5
1,293
143 51 6
1,362
151 54
More precise figures can be calculated using G*Power

30. How many variables do you have?
Are you looking for differences between conditions, or relationships among variables?
Do you have a between-participants or a within-participants design?
Do you want a regression equation, or simply the strength of a relationship?
Are you interested in a regression equation, or exploring clusters of correlations?
Do you have more than one DV?
Do you have more than one IV?
Do you want to look for differences between conditions while controlling for the effects of another variable?
Independent t-test
Related t-test
Pearson’s Product Moment Correlation Coefficient
Linear regression
Multiple regression
Factor analysis
One-way ANOVA
ANOVA for multiple IVs
MANOVA
Analysis of covariance
Two
More than two
Differences
Relationships
Between
Within
Strength
Regression
Correlation
Yes No
Dancey & Reidy (2007) Statistics without maths for psychologists. Prentice Hall (p. 157)

31. How many variables do you have?
Are you looking for differences between conditions, or relationships among variables?
Do you have a between-participants or a within-participants design?
Do you want a regression equation, or simply the strength of a relationship?
Are you interested in a regression equation, or exploring clusters of correlations?
Do you have more than one DV?
Do you have more than one IV?
Do you want to look for differences between conditions while controlling for the effects of another variable?
Independent t-test
Related t-test
Pearson’s Product Moment Correlation Coefficient
Linear regression
Multiple regression
Factor analysis
One-way ANOVA
ANOVA for multiple IVs
MANOVA
Analysis of covariance
Two
More than two
Differences
Relationships
Between
Within
Strength
Regression
Correlation
Yes No
Dancey & Reidy (2007) Statistics without maths for psychologists. Prentice Hall (p. 157)

32. Your turn
Design a study to test whether caffeine affects level of anxiety.

33. Your turn again
Design a study to test whether mathematical ability is related to musical ability

34. One more time
Design a study to test whether there is a relationship between doing sport and intention to vote

35. Coursework: variables and hypotheses
What are you trying to improve?
eg group identity, social presence, group effectiveness
This is your DV
What are you changing?
eg additional features, different layout, modified presentation
Your IV is WebCT version: may be old/new, or +/- combined factors
What do you expect?
Eg adding feature X leads to reduced social loafing
This is your hypothesis
Null hypothesis is “there is no difference”

36. Coursework: research design
Levels of IV
How to measure DV
Experimental groups
Within subjects or between subjects
Eliminating confounding variables and unwanted effects
Quality of system – create an “as-is” Powerpoint
Ordering effects
Demand effects
Researcher bias

37. Statistical test Method 1 Method 2 Use one method to check the other
What question are you asking?
Which practical example matches?
Method 2
Follow the flowchart
Use one method to check the other