Statistics and Research methods Wiskunde voor HMI Bijeenkomst 3 Relating statistics and experimental design

Contents Multiple regression Inferential statistics Basic research designs Hypothesis testing – Learn to select the appropriate statistical test in a particular research design

Multiple Regression Multiple correlation – The association between a criterion variable and two or more predictor variables Multiple regression – Making predictions with two or more predictor variables

Multiple Regression Multiple regression prediction models – Each predictor variable has its own regression coefficient – e.g., Z-score multiple regression formula with three predictor variables: Standardized regression coefficients

Note: the betas are not the same as the correlation coefficients for each predictor variable (because predictors “overlap”) Standardized regression coefficient (Beta) of a variable: about unique, distinctive contribution of that variable (overlap excluded) There is also a corresponding raw score prediction formula for multiple regression: Ŷ = a + (b 1 )(X 1 ) + (b 2 )(X 2 ) + (b 3 )(X 3 ) Multiple Regression

Multiple correlation coefficient R In SPSS output: Multiple R R is usually smaller than the sum of individual correlation coefficients in bivariate regression R 2 is proportionate reduction in error = proportion of variance accounted for

Research Example

Inferential Statistics Make decisions about populations based on information in samples (as opposed to descriptive statistics, which summarize the attributes of known data) Notations in statistical test theory

Sample and population

The Normal Distribution (Z-scores) Normal curve and percentage of scores between the mean and 1 and 2 standard deviations from the mean

Basic research methods Experimental method – manipulation of variables and measure effects Field studies – observation – No outside intervention, e.g. ethnography Quasi-experimental method – Combination of elements of other two We concentrate on experiments and quasi-experiments

Experimental method Manipulation of (levels of) one or more independent variables (e.g. medication: pill or placebo; different versions of a user interface)  experimental conditions Control (keep constant) other possibly intervening variables Measure dependent variables (e.g. effectiveness, performance, satisfaction) Test for differences between the conditions

Experimental design How to assign subjects to conditions? Between-subjects design – a subject is assigned to only one of the conditions Within-subjects design or Repeated measures design – Each subjects receives all the experimental conditions

Between-subjects design Randomization: assign subject at random to different conditions Matching: random assignment but control for variable that is expected to be very relevant Example:(if sex is important) seperately assign men to experimental groups assign women to experimental groups Equal amount of men and women in conditions. “the subjetcs in each condition were matched on sex”

Between-subjects design (continued) Matched pairs – Two subjects that are similar (on relevant variable(s)) assigned to different conditions Randomized blocks design – Extension of matched pairs for more than two conditions, e.g. 3 conditions – Form blocks of 3 similar subjects – Assign subjects in one block randomly to different conditions

Between-subjects design (continued) Factorial designs – More than one independent variable – Study separate effects of each variable (main effects) but also interaction between variables – Interaction effect: the impact of one variable depends on the level of the other variable – Two-way factorial research design (two independent variables); three-way with three indep. variables – 2x2 if independent variables have two levels (condions) or 3x3 with three levels

Within-subjects design Same subjects in each experimental condition Repeated measures design – Within-subjects design required if change is measured as a consequence of an experimental treatment (e.g. testscores before and after a training) In other situations: carryover effects – experimental conditions need to be counterbalanced – One half sequence AB the other half BA

Quasi-experimental method Combination of elements from experimental methods and field research

Hypotheses Testing H 0 : Null hypothesis – No difference – The Independent variable has no effect e.g. pill or placebo make no difference H 1 (or H a ): Alternative hypothesis – Significant difference – The Independent variable has an effect

Hypothesis Testing Errors Type I Error: – Null hypothesis is rejected but true. – Alpha (α) probability of making type I error Type II Error: – Null hypothesis is not rejected but false. – Beta (β) probability of making type II error No effect, but you say there is. Real effect, but you say there’s not.

Type I and II errors α usually 0.05 or 0.01 β usually 0.20

Statistical Power Power: The probability that a test will correctly reject a false null hypothesis (1- β )

An Example of Hypothesis Testing A person claims to be able to identify people of above- average intelligence (IQ) with her eyes closed We devise a test – take her to a stadium full of randomly selected people from the population and ask her to pick someone with her eyes closed who is of above average IQ. If she does, we’ll be convinced. But she might pick someone with an above-average IQ just by chance.

Distribution of IQ Scores

The Hypothesis Testing Process 1. Restate the question as a research hypothesis and a null hypothesis about the populations  Population 1  Population 2  Research hypothesis or alternative hypothesis  Null hypothesis

The Hypothesis Testing Process 2. Determine the characteristics of the comparison distribution  Comparison distribution: distribution of the sort you would have if the null-hypothesis were true.

The Hypothesis Testing Process 3. Determine the cutoff sample score on the comparison distribution at which the null hypothesis should be rejected  Cutoff sample score  Conventional levels of significance: p <.05, p <.01

The Hypothesis Testing Process 4. Determine your sample’s score on the comparison distribution 5. Decide whether to reject the null hypothesis

One-Tailed and Two-Tailed Hypothesis Tests Directional hypotheses – One-tailed test Nondirectional hypotheses – Two-tailed test

Determining Cutoff Points With Two- Tailed Tests Divide up the significance between the two tails