PSYC512: Research Methods PSYC512: Research Methods Lecture 15 Brian P. Dyre University of Idaho

PSYC512: Research Methods Lecture 15 Outline Review of Lecture 14 Review of Lecture 14 Multi-factor research designs Multi-factor research designs Today: other specialized research designs Today: other specialized research designs Statistical “nuts and bolts” in conducting hypothesis tests of means Statistical “nuts and bolts” in conducting hypothesis tests of means Combining experimental and correlational designs (Analysis of covariance or ANCOVA) Combining experimental and correlational designs (Analysis of covariance or ANCOVA) Quasi-Experimental Designs Quasi-Experimental Designs Developmental Designs Developmental Designs Small-n designs and Psychophysical Methods Small-n designs and Psychophysical Methods

PSYC512: Research Methods Statistical “nuts and bolts” in testing hypotheses about means The central limit theorem The central limit theorem The mean of the sampling distribution of means is equal to the mean of the population from which the samples were drawn. The mean of the sampling distribution of means is equal to the mean of the population from which the samples were drawn. The variance of the sampling distribution of means is equal to the variance of the population from which the samples were drawn divided by the size of the samples. Expressed as the standard deviation: The variance of the sampling distribution of means is equal to the variance of the population from which the samples were drawn divided by the size of the samples. Expressed as the standard deviation: If the original population is distributed normally (i.e. it is bell shaped), the sampling distribution of means will also be normal. If the original population is not normally distributed, the sampling distribution of means will increasingly approximate a normal distribution as sample size increases. (i.e. when increasingly large samples are drawn) If the original population is distributed normally (i.e. it is bell shaped), the sampling distribution of means will also be normal. If the original population is not normally distributed, the sampling distribution of means will increasingly approximate a normal distribution as sample size increases. (i.e. when increasingly large samples are drawn)

PSYC512: Research Methods Does a sample mean significantly differ from a population mean?  of population is known  of population is known z test of sample mean z test of sample mean  of population is estimated as s  of population is estimated as s t-test of sample mean t-test of sample mean As N approaches infinity, t approaches z As N approaches infinity, t approaches z For small N distribution of sample variance is skewed For small N distribution of sample variance is skewed Df = N-1 Df = N-1

PSYC512: Research Methods The t-test Differences in means of matched samples Differences in means of matched samples Differences in means of Independent samples Differences in means of Independent samples

PSYC512: Research Methods Specialized Research Designs Combining between-subjects and within-subjects factors in research design – mixed designs Combining between-subjects and within-subjects factors in research design – mixed designs Combining experimental and correlational designs (Analysis of covariance or ANCOVA) Combining experimental and correlational designs (Analysis of covariance or ANCOVA) Quasi-Experimental Designs Quasi-Experimental Designs Pretest-posttest designs Pretest-posttest designs Developmental designs (Longitudinal or cross-sectional) Developmental designs (Longitudinal or cross-sectional)

PSYC512: Research Methods Combining Experimental and Correlational Designs Covariates in experimental designs Covariates in experimental designs Measure your subjects on a covariate—a variable that you believe may be correlated with your dependent variable Measure your subjects on a covariate—a variable that you believe may be correlated with your dependent variable If left unmeasured these covariates add error variance and might obscure significant effects If left unmeasured these covariates add error variance and might obscure significant effects Measuring the covariate allows you to use correlational statistical techniques in your analysis (e.g., Analysis of Covariance or ANCOVA) to “subtract out” the error variance associated with the covariate, thereby increasing the statistical power of your experiment Measuring the covariate allows you to use correlational statistical techniques in your analysis (e.g., Analysis of Covariance or ANCOVA) to “subtract out” the error variance associated with the covariate, thereby increasing the statistical power of your experiment Example: measuring IQ in a learning experiment Example: measuring IQ in a learning experiment

PSYC512: Research Methods Combining Experimental and Correlational Designs Quasi-independent variable in experimental designs Quasi-independent variable in experimental designs “Quasi” means “kind of, but not really” “Quasi” means “kind of, but not really” Similar to including a covariate, except Similar to including a covariate, except measurement of covariate is used to assign Ss to groups measurement of covariate is used to assign Ss to groups Covariate is thus treated as an quasi-independent variable Covariate is thus treated as an quasi-independent variable Quasi-independent variables are referred to as “quasi” because they cannot be manipulated, they are essentially dependent variables (measures) that are treated as independent variables in the experimental design and analysis Quasi-independent variables are referred to as “quasi” because they cannot be manipulated, they are essentially dependent variables (measures) that are treated as independent variables in the experimental design and analysis

PSYC512: Research Methods Quasi-experimental Designs Quasi-experimental designs are those in which only quasi- independent variables are used Quasi-experimental designs are those in which only quasi- independent variables are used Time series vs. pretest-posttest designs Time series vs. pretest-posttest designs Time series: Measure behavior several times prior to and following a treatment (time series design) or change in your quasi-independent variable (interrupted time series design) Time series: Measure behavior several times prior to and following a treatment (time series design) or change in your quasi-independent variable (interrupted time series design) Pretest-posttest: Measure behavior once prior to and once following the change in your independent variable Pretest-posttest: Measure behavior once prior to and once following the change in your independent variable

PSYC512: Research Methods Quasi-experimental Designs Equivalent time samples design Equivalent time samples design Time-series design especially useful for treatments with transient effects Time-series design especially useful for treatments with transient effects Repeatedly measure behavior following multiple applications and withdrawals of the treatment Repeatedly measure behavior following multiple applications and withdrawals of the treatment Non-equivalent control group designs – helps control for history confounds which should affect both groups equally Non-equivalent control group designs – helps control for history confounds which should affect both groups equally

PSYC512: Research Methods Developmental Designs Used to assess changes in behavior related to a person’s chronological age, which serves as a quasi-independent variable Used to assess changes in behavior related to a person’s chronological age, which serves as a quasi-independent variable Cross-sectional designs Cross-sectional designs Simultaneously test subjects assigned to two or more age groups Simultaneously test subjects assigned to two or more age groups Generational effects can confound the age variable Generational effects can confound the age variable Longitudinal designs Longitudinal designs Repeatedly test a single group of subjects over time Repeatedly test a single group of subjects over time Controls for generational effects—but, may still limit external validity Controls for generational effects—but, may still limit external validity May be confounded by history, mortality, and/or multiple observation effects May be confounded by history, mortality, and/or multiple observation effects Cohort-sequential design Cohort-sequential design Combines longitudinal and cross-sectional designs by measuring multiple age groups over time which allows evaluation of generational or historical confounds Combines longitudinal and cross-sectional designs by measuring multiple age groups over time which allows evaluation of generational or historical confounds

PSYC512: Research Methods Single Subject Research Designs Research that focuses on identifying functional relationships between variables and performance of a single subject (e.g., behavioral analysis and psychophysics) Research that focuses on identifying functional relationships between variables and performance of a single subject (e.g., behavioral analysis and psychophysics) Typically involve Typically involve Large number of observations Large number of observations Rigid experimental control Rigid experimental control Investigations of powerful variables whose effects are easily detected Investigations of powerful variables whose effects are easily detected

PSYC512: Research Methods Baseline Designs Same as time-series design Same as time-series design Time series: Measure behavior several times prior to and following a treatment Time series: Measure behavior several times prior to and following a treatment Two phases (A and B) Two phases (A and B) A: baseline phase to establish behavioral baseline  performance on DV prior to treatment, requires that a stability criterion be reached A: baseline phase to establish behavioral baseline  performance on DV prior to treatment, requires that a stability criterion be reached B: intervention phase that measures performance on DV after treatment B: intervention phase that measures performance on DV after treatment

PSYC512: Research Methods Baseline Reversal Designs Problem: Time confound Problem: Time confound Solution: ABA Design— reverse the treatment by removing it and see if performance returns to baseline (unlikely to occur by coincidence) Solution: ABA Design— reverse the treatment by removing it and see if performance returns to baseline (unlikely to occur by coincidence) Problem with reversal: now behavior is at baseline again Problem with reversal: now behavior is at baseline again Solution: ABAB design Solution: ABAB design A B A B

PSYC512: Research Methods Multiple Baseline Designs Used to assess irreversible changes in behavior Used to assess irreversible changes in behavior Assess multiple independent behaviors and introduce treatment to only one behavior at a time Assess multiple independent behaviors and introduce treatment to only one behavior at a time Controls for time effects (history, maturation) Controls for time effects (history, maturation)

PSYC512: Research Methods Discrete Trials Designs (Psychophysical Techniques) Used to determine thresholds and difference thresholds (just-noticeable differences or JNDs) Used to determine thresholds and difference thresholds (just-noticeable differences or JNDs) Ss receive dozens or hundreds of trials under tightly controlled conditions Ss receive dozens or hundreds of trials under tightly controlled conditions Methods Methods Method of Constant Stimuli Method of Constant Stimuli Method of Adjustment Method of Adjustment Method of Limits Method of Limits

PSYC512: Research Methods Classical Psychophysical Methods Fechner’s Elements of Psychophysics (1860) Fechner’s Elements of Psychophysics (1860) Absolute threshold (limen): how much energy must exist in a stimulus for it to be detectable? Absolute threshold (limen): how much energy must exist in a stimulus for it to be detectable? Subliminal: below threshold Superliminal: above threshold Amount of Stimulus Energy Predicted by absolute threshold Distribution of “absolute” threshold Probability of Detection “Real World”

PSYC512: Research Methods Methods for Determining Thresholds Method of adjustment Method of adjustment Intensity or feature of stimulus is changed until it matches a standard Intensity or feature of stimulus is changed until it matches a standard Hysteresis requires both ascending and descending trials Hysteresis requires both ascending and descending trials Average match across ascending and descending determines threshold Average match across ascending and descending determines threshold Fast, but least accurate Fast, but least accurate

PSYC512: Research Methods Methods for Determining Thresholds Method of Limits (discrete method of adjustment) Method of Limits (discrete method of adjustment) Like method of adjustment except adjustment is done in discrete steps whose size is controlled by the experimenter Like method of adjustment except adjustment is done in discrete steps whose size is controlled by the experimenter Hysteresis requires both ascending and descending trials Hysteresis requires both ascending and descending trials Average match across ascending and descending determines threshold Average match across ascending and descending determines threshold Variant: Staircase method Variant: Staircase method

PSYC512: Research Methods Methods for Determining Thresholds Method of Constant Stimuli Method of Constant Stimuli Choose 5-9 stimuli, some above, some below threshold Choose 5-9 stimuli, some above, some below threshold Present in random order Present in random order threshold equals amount of stimulus energy that detected 50% of the time threshold equals amount of stimulus energy that detected 50% of the time Slowest but most accurate Slowest but most accurate For all Methods: sensitivity = 1/threshold Amount of Stimulus Energy Probability of Detection Threshold

PSYC512: Research Methods Next Time… On-campus (“Live”) students: please me article presentation time slot preferences On-campus (“Live”) students: please me article presentation time slot preferences