PSYC512: Research Methods PSYC512: Research Methods Lecture 6 Brian P. Dyre University of Idaho

PSYC512: Research Methods Lecture 5 Outline Questions about material covered in Lecture 5 Questions about material covered in Lecture 5 Scientific Method: Proof and disproof & Strong Inference Scientific Method: Proof and disproof & Strong Inference Operational definitions Operational definitions Issues in Measurement Issues in Measurement Choosing Measures Choosing Measures Scales of Measurement Scales of Measurement Variables: Reliability and validity Variables: Reliability and validity sampling sampling

PSYC512: Research Methods Choosing Measures Research tradition e.g., operant conditioning—lever pressing e.g., cognition—accuracy and reaction time e.g., sensation and perception—discrimination accuracy e.g., personality—surveys, inventories (self-reports) Research tradition e.g., operant conditioning—lever pressing e.g., cognition—accuracy and reaction time e.g., sensation and perception—discrimination accuracy e.g., personality—surveys, inventories (self-reports) Theory e.g., the psychophysical postulate – discrimination accuracy e.g., Serial vs. parallel processes in visual search – RT Theory e.g., the psychophysical postulate – discrimination accuracy e.g., Serial vs. parallel processes in visual search – RT Availability of new techniques Availability of new techniques Availability of equipment Availability of equipment

PSYC512: Research Methods Features of Measures: Scale of Measurement (Stevens, 1946) Four types: nominal, ordinal, interval, and ratio Four types: nominal, ordinal, interval, and ratio Nominal scales Nominal scales set of unique cases, types, or categories with NO ORDER set of unique cases, types, or categories with NO ORDER Only non-parametric operations are valid: counting frequencies, modes, chi-square, point-biserial correlation Only non-parametric operations are valid: counting frequencies, modes, chi-square, point-biserial correlation Ordinal scales Ordinal scales different categories that can be ranked along a continuum different categories that can be ranked along a continuum more or less, but not how much more or less more or less, but not how much more or less Only non-parametric operations are valid : counting frequencies, modes, medians, chi-square, rank-order correlation Only non-parametric operations are valid : counting frequencies, modes, medians, chi-square, rank-order correlation

PSYC512: Research Methods Features of Measures: Scale of Measurement (Stevens, 1946) Interval Interval intervals of the scale are equal in magnitude intervals of the scale are equal in magnitude Necessary but not sufficient condition for parametric statistical tests Necessary but not sufficient condition for parametric statistical tests valid operations: all mathematical operations, means, standard deviations, etc. may be calculated valid operations: all mathematical operations, means, standard deviations, etc. may be calculated If other distributional assumptions are met: linear and non-linear regression, t-tests, ANOVA are also valid If other distributional assumptions are met: linear and non-linear regression, t-tests, ANOVA are also valid no fundamental zero—no ratio statements allowed no fundamental zero—no ratio statements allowed Ratio Ratio Like interval but also has a fundamental zero point—allows ratio statements such as “A is twice as much as B” Like interval but also has a fundamental zero point—allows ratio statements such as “A is twice as much as B” Generally interval or ratio scales should be used if possible Generally interval or ratio scales should be used if possible More powerful and flexible statistical tests More powerful and flexible statistical tests More precision in evaluating quantitative hypotheses More precision in evaluating quantitative hypotheses

PSYC512: Research Methods Features of Measures: Sensitivity Sensitivity: measure must show changes in response to changes in the independent variable Sensitivity: measure must show changes in response to changes in the independent variable Range effects Range effects Ceiling effects: variable reaches its highest possible value and gets truncated (test is too easy) Ceiling effects: variable reaches its highest possible value and gets truncated (test is too easy) Floor effects: variable reaches its lowest possible value and gets truncated (test is too hard) Floor effects: variable reaches its lowest possible value and gets truncated (test is too hard)

PSYC512: Research Methods Features of Measures: Reliability the ability of a measure to produce consistent results when repeated measurements are taken under identical conditions the ability of a measure to produce consistent results when repeated measurements are taken under identical conditions Types: Types: precision: physical measurement (1/noise) precision: physical measurement (1/noise) margin of error: sampling in surveys margin of error: sampling in surveys interrater reliability: observers viewing the same behavior interrater reliability: observers viewing the same behavior Test-retest, parallel forms and split-half reliabilities: psychological tests Test-retest, parallel forms and split-half reliabilities: psychological tests

PSYC512: Research Methods Other Features of Measures Accuracy Accuracy does a measure produce results that agree with a known standard? does a measure produce results that agree with a known standard? Accuracy vs. Precision Accuracy vs. Precision Validity Validity Measurement validity: the extent to which your measure indeed measures what it is intended to measure Measurement validity: the extent to which your measure indeed measures what it is intended to measure Types: Face validity, Content validity, Criterion- related validity (concurrent vs. predictive), Construct validity Types: Face validity, Content validity, Criterion- related validity (concurrent vs. predictive), Construct validity Relationship between reliability and validity Relationship between reliability and validity

PSYC512: Research Methods Probability and Statistics Why are probability and statistics important? Why are probability and statistics important? Used to assess variability in data Used to assess variability in data Treatment Variance Treatment Variance Variability due to different levels of independent variable Variability due to different levels of independent variable Good variance that we want to maximize Good variance that we want to maximize Error Variance Error Variance Variability in data due to factors other than the treatment Variability in data due to factors other than the treatment Bad variance that we want to minimize Bad variance that we want to minimize Probability and Statistics are simply tools used to assess and compare these sources of variability Probability and Statistics are simply tools used to assess and compare these sources of variability

PSYC512: Research Methods Visualizing Variability: Distributions of Frequency and the Histogram Histograms: used to represent frequencies of data in different classes or categories Histograms: used to represent frequencies of data in different classes or categories BinFrequency

PSYC512: Research Methods Displaying Histograms: Stem and Leaf Plots Stem and Leaf plots are used to display histograms graphically (on their side) using only typed characters Stem and Leaf plots are used to display histograms graphically (on their side) using only typed characters StemLeaf(hypothetical histogram for IQ)

PSYC512: Research Methods Distributions of Probability Density Similar to frequency histogram except y-axis now represents probability density (mass) rather than frequency Similar to frequency histogram except y-axis now represents probability density (mass) rather than frequency Probability density = Frequency/N Probability density = Frequency/N

PSYC512: Research Methods Some Types of Distributions NormalGamma

PSYC512: Research Methods Measures of the Center of a Distribution Measures of center represent the general magnitude of scores in a distribution Measures of center represent the general magnitude of scores in a distribution Mode: most frequent score Mode: most frequent score Median: the middle score of an ordered distribution Median: the middle score of an ordered distribution Mean (average):where X is the data and Mean (average):where X is the data and N is the total number of observations

PSYC512: Research Methods Measures of the Spread of a Distribution Measures of spread are used to assess the consistency of scores in a distribution Measures of spread are used to assess the consistency of scores in a distribution Range = max score – min score Range = max score – min score Interquartile range = score(Q3) – score(Q1) Interquartile range = score(Q3) – score(Q1) Variance (  2 ) and standard deviation (  ) Variance (  2 ) and standard deviation (  ) where X is the data, m is the mean of the data, and N is the total number of observations

PSYC512: Research Methods More on Variance Standard Deviation (  ) = sqrt(variance) Standard Deviation (  ) = sqrt(variance) where X is the data, m is the mean of the data, and N is the total number of observations Why N instead of N-1? Populations vs. Samples Remembering how to compute variance Remembering how to compute variance “the mean of the squares – square of the means”

PSYC512: Research Methods Describing Distributions Parametrically: Statistical Moments Any distribution based on interval or ratio data can be summarized by its statistical moments Any distribution based on interval or ratio data can be summarized by its statistical moments First Moment: Mean—location of distribution on x-axis First Moment: Mean—location of distribution on x-axis Second Moment: Variance—dispersion of distribution Second Moment: Variance—dispersion of distribution Third Moment: Skewness—symmetry of distribution Third Moment: Skewness—symmetry of distribution Fourth Moment: Kurtosis—degree of “peakedness” Fourth Moment: Kurtosis—degree of “peakedness”

PSYC512: Research Methods Estimators Sample statistics estimate population parameters Sample statistics estimate population parameters Mean: M or vs.  Mean: M or vs.  Variance: s 2 vs.  2 Variance: s 2 vs.  2 Properties of Estimators Properties of Estimators Sufficiency: uses all information in sample (mean and variance are sufficient, mode and range are not) Sufficiency: uses all information in sample (mean and variance are sufficient, mode and range are not) Unbiasedness: expected value approaches real value with increased sampling Unbiasedness: expected value approaches real value with increased sampling Efficiency: tightness of cluster of sample statistics relative to the population parameter Efficiency: tightness of cluster of sample statistics relative to the population parameter Resistance: influence of outliers on sample statistic Resistance: influence of outliers on sample statistic

PSYC512: Research Methods Next Time… Topic: descriptive statistics, variables, sampling, and more on hypothesis testing Topic: descriptive statistics, variables, sampling, and more on hypothesis testing Be sure to: Be sure to: Read the assigned readings (Howell chapters 3-4) Read the assigned readings (Howell chapters 3-4) Continue searching and reading the scientific literature for your proposal Continue searching and reading the scientific literature for your proposal