CHOOSING A RESEARCH DESIGN CHAPTER 6 CHOOSING A RESEARCH DESIGN

DESIGNING A STUDY TO ANSWER A QUESTION The type of study you conduct depends on the type of question you are asking Research design – Specific methods and procedures used to answer a research question A set of rules indicating how to collect observations Types of research questions are categorized as exploratory, descriptive, or relational

DESIGNING A STUDY TO ANSWER A QUESTION

CATEGORIES OF RESEARCH DESIGN Distinguished by the level of control established in the design Control: Used to describe (a) the manipulation of a variable and (b) holding all other variables constan

Is experimental control a good thing?

How do we make out blurred images? Experiment 1 Experiment 2 Selected images from web Photoshop: “Gaussian blur” Data collection online, participation from home “vetted” images Matlab: extract spatial frequencies Data collection in-person, fixed distance from monitor (chin rest), calibrated light intensity, lighting in room, size of monitor Full vision screening

CATEGORIES OF RESEARCH DESIGN Non-experimental research design – Use of methods and procedures to make observations in which the behavior or event is observed “as is” or without an intervention from the researcher Strength: Can be used to make observations in settings that the behaviors and events being observed naturally operate Limitation: Lacks control needed to demonstrate cause and effect

CATEGORIES OF RESEARCH DESIGN

CATEGORIES OF RESEARCH DESIGN Experimental research design – Use of methods and procedures to make observations in which the researcher fully controls the conditions and experiences of participants. Used to make causal inferences. Required elements of control: Randomization Random sampling: Select a sample at random Random assignment: Assign to group at random Individual differences: Unique characteristics of participants in a sample that can differ from one participant to another Confound or confound variable: Unanticipated variable not accounted for in a research study that could be causing or associated with observed change in one or more measured variables (vs extraneous variable) Strength: Capable of demonstrating cause and effect Limitation: Behavior that occurs under controlled conditions may not be the same as behavior that occurs in a natural environment

DEMONSTRATING CAUSE IN AN EXPERIMENT Independent variable (IV) or factor: The variable that is manipulated. It is the “presumed cause” Level: Specific conditions or groups created by manipulating that factor Dependent variable (DV): Variable that is believed to change in the presence of the IV. It is the “presumed effect”

Demonstrating Cause in an Experiment Laboratory experiment – Takes place in a laboratory setting in which the researcher has greatest control over variables, regardless of whether it is made to look natural or not Field experiment – Takes place in an environment within which the behavior or event being observed would naturally operate

Implicit Association Test : http://implicit.harvard.edu

CATEGORIES OF RESEARCH DESIGN Quasi-experimental research design – Use of methods and procedures to make observations in a study that is structured similar to an experiment, but the conditions and experiences of participants lack some control (no random assignment) Quasi-independent variable: Any factor in which the levels of that factor are preexisting Examples: smoking and pregnancy, autism & vaccines, gender, mental health status Strength: Allows researchers to study factors related to the unique characteristics of participants Limitation: Cannot demonstrate cause and effect

INTERNAL AND EXTERNAL VALIDITY Internal validity – Extent to which a research design includes enough control of the conditions and experiences of participants that it can demonstrate a single unambiguous explanation for a manipulation, that is cause and effect External validity – Extent to which observations made in a study generalize beyond the specific manipulations or constraints in the study Constraint: Any aspect of the research design that can limit observations to the specific conditions or manipulations in a study

ETHICS IN FOCUS: BENEFICENCE AND RANDOM ASSIGNMENT Random assignment ensures that all participants have an equal chance of being assigned to a group and receiving whatever benefits and costs that are associated with that group May not be sufficient when one group has obviously greater benefits than another group Researchers often compensate the disadvantaged group, referred to as compensatory equalization of treatments This is provided in order to meet the ethical standard of beneficence, as required by the APA code of conduct Beneficence: Equal distribution of potential costs and benefits of participation

Threats to the Internal Validity of a Research Study Instrumentation and measurement – Measurement of the DV changes due to an error during the course of a research study Attrition or experimental mortality – Participant does not show up for a study at scheduled time or fails to complete the study Environmental factors – Characteristics of the study itself and the actions of the researchers Can only threaten internal validity when it varies systematically with the levels of an IV Test all of your participants in the same place if you can help it!

THREATS TO THE INTERNAL VALIDITY OF A RESEARCH STUDY (quasi-experimental designs) History effects – An unanticipated event that co-occurs with a treatment or manipulation in a study (Sesame Street and the Head Start Program)—only relevant to single-group pre-post Maturation – A participant’s physiological or psychological state changes over time during a study Regression toward the mean – Change or shift in a participant’s performance toward a level or score that is closer to or more typical of true potential or mean ability on some measure, after previously scoring unusually high or low on the same measure (often when selection is biased) Testing effects – The improved performance on a test or measure the second time it is taken due to the experience of taking a test

Threats to the External Validity of a Research Study Population validity – Extent to which results observed in a study will generalize to the population from which a sample was selected Homogeneous attrition: Rates of attrition are about the same in each group Ecological validity – Extent to which results observed in a study will generalize across settings or environments Temporal validity – Extent to which results observed in a study will generalize across time and at different points in time Outcome validity – Extent to which results observed in a study will generalize across different but related DVs

External Validity, Experimentation, and Realism Mundane realism – Extent to which a research setting physically resembles or looks like the natural or real-world environment being stimulated Experimental realism – Extent to which the psychological aspects of a research setting are meaningful or feel real to participants To enhance this, it is important that the manipulations are meaningful to participants Increasing mundane and experimental realism will increase the external validity of a research result