9 Quantitative Research Designs

Learning Objectives Identify Criteria For Exploratory, Descriptive, And Explanatory Studies Define Experimental Research Differentiate Between Internal And External Validity In Experimental Designs Identify Six Threats To Internal Validity Identify Three Threats To External Validity

Learning Objectives Distinguish Among True Experimental, Quasi-Experimental, And Pre-Experimental Designs Describe Three True Experimental Designs Describe Two Quasi-experimental Designs Describe Two Pre-experimental Designs 3

Learning Objectives Discuss Four Types Of Nonexperimental Research Designs Recognize Two Types Of Settings In Which Research Is Conducted Identify Factors That Influence The Choice Of Research Designs Critique The Design Section Of Quantitative Studies 4

Learning Objective One Identify Criteria For Exploratory, Descriptive, And Explanatory Studies

Exploratory Studies Little known about phenomenon Flexible data collection approach Qualitative and quantitative Hypotheses not appropriate

Descriptive Studies Phenomena described Relationship between variables examined More information about variable(s) Test hypotheses

Explanatory Studies Explanations for relationships among phenomena Rigorous Experimental research Control over research conditions Manipulate one or more variables

Learning Objective Two Define Experimental Research 9

Experimental Research Cause + effect Manipulate and control independent variable Measure dependent variable

Problems With Experimental Research Casual relationships difficult to establish Avoid using word prove Controls difficult to apply to human beings

Learning Objective Three Differentiate Between Internal And External Validity In Experimental Designs 12

Internal Validity Degree to which changes in effect can be attributed to cause Threats Other factors that influence dependent variable Constitute rival explanations or competing hypotheses

External Validity Degree to which results can be generalized Questions to ask With what degree of confidence can findings be transferred to the entire population? Will these findings hold true with other groups and in other times and places?

Relationship Between Internal and External Validity As control for internal increases, external decreases As concern for external increases, internal may be affected Need to find balance

Learning Objective Four Identify Six Threats To Internal Validity 16

Six Threats to Internal Validity Selection bias History Maturation Testing Instrumentation change Mortality

Selection Bias Results due to subject differences Not due to independent variable manipulation Means to control Random group assignment

History Event other than the experimental treatment occurs during the course of study. Event influences dependent variable. Means to control Simultaneous control and comparison groups Random assignment of subjects to groups

Maturation Changes occur within subjects during study. Changes influence the study results. Means to control Simultaneous control and comparison groups

Testing Influence of pretest or baseline data knowledge on posttest score

Instrumentation Change Difference between pretest and posttest measurement Caused by change in accuracy rather than experimental treatment Means to control Judge training sessions Trial instrument runs to check for changes Continue to check instrument accuracy

Mortality Subject does not complete study. Attrition rate different between groups Means to control No research design to control Establish strong researcher-participant relationship

Learning Objective Five Identify Three Threats To External Validity 24

Major Threats to External Validity Hawthorne effect Experimenter effect Reactive effects of the pretest

Hawthorne Effect Participants’ responses influenced by knowing they are being observed Means to control Double-blind experiment

Experimenter Effect Experimental research Researcher characteristics or behaviors influence subject behaviors. Examples of influential characteristics Facial expression Clothing Age Gender Body build

Rosenthal Effect Nonexperimental research Interviewer characteristics or behaviors influence respondent’s answers.

Reactive Effects of the Pretest Subjects sensitized to experimental treatment because of pretest Examples of pretests Paper-and-pencil test Knowledge of baseline data

Difference Between Internal and External Pretest Threats Internal threat: pretest or baseline data knowledge cause of posttest results External threat: pretest or baseline data knowledge catalyst (indirect cause)

Learning Objective Six Distinguish Among True Experimental, Quasi-Experimental, And Pre-Experimental Designs 31

Experimental Research Designs True experimental Quasi-experimental Pre-experimental

True Experimental Design Great deal of control Internal validity threats minimized Causality inferred with confidence

True Experimental Design Criteria Researcher manipulates the experimental variable(s). One experimental group and one comparison group Subjects randomly assigned to groups

Quasi-Experimental Design No comparison group Subjects not randomly assigned to groups

Advantages and Disadvantages of Quasi-Experimental Design Real world more closely approximated Disadvantages Not as much control as true experimental design

Pre-Experimental Design Considered weak Researcher has little control.

Learning Objective Seven Describe Three True Experimental Designs 38

Experimental Designs Pretest-posttest control group design Posttest-only design Solomon four-group design

The Pretest-Posttest Control Group Design Most frequently used experimental design Criteria Subjects randomly assigned to groups Pretest given to both groups Experimental group receives experimental treatment. Comparison group receives routine treatment or no treatment. Posttest given to both groups

The Pretest-Posttest Control Group Design (cont’d) Advantages Controls for all internal validity threats Controls for initial differences by adjusting posttest scores statistically Disadvantages External threat of reactive effects of the pretest Can only be generalized to situations in which pretest is administered

The Posttest-Only Control Group Design Subjects randomly assigned to groups Experimental group receives the experimental treatment. Comparison group receives routine treatment or no treatment. Posttest given to both groups

Advantages of Posttest-Only Control Group Design Easier to carry out Eliminates reactive effects of the pretest on the posttest

The Solomon Four-Group Design All subjects are randomly assigned to one of four groups. Two groups—experimental group 1 and comparison group 1— pretested Two groups—experimental group 1 and experimental group 2—receive experimental treatment. Two groups—comparison group 1 and comparison group 2—receive routine treatment or no treatment.

Solomon Four-Group Design Posttest given to all four groups Advantages Minimizes threats to both internal and external validity Differences between groups can be associated with the experimental treatment. Disadvantages Requires a large sample Statistical analysis is complicated.

Learning Objective Eight Describe Two Quasi-Experimental Designs 46

Quasi-Experimental Designs Nonequivalent control group design Time-series design

The Nonequivalent Control Group Design Similar to pretest-posttest control group design No random assignment of subjects to groups

Internal Validity in Nonequivalent Control Group Design Threats to internal validity controlled History Testing Maturation Instrumentation change Threats to internal validity that remain Selection bias

Time-Series Design Periodic observations or measurements of subjects Experimental treatment administered between two of the observations

Time-Series Design Advantages Disadvantages Numerous observations or measurements of dependent variable Strengthens validity Disadvantages Internal validity threats: history and testing

Learning Objective Nine Describe Two Pre-Experimental Designs 52

Pre-Experimental Designs One-shot case study One-group pretest-posttest design

The One-Shot Case Study Single group exposed to experimental treatment Observed after treatment Weak design Doesn’t control for threats to internal validity

The One-Group Pretest-Posttest Design Comparison between a group of subjects before and after experimental treatment Threats to internal validity History Maturation Testing Instrumentation change

Learning Objective Ten Discuss Four Types Of Nonexperimental Research Designs 56

Nonexperimental Research Describes phenomena as it exists No manipulation No control of variables

Nonexperimental Research (cont’d) Survey research Correlational research Comparative research Methodological research

Survey Research Self-report data Phone Mail Internet Personal contacts

Strengths of Survey Research Advantages Provide accurate information on populations Use relatively small samples Collect large amounts of data quickly Minimal cost Disadvantages Responses may be unreliable.

Correlational Research Examines strength of relationship between two or more variables How is change in one variable associated with changes in another?

Correlational Coefficient Indicates the strength and direction of relationships Indicates positive (+) or negative (–) information +1.00: perfect positive correlation –1.00: perfect negative correlation 0.00: no correlation or relationship

Comparative Studies Examines differences between groups on dependent variable No manipulation of the independent variable

Two Types of Comparative Studies Retrospective or ex post facto Examine an effect Look back in time to determine cause Prospective Determination of a cause Looks forward in time to determine effect on subjects

Prospective Studies Advantages Prospective Experimental approach Manipulate independent variable Prospective Costly

Methodological Studies Research instruments and methods Development Testing Evaluation Seen with increasing frequency Develop stockpile of nursing-appropriate instruments

Secondary Analysis Studies Analyses data gathered in previous study Test new hypotheses Ask new research questions Cost effective Efficient

Learning Objective Eleven Recognize Two Types Of Settings In Which Research Is Conducted 68

Research Settings Laboratory studies Field studies

Laboratory Studies Conducted in specially created environments Research environment controlled by researcher

Field Studies Real-life setting Little control maintained by researcher Very appropriate for nursing research

Learning Objective Twelve Identify Factors That Influence The Choice Of Research Designs 72

Cause and Effect Versus Describing a Phenomenon Cause-and-effect research designs Experimental designs Describing a phenomenon research designs Nonexperimental designs

Level of Control Least control More control Most control Pre-experimental designs More control Quasi-experimental designs Most control True experimental designs

Other Determinant Factors Cost effective Secondary analysis studies Survey research Costly Prospective studies Ethical reasons Nonexperimental comparative studies

Learning Objective Thirteen Critique The Design Section Of Quantitative Studies 76

Critique of Quantitative Research Designs Read the entire research report carefully Determine if the appropriate design is used

Questions for Critique of the Design Section of Quantitative Studies Is the design clearly identified and described? Is the design appropriate to test the hypothesis or hypotheses or research question(s)? If an experimental design was used, was it appropriate?

Questions for Critique of the Design Section of Quantitative Studies If an experimental design was used, what means were used to control for threats to internal and external validity? Was the assignment of subjects to experimental and control groups clearly described? Does the research design allow the researcher to draw a cause-and-effect relationship between the variables?

Questions for Critique of the Design Section of Quantitative Studies If a nonexperimental design was used, would an experimental design have been more appropriate? If a nonexperimental design was used, what means were used to control for extraneous variables?