Presentation is loading. Please wait.

Presentation is loading. Please wait.

Module 6 Darcy Freedman, MPH, PhD June 18, 2014. Assumptions in Scientific Research Nature is orderly and regular To some extent, events are consistent.

Similar presentations


Presentation on theme: "Module 6 Darcy Freedman, MPH, PhD June 18, 2014. Assumptions in Scientific Research Nature is orderly and regular To some extent, events are consistent."— Presentation transcript:

1 Module 6 Darcy Freedman, MPH, PhD June 18, 2014

2 Assumptions in Scientific Research Nature is orderly and regular To some extent, events are consistent and predictable Events or conditions have one or more causes that can be discovered This enables establishing cause and effect relationships

3 The Scientific Method The scientific method has been defined as a systematic, empirical, controlled and critical examination of hypothetical propositions about the association among natural phenomena. 1,2 1.Kerlinger FN. Foundations of Behavioral Research. New York: Holt, Rinehart & Winston, 1973. 2.Portney LG, Watkins MP. 2000. Foundations of Clinical Research: Applications to Practice. 2 nd Ed. Upper Saddle River, NJ: Prentice Hall Health.

4 Properties of scientific method Systematic Use of orderly procedures to ensure reliability Logical sequence is used from problem identification, through data collection, analysis, & interpretation Empirical Documentation of objective data through direct observation (or other systematic methods) Findings are grounded in the objective observation of phenomena rather than the personal bias or subjective belief of the researcher Control In order to understand how one phenomenon relates to another, factors are controlled that are not directly related to the variables in question Investigators have confidence in their research outcomes to the extent that they control extraneous influences

5 Limitations Science is imperfect, especially when it is applied to human behavior and performance

6 Limitations Science is imperfect, especially when it is applied to human behavior and performance Sources of uncertainty: Complexity and variability within nature The unique psychosocial and physiological capacities of individuals

7 Limitations Science is imperfect, especially when it is applied to human behavior and performance Sources of uncertainty: Complexity and variability within nature The unique psychosocial and physiological capacities of individuals Social science researchers must be acutely aware of extraneous influences in order to interpret findings in a meaningful way

8 Types of Research Descriptive Case study Cross-sectional study Qualitative study Exploratory Cohort study Case control study Experimental True experimental designs Quasi-experimental designs

9 Descriptive Research Descriptive: investigator attempts to describe a group of individuals on a set of variables or characteristics. Enables classification and understanding Methods: survey research, case study, qualitative, developmental (natural history of something, patterns of growth and change), normative, evaluation

10 Store Type (33 stores) Quality Composite Score (mean) Example Convenience Stores (70%) -0.74 Local Markets (24%) -0.38 Supermarkets (6%)6.5 Composite score = sum of scores for access to fresh fruit, fresh vegetables, lean meats, low-fat milk, tobacco products, alcohol. Chronbach’s alpha =.76 Source: Freedman & Bell, 2009

11 Exploratory Research Investigator examines a phenomenon of interest and explores its dimensions, including how it relates to other factors. Proven relationships between the phenomenon and other factors can lead to predictive models Correlational studies, cohort and case control, secondary analysis, historical research

12 Freedman, Blake, & Liese, 2013

13 Figure 2. Simplified Path Analytic Model 1 of Environmental Influence on FV Intake Source: Liese et al., 2013. *p<.05

14 Freedman et al., under review

15 Experimental Research Provides a basis for comparing 2 or more conditions Controls or accounts for the effects of extraneous factors, providing the highest degree of confidence in the validity of outcomes Enables the researcher to draw meaningful conclusions about observed differences Randomized controlled trials, single subject designs, sequential clinical trials, evaluation research, quasi- experimental research, meta-analysis

16 Individual-level Change in Fruit and Vegetable Consumption Design: Longitudinal; no comparison group Sample: 45 diabetic patients at FQHC Intervention: FQHC-based farmers’ market + financial incentive (up to $50) Outcome measure: F/V consumption measured with NCI screener Results: Dose-response relationship between improvement in F/V consumption and use of market Improvers more likely to rely on financial incentive to purchase foods at market Source: Freedman et al., 2013

17 Continuum of Research DescriptiveExploratoryExperimental Describe populationsFind relationshipsCause and Effect Case StudyExperimental randomized controlled trial (RCT) Quasi-experimental designs ----------------------------------------------Survey research-------------------------------- ----------------------Qualitative research----------------Sequential clinical trial Correlational research Single subject designs Evaluation research ------------------------Secondary analysis-----------------Meta-analysis -----Cohort/Case-Control Study----- Historical research Based on: Portney LG, Watkins MP. 2000. Foundations of Clinical Research: Applications to Practice. 2 nd Ed. Upper Saddle River, NJ: Prentice Hall Health., p. 13.

18 Community-engaged research Philosophy versus method Who are the “knowers” of phenomenon? Participatory processes during some or all stages of research Knowledge for action/change Can be used with any research approach Example: Community Visions Photovoice Project http://www.youtube.com/watch?v=95IMZlKLs2chttp://www.youtube.com/watch?v=95IMZlKLs2c (~9 min)

19 Quantitative/Qualitative Quantitative research involves measurement of outcomes using numerical data under standardized conditions May be used along the continuum of research Qualitative research is concerned with narrative information under less structured conditions that often takes the research context into account Descriptive and exploratory research Purposes: describing conditions, exploring associations, formulating theory, generating hypotheses

20 Choosing evaluation methods 20

21 Descriptive Research Case study Cross-sectional study Qualitative study

22 Case Study Design Often a description of a individual case’s condition or response to an intervention can focus on a group, institution, school, community, family, etc. data may be qualitative, quantitative, or both Case series: observations of several similar cases are reported

23 Case Study Example In 1848, young railroad worker, Phineas Gage, was forcing gun powder into a rock with a long iron rod when the gun powder exploded. The iron rod shot through his cheek and out the top of his head, resulting in substantial damage to the frontal lobe of his brain. Incredibly, he did not appear to be seriously injured. His memory and mental abilities were intact, and he could speak and work. However, his personality was markedly changed. Before the accident, he had been a kind and friendly person, but afterward he became ill-tempered and dishonest. Phineas Gage’s injury served as a case study for the effects of frontal lobe damage. He did not lose a specific mental ability, such as the ability to speak or follow directions. However, his personality and moral sense were altered. It is now known that parts of the cortex (called the association areas) are involved in general mental processes, and damage to those areas can greatly change a person’s personality.

24 Case Study Design Strengths Enables understanding of the totality of an individual’s (or organization, community) experience The in-depth examination of a situation or ‘case’ can lead to discovery of relationships that were not obvious before Useful for generating new hypotheses or for describing new phenomena Weaknesses No control group Prone to selection bias and confounding The interaction of environmental and personal characteristics make it weak in internal validity Limited generalizability

25 Cross-sectional Study Researcher studies a stratified group of subjects at one point in time Draws conclusions by comparing the characteristics of the stratified groups Well-suited to describing variables and their distribution patterns Can be used for examining associations; determination of which variables are predictors and which are outcomes depends on the hypothesis eg. Does lead paint ingestion cause hyperactivity or does hyperactivity lead to lead paint ingestion?

26 Cross-sectional Study Example: What is the prevalence of chlamydia in women age 18-35 in Cleveland, and is it associated with the use of oral contraceptives? Select a sample of 100 women attending an STD clinic in the city of Cleveland Measure the predictor and outcome variables by taking a history of oral contraceptive use and sending a cervical swab to the lab for chlamydia culture A questionnaire may be used to gather information abut oral contraceptive history

27 Cross-sectional Study Strengths Fast and inexpensive No loss to follow-up (no follow-up) Ideal for studying prevalence Convenient for examining potential networks of causal links e.g., in analysis, examine age as a predictor of oral contraceptive use, and then examine oral contraceptive use as a predictor for chlamydia infection Weaknesses: Difficult to establish a causal relationship from data collected in a cross-sectional time-frame (Lack of a temporal relationship between predictor variables and outcome variables - Does not establish sequence of events) Not practical for studying rare phenomena

28 Qualitative Study Seeks to describe how individuals perceive their own experiences within a social context Emphasizes in-depth, nuanced understanding of human experience and interactions Methods include in-depth interviews, direct observations, examining documents, focus groups Data are often participants’ own words and narrative summaries of observed behavior

29 Qualitative Study Example A researcher wants to understand how provision of healthcare to undocumented persons affects the people and institutions involved In 3 communities, information is gathered from undocumented patients, FQHC primary care clinicians, specialists, and hospital administrators Methods: in-depth interviews, key informant interviews, participant observations, case studies, focus groups

30 Qualitative Study Strengths Data based on the participants’ own categories of meaning Useful for studying a limited number of cases in depth or describing complex phenomena Provides understanding and description of people’s personal experiences of phenomena Can describe in rich detail phenomena as they are embedded in local contexts The researcher can study dynamic processes (i.e., document sequential patterns/change) Weaknesses Knowledge produced might not generalize to other people or other settings It is difficult to make quantitative predictions It might have lower credibility with some administrators and commissioners of programs Takes more time to collect and analyze the data when compared to quantitative research The results are more easily influenced by the researcher’s personal biases and idiosyncrasies

31 Exploratory Research Cohort study Case control study

32 Cohort Study A group of individuals who do not yet have the outcome of interest are followed together over time to see who develops the condition Participants are interviewed or observed to determine the presence or absence of certain exposures, risks, or characteristics May be simply descriptive May identify risk by comparing the incidence of specific outcomes in exposed and not exposed participants

33 Cohort Study Example To determine whether exercise protects against coronary heart disease (CHD). 1. Assemble the cohort: 16,936 Harvard alumni were enrolled 2. Measure predictor variables: Administer a questionnaire about activity and other potential risk factors, collected data from college records 3. 10 years later, sent a follow-up questionnaire about CHD and collected data about CHD from death certificates

34 Cohort Study Strengths Powerful strategy for defining incidence and investigating potential causes of an outcome before it occurs Time sequence strengthens inference that the factor may cause the outcome Weaknesses Expensive – many subjects must be studied to observe outcome of interest Potential confounders: eg, cigarette smoking might confound the association between exercise and CHD

35 Case-Control Study Generally retrospective Identify groups with or without the condition Look backward in time to find differences in predictor variables that may explain why the cases got the condition and the controls did not Assumption is that differences in exposure histories should explain why the cases have the condition Data collection via direct interview, mailed questionnaire, chart review

36 Case-Control Study Strengths Useful for studying rare conditions Short duration & relatively inexpensive High yield of information from relatively few participants Useful for generating hypotheses Weaknesses Increased susceptibility to bias: Separate sampling of cases and controls Retrospective measurement of predictor variables No way to estimate the excess risk of exposure Only one outcome can be studied

37 Case-Control Study Example Purpose: To determine whether there is an association between the use of aspirin and the development of Reye’s syndrome in children. 1. Draw the sample of cases – 30 patients who have had Reye’s syndrome 2. Draw the sample of controls – 60 patients from the much larger population who have had minor viral illnesses without Reye’s syndrome 3. Measure the predictor variable: ask patients in both groups about their use of aspirin

38 Experimental Research True experimental designs Quasi-experimental designs

39 Efficacy vs. Effectiveness Efficacy: the benefit of an intervention compared to a control or standard program under controlled, randomized conditions Randomized controlled trial (RCT) design often used Effectiveness: the benefit of an intervention under less controlled ‘real world’ conditions Quasi-experimental design often used

40 1. One group posttest only design PT 2 P = Program or intervention T 2 = Posttest Types of designs 40

41 2. Before and After Design One group pretest-post-test design T 1 P T 2 T 1 = Pretest (treatment group) T 2 = Posttest (treatment group) P = Program or intervention Types of designs 41

42 How much of the effect is due to the program? Time ( X ) Desired Outcome ( Y ) T C Gross Effect PrePost Net Effect 42

43 2. Comparison Group Design T 1 PT 2 C 1 C 2 T1 = Pretest (treatment group) T2 = Posttest (treatment group) P = Program or intervention C1 = Pretest (comparison group) C2 = Posttest (comparison group) Types of designs 43

44 Experimental Design True experimental design: Subjects are randomly assigned to at least 2 comparison groups Purpose is to compare 2 or more groups that are formed by random assignment The groups differ solely on the basis of what occurs between measurements (ie, intervention) Changes from pretest to posttest can be reasonably attributed to the intervention Most basic is the pretest-posttest control group design (randomized controlled trial, RCT)

45 Experimental Design Example: Researchers conducted an RCT to study the effect of progressive resistance exercises in depressed elders. They studied 35 volunteers who had depression. Participants were randomly assigned to an exercise group, which met three times per week for 10 weeks, or a control group which met 2 times per week for an interactive health education program. The outcome variables were: level of depression, functional status, and quality of life, using standardized instruments. Pretest and posttest measures were taken for both groups and differences were compared.

46 Experimental Design Strengths Controls the influence of confounding variables, providing more conclusive answers Randomization eliminates bias due to pre-randomization confounding variables Blinding the interventions eliminates bias due to unintended interventions Weaknesses Costly in time and money Many research questions are not suitable for experimental designs Usually reserved for more mature research questions that have already been examined by descriptive studies Experiments tend to restrict the scope and narrow the study question

47 Quasi-Experimental Design Quasi-Experimental designs do not use randomized assignments for comparisons

48 Quasi-Experimental Design Example: A study was designed to examine the effect of electrical stimulation on passive range of motion of wrist extension in 16 patients who suffered a stroke. Outcomes: effects of treatment on sensation, range of motion, & hand strength. Patients were given pretest and posttest measurements before and after a 4-week intervention program. Note: No randomization, and no comparison group

49 Quasi-Experimental Design Strengths Q-E designs are a reasonable alternative to RCT Useful where pre-selection and randomization of groups is difficult Saves time and resources vs. experimental designs Weaknesses Nonequivalent groups may differ in many ways -- in addition to the differences between treatment conditions, introducing bias Non-blinding allows the possibility of unintended interventions; blinding can be used in some Q-E studies Must document participant characteristics extensively Potential biases of the sample must be acknowledged when reporting findings Causal inferences are weakened by the potential for biases vs. experimental designs

50 Compared to what? Post-test only Pre & Post test Nonequivalent comparison group Quasi-experiment (matched groups, regression discontinuity) Randomized experiment Low Ability to Attribute Effect High Ability to Attribute Effect  Between groups  Randomly composed  Naturally occurring (waitlist, other programs)  National norms/standards  Over time  Pre to post  Longitudinal 50

51 51

52 Major sources Portney LG, Watkins MP. 2000. Foundations of Clinical Research: applications to practice. 2 nd Ed. Upper Saddle River, NJ: Prentice Hall Health Hulley SB, Cummings SR. 1998. Designing Clinical Research: an epidemiologic approach. Baltimore, MD: Williams and Wilkins Cook TD, Campbell DT. 1979. Quasi-Experimentation: design & analysis issues for field settings. Boston, MA: Houghton Mifflin Company


Download ppt "Module 6 Darcy Freedman, MPH, PhD June 18, 2014. Assumptions in Scientific Research Nature is orderly and regular To some extent, events are consistent."

Similar presentations


Ads by Google