Presentation is loading. Please wait.

Presentation is loading. Please wait.

研究設計 Research Design Emily Lin, PhD. ( 林永芬 ) Department of Communication Disorders University of Canterbury Christchurch, New Zealand Taiwan Academy of.

Similar presentations


Presentation on theme: "研究設計 Research Design Emily Lin, PhD. ( 林永芬 ) Department of Communication Disorders University of Canterbury Christchurch, New Zealand Taiwan Academy of."— Presentation transcript:

1 研究設計 Research Design Emily Lin, PhD. ( 林永芬 ) Department of Communication Disorders University of Canterbury Christchurch, New Zealand Taiwan Academy of Physical Medicine and Rehabilitation Conference: Current Intervention for Children with Developmental Delay Taoyuan, Taiwan December 2, 2006

2 Outline Basic Concepts Experimental Research  Group Design  Single-Subject Design E. Lin

3 What is Research? “Research is the process of investigating scientific questions.” To satisfy the need to: 1. explain events 2. solve practical problems 3. demonstrate certain effects legal, social, professional, and scientific considerations (Hegde, 2003) E. Lin

4 Research Process Phase I: Phase I: Identify the research question define the research problem review literature; provide theoretical framework identify target population identify variables state research rationale clarify objectives state specific purposes or hypotheses Phase II: Design the study Phase II: Design the study (i.e., design protocol, select a sample) Phase III: Methods Phase III: Methods (i.e., collect data, reduce data) Phase IV: Data Analysis Phase IV: Data Analysis (i.e., analyze data, provide interpretation) Phase V: Communication Phase V: Communication (i.e., report findings, suggest future studies) E. Lin (Portney & Watkins, 2000)

5 Purpose of Research To determine the relationship between variables* *Some basic terms: 1. constant ( 常數 ) 2. variable ( 變數 ) : -independent variable -dependent variable -extraneous variable E. Lin

6 e.g., Experimental (randomized controlled) Quasi- experimental Sequential clinical trial Single-subject designs e.g., Case study Developmental research Normative research Qualitative research Correlational research Descriptive (Describe population) Exploratory (Find relationships) Experimental (Cause and effect) Types of Research E. Lin (Portney & Watkins, 2000)

7 What is an experiment ( 實驗 ) ? 1. 1. Manipulation of variables: Independent variables are manipulated through: -administration of treatment, or -deliberate operation imposing predetermined experimental conditions combined with classification (Quasi-experiment) 2. 2. Random selection/assignment: 1. 1. Obtain a representative sample 2. 2. Establish equivalency between comparison groups E. Lin

8 Selecting a Study Sample E. Lin Participants (Study sample) Non-participants Group assignment Experimental group Control group (comparison groups) Target population (reference population) (Portney & Watkins, 2000) Accessible population (experimental population) Subject selection

9 Sampling Strategy : Probability sampling: Simple random sampling Systematic sampling Stratified random sampling Disproportional sampling Cluster sampling : Nonprobability sampling: Convenience sampling Quota sampling Purposive sampling Snowball sampling E. Lin (Portney & Watkins, 2000)

10 Compensations for Lack of Random Sampling Homogeneous groups  Matching  Control Build in extraneous factors  Blocking E. Lin

11 Validity ( 效度 ) Internal validity: Internal validity: the degree a cause-and-effect inference can be made based on the observed relationship between the variables “Measure what is claimed to be measured” External validity: External validity: generalization “Generalize to other situations” E. Lin

12 Threats to Internal Validity Potential for confounding factors to interfere with the relationship between the independent and dependent variables: e.g., History History Maturation Maturation Mortality/attrition Mortality/attrition Testing or test-practice effects Testing or test-practice effects Statistical regression Statistical regression Differential selection of subjects Differential selection of subjects Instrumentation Instrumentation E. Lin (Schiavetti & Metz, 1997)

13 Threats to External Validity Factors limiting “the degree to which internally valid results can be generalized”: e.g. Subject selection Subject selection Reactive or interactive effects of pretesting Reactive or interactive effects of pretesting Reactive arrangement (Hawthorne effect) Reactive arrangement (Hawthorne effect) Multiple-treatment interference Multiple-treatment interference E. Lin (Schiavetti & Metz, 1997)

14 Measurement Purpose: to provide a mechanism for achieving a degree of precision in the understanding of the characteristics of the object of interest Key elements: Construct Rules Evaluation of a measurement: 1. validity ( 效度 ): measuring what was intended 2. reliability ( 信度 ): yielding consistent results E. Lin

15 Level of Measurement E. Lin Category Ranking Equal intervals With absolute zero

16 Test Validity 1. Face validity: 1. Face validity: appears to test what is supposed to test. 2. Content validity: 2. Content validity: consists of items that adequately sample the content that defines the variable being measured. 3. Criterion-related validity: 3. Criterion-related validity: yields outcomes that can be used as a substitute measure for an established gold standard criterion test. a. a. Concurrent validity b. b. Predictive validity 4. Construct validity: 4. Construct validity: the degree the test measures an abstract construct E. Lin

17 Threats to Test Validity Length effect: e.g., fatigue, learning Enabling behaviors required of the test taker The representativeness of the norm Bias Reliability: e.g., test-retest, inter-judge E. Lin

18 Reliability Repeat the whole test or a portion of test Conduct a parallel test Split-half method (internal consistency) 1. Test-retest reliability: Total reliability Trial-by-trial (point-by-point) reliability Occurrence reliability Nonoccurrence reliability 2. Inter and intra-judge reliability: E. Lin

19 Three Basic Measurements In Descriptive Statistics 1.Central tendency: the average (“center”) score of a distribution; mean, median, mode 2.Variability: the dispersion of scores; range, standard deviation (SD) 3. Relative position: a score’s position within a distribution; percentile, z-score E. Lin

20 Inferential Statistics Sample Population Known Unknown Inferential statistics: Decision-making process To estimate population characteristics from sample data Assumptions made about how well the sample represents the larger population. The assumptions are based on two concepts of statistical reasoning:  Probability  Sampling error *Difference between subjective inference and statistical inference: Statistical inference requires objective criteria to make decisions. E. Lin

21 Hypothesis Testing Null hypothesis (statistical hypothesis; H 0 ): the group difference is due to sampling error Alternative hypothesis (research/scientific hypothesis; H 1 ): the research hypothesis is correct The purpose of posing a research hypothesis: usually with the intention to reject the null hypothesis E. Lin

22 Hypothesis Testing: Type I and II errors Type II error (  ) Correct decision (1-  Accept H 0 Correct decision (1-  (Power of test) Type I error (  ) Reject H 0 (Decision) (Decision) H 0 is false H 0 is true (Real Situation ) (Real Situation )  : significance level Power (1-  ): the probability that a test will produce a significant difference at a given significance level Type I error: The error that results when null hypothesis is falsely rejected. Type II error: The error that results when null hypothesis is falsely accepted. E. Lin

23 Hypothesis Testing E. Lin (Portney & Watkins, 2000)

24 Group Design One-shot case study One-group pretest-posttest design Static-group comparisons Pretest-posttest control group design Posttest-only control group design Solomon four-group design Multigroup pretest-posttest design Multigroup posttest-only design E. Lin

25 Group Design -continued Factorial designs Single-group time-series design Multiple-group time-series design One-group single-treatment counter-balanced design Crossover design Correlational analysis E. Lin

26 Group Design: One-Way Design for Independent Groups S 22 S 22 S 23 S 23 S 24 S 24 S 25 S 25 S 26 S 26 S 27 S 27 S 28 S 28 S 29 S 29 S 30 S 30 S 12 S 12 S 13 S 13 S 14 S 14 S 15 S 15 S 16 S 16 S 17 S 17 S 18 S 18 S 19 S 19 S 20 S 20 S 2 S 2 S 3 S 3 S 4 S 4 S 5 S 5 S 6 S 6 S 7 S 7 S 8 S 8 S 9 S 9 S 10 S 10 10 S 21 S 2110 S 11 S 1110 S 1 S 1 A 3 A 3 A 2 A 2 A 1 A 1 Analysis Method: One-way ANOVA E. Lin

27 Group Design: One-way Within-Subjects Design S 1 S 1 S 2 S 2 S 3 S 3 S 4 S 4 S 5 S 5 S 6 S 6 S 7 S 7 S 8 S 8 S 9 S 9 S 10 S 10 S 1 S 1 S 2 S 2 S 3 S 3 S 4 S 4 S 5 S 5 S 6 S 6 S 7 S 7 S 8 S 8 S 9 S 9 S 10 S 10 S 1 S 1 S 2 S 2 S 3 S 3 S 4 S 4 S 5 S 5 S 6 S 6 S 7 S 7 S 8 S 8 S 9 S 9 S 10 S 10 101010 A 3 A 3 A 2 A 2 A1A1A1A1 Analysis Method: One-way Repeated Measures ANOVA E. Lin

28 Two-way Factorial Design Factors: -Factor A (teaching method): 2 levels (oral vs. visual) -Factor B (gender): 2 levels (female vs. male) This is a two-by-two (2X2) design. S 16 S 16 S 17 S 17 S 18 S 18 S 19 S 19 S 20 S 20 S 6 S 6 S 7 S 7 S 8 S 8 S 9 S 9 S 10 S 10 B2 B2 B2 B2 55 S 11 S 11 S 12 S 12 S 13 S 13 S 14 S 14 S 15 S 15 S 1 S 1 S 2 S 2 S 3 S 3 S 4 S 4 S 5 S 5 B1 B1 B1 B1 55 A 2 A 2 A1A1A1A1 E. Lin

29 S 16 S 16 S 17 S 17 S 18 S 18 S 19 S 19 S 20 S 20 S 6 S 6 S 7 S 7 S 8 S 8 S 9 S 9 S 10 S 10 B2 B2 B2 B2 55 S 11 S 11 S 12 S 12 S 13 S 13 S 14 S 14 S 15 S 15 S 1 S 1 S 2 S 2 S 3 S 3 S 4 S 4 S 5 S 5 B1 B1 B1 B1 5 5 A 2 A 2 A1A1A1A1 Main Effect Effect of Factor A (Teaching method) E. Lin

30 S 16 S 16 S 17 S 17 S 18 S 18 S 19 S 19 S 20 S 20 S 6 S 6 S 7 S 7 S 8 S 8 S 9 S 9 S 10 S 10 B2 B2 B2 B2 55 S 11 S 11 S 12 S 12 S 13 S 13 S 14 S 14 S 15 S 15 S 1 S 1 S 2 S 2 S 3 S 3 S 4 S 4 S 5 S 5 B1 B1 B1 B1 5 5 A 2 A 2 A1A1A1A1 Main Effect Effect of Factor B (Gender) E. Lin

31 S 16 S 16 S 17 S 17 S 18 S 18 S 19 S 19 S 20 S 20 S 6 S 6 S 7 S 7 S 8 S 8 S 9 S 9 S 10 S 10 B2 B2 B2 B2 5 5 S 11 S 11 S 12 S 12 S 13 S 13 S 14 S 14 S 15 S 15 S 1 S 1 S 2 S 2 S 3 S 3 S 4 S 4 S 5 S 5 B1 B1 B1 B1 5 5 A 2 A 2 A1A1A1A1 Interaction Effect Effect of Interaction between Factors A and B E. Lin

32 No interactionInteraction E. Lin

33 Single-Subject Design Synonym:  Single-case design  Single-system design  Time series experimentation Definition: an experimental design involving the systematic collection of repeated measurements of a behavioural response over time, usually at frequent and regular intervals E. Lin

34 Single-Subject Design: Length of Phases Two choices:  Equal phase lengths: preset a short period time to minimize maturation, motivational changes over prolonged periods  Unequal phase lengths: extend baseline or intervention phases until stability is achieved E. Lin

35 Single-Subject Design: Structure  Repeated measurement  Two design phases: 1.Baseline phase: period prior to treatment 2.Intervention phase: Period during treatment. E. Lin (Portney & Watkins, 2000)

36 Single-Subject Design: Assumption Baseline data reflect the ongoing effects of background variables, such as daily activities, other treatments, and personal characteristics, on the target behaviours. Therefore, when treatment is initiated, changes from baseline to the intervention phase should be attributable to intervention. E. Lin (Portney & Watkins, 2000)

37 Single-Subject Design: Length of Phases Why not 1 or 2 sessions only?  Stability: a minimum of 3 to 4 data points in each phase (the greater the number of data points, the more obvious trends will become) Why not 100 sessions?  Efficiency: also to avoid maturation, history, and other confounding factors. E. Lin (Portney & Watkins, 2000)

38 Single-Subject Design: Measuring the Target Behaviour  Frequency: the number of occurrence of a certain behaviour within a fixed time interval or a fixed number of trials  Duration: how long the target behaviour lasts  Magnitude: some form of instrumentation that provides a quantitative score E. Lin

39 Single-Subject Design Why use it?  Practical: fewer subjects are required.  Emphasis on individual performance: allowing for differentiation between subjects who respond favourably to treatment from those who are not affected by treatment. E. Lin

40 Types of Single-Subject Design  Withdrawal design A-B-A design A-B-A-B design  Multiple treatment designs A-B-C-B design Interactive design: A-B-BC-B-BC Alternating treatment design  Multiple baseline designs E. Lin

41 Withdrawal Design: A-B-A Design E. Lin (Portney & Watkins, 2000)

42 Withdrawal Design: A-B-A-B Design E. Lin (Portney & Watkins, 2000)

43 Multiple Treatment Design: A-B-C-B Design E. Lin (Portney & Watkins, 2000)

44 Interactive Design: A-B-BC-B-BC Design E. Lin (Portney & Watkins, 2000)

45 Alternating Treatment Design E. Lin (Portney & Watkins, 2000)

46 Multiple Baseline Designs 1.Multiple baseline design across subjects 2.Multiple baseline design across conditions 3.Multiple baseline design across behaviours E. Lin (Portney & Watkins, 2000)

47 Split-Half Method E. Lin (Portney & Watkins, 2000)

48 Split-Half Method (continued) E. Lin (Portney & Watkins, 2000)

49 Two Standard Deviation Method E. Lin (Portney & Watkins, 2000)

50 References Hegde, M. N. (2003). Clinical Research in Communicative Disorders: Principles and Strategies (3 rd Edition). Austin, TX: Pro-ed. Jadad, A. (1998). Randomised Controlled Trials. London: BMJ Books. Portney, L. G. & Watkins, M. P. (2000). Foundations of Clinical Research: Application to Practice (2 nd Edition). Upper Saddle River, NJ: Prentice Hall Health. Schiavetti N., & Metz, D. (2002). Evaluating Research in Communicative Disorders (4 th Edition). Sydney: Allyn & Bacon. E. Lin


Download ppt "研究設計 Research Design Emily Lin, PhD. ( 林永芬 ) Department of Communication Disorders University of Canterbury Christchurch, New Zealand Taiwan Academy of."

Similar presentations


Ads by Google