1. 研究設計 Research Design Emily Lin, PhD. (林永芬)
Department of Communication Disorders
University of Canterbury
Christchurch, New Zealand
Taiwan Academy of Physical Medicine and Rehabilitation Conference: Current Intervention for Children with Developmental Delay
Taoyuan, Taiwan
December 2, 2006

2. Outline Basic Concepts Experimental Research Group Design
Single-Subject Design
E. Lin

3. What is Research? “Research is the process of investigating
scientific questions.”
To satisfy the need to:
1. explain events
2. solve practical problems
3. demonstrate certain effects
legal, social, professional, and scientific considerations
(Hegde, 2003)
E. Lin

4. Research Process Phase I: Identify the research question
define the research problem
review literature; provide theoretical framework
identify target population
identify variables
state research rationale
clarify objectives
state specific purposes or hypotheses
Phase II: Design the study (i.e., design protocol, select a sample)
Phase III: Methods (i.e., collect data, reduce data)
Phase IV: Data Analysis (i.e., analyze data, provide interpretation)
Phase V: Communication (i.e., report findings, suggest future studies)
(Portney & Watkins, 2000)
E. Lin

5. Purpose of Research To determine the relationship between variables*
*Some basic terms:
1. constant (常數)
2. variable (變數):
-independent variable
-dependent variable
-extraneous variable
E. Lin

6. Types of Research e.g., e.g., Case study Experimental Developmental
Descriptive
Exploratory
Experimental
(Describe population)
(Find relationships)
(Cause and effect)
e.g.,
Case study
Developmental
research
Normative
Qualitative
Correlational
e.g.,
Experimental
(randomized
controlled)
Quasi-
experimental
Sequential
clinical trial
Single-subject
designs
(Portney & Watkins, 2000)
E. Lin

7. What is an experiment (實驗)?
Manipulation of variables:
Independent variables are manipulated through:
-administration of treatment, or
-deliberate operation imposing predetermined experimental conditions combined with classification (Quasi-experiment)
Random selection/assignment:
Obtain a representative sample
Establish equivalency between comparison groups
E. Lin

8. Selecting a Study Sample
Target population (reference population)
Accessible population (experimental population)
Subject selection
Participants (Study sample)
Non-participants
Group assignment
Experimental
group
Control group
(comparison groups)
(Portney & Watkins, 2000)
E. Lin

9. Sampling Strategy Probability sampling: Nonprobability sampling:
Simple random sampling
Systematic sampling
Stratified random sampling
Disproportional sampling
Cluster sampling
Nonprobability sampling:
Convenience sampling
Quota sampling
Purposive sampling
Snowball sampling
(Portney & Watkins, 2000)
E. Lin

10. Compensations for Lack of Random Sampling
Homogeneous groups
Matching
Control
Build in extraneous factors
Blocking
E. Lin

11. Validity (效度) Internal validity: External validity: generalization
the degree a cause-and-effect inference can be made based on the observed relationship between the variables
“Measure what is claimed to be measured”
External validity: generalization
“Generalize to other situations”
E. Lin

12. Threats to Internal Validity
Potential for confounding factors to interfere
with the relationship between the independent
and dependent variables:
e.g.,
History
Maturation
Mortality/attrition
Testing or test-practice effects
Statistical regression
Differential selection of subjects
Instrumentation
(Schiavetti & Metz, 1997)
E. Lin

13. Threats to External Validity
Factors limiting “the degree to which
internally valid results can be generalized”:
e.g.
Subject selection
Reactive or interactive effects of pretesting
Reactive arrangement (Hawthorne effect)
Multiple-treatment interference
(Schiavetti & Metz, 1997)
E. Lin

14. Measurement
Purpose: to provide a mechanism for achieving a degree of precision in the understanding of the characteristics of the object of interest
Key elements:
Construct
Rules
Evaluation of a measurement:
1. validity (效度): measuring what was intended
2. reliability (信度): yielding consistent results
E. Lin

15. Level of Measurement With absolute zero Equal intervals Ranking
Category
E. Lin

16. Test Validity
1. Face validity: appears to test what is supposed to test.
2. Content validity: consists of items that adequately sample the content that defines the variable being measured.
3. Criterion-related validity: yields outcomes that can be used as a substitute measure for an established gold standard criterion test.
Concurrent validity
Predictive validity
4. Construct validity: the degree the test measures an abstract construct
E. Lin

17. Threats to Test Validity
Length effect: e.g., fatigue, learning
Enabling behaviors required of the test taker
The representativeness of the norm
Bias
Reliability: e.g., test-retest, inter-judge
E. Lin

18. Reliability 1. Test-retest reliability:
Repeat the whole test or a portion of test
Conduct a parallel test
Split-half method (internal consistency)
2. Inter and intra-judge reliability:
Total reliability
Trial-by-trial (point-by-point) reliability
Occurrence reliability
Nonoccurrence reliability
E. Lin

19. Three Basic Measurements In Descriptive Statistics
Central tendency: the average (“center”) score
of a distribution; mean, median, mode
Variability: the dispersion of scores;
range, standard deviation (SD)
3. Relative position: a score’s position within
a distribution; percentile, z-score
E. Lin

20. Inferential Statistics
Sample Population
Known Unknown
*Difference between subjective inference and statistical inference:
Statistical inference requires objective criteria to make decisions.
Inferential statistics:
Decision-making process
To estimate population characteristics from sample data
Assumptions made about how well the sample represents
the larger population. The assumptions are based on
two concepts of statistical reasoning:
Probability
Sampling error
E. Lin

21. Hypothesis Testing Null hypothesis (statistical hypothesis; H0):
the group difference is due to sampling error
Alternative hypothesis (research/scientific hypothesis;
H1): the research hypothesis is correct
The purpose of posing a research hypothesis:
usually with the intention to reject the null hypothesis
E. Lin

22. Hypothesis Testing: Type I and II errors
(Real Situation )
H0 is true
H0 is false
Reject H0
Type I error
(a)
Correct decision
(1-b)
(Power of test)
(Decision)
Accept H0
Correct decision
(1-a)
Type II error
(b)
a : significance level
Power (1-b): the probability that a test will produce a significant difference at a given significance level
Type I error: The error that results when null hypothesis is falsely rejected.
Type II error: The error that results when null hypothesis is falsely accepted.
E. Lin

23. Hypothesis Testing
(Portney & Watkins, 2000)
E. Lin

24. Group Design One-shot case study One-group pretest-posttest design
Static-group comparisons
Pretest-posttest control group design
Posttest-only control group design
Solomon four-group design
Multigroup pretest-posttest design
Multigroup posttest-only design
E. Lin

25. Group Design -continued
Factorial designs
Single-group time-series design
Multiple-group time-series design
One-group single-treatment counter-balanced
design
Crossover design
Correlational analysis
E. Lin

26. Group Design: One-Way Design for Independent Groups10
S11 10
S21 10 S2 S3 S4 S5 S6 S7 S8 S9
S10
S12
S13
S14
S15
S16
S17
S18
S19
S20
S22
S23
S24
S25
S26
S27
S28
S29
S30
Analysis Method: One-way ANOVA
E. Lin

27. Group Design: One-way Within-Subjects Design10 10 10 S1 S2 S3 S4 S5 S6 S7 S8 S9
S10 S1 S2 S3 S4 S5 S6 S7 S8 S9
S10 S1 S2 S3 S4 S5 S6 S7 S8 S9
S10
Analysis Method: One-way Repeated Measures ANOVA
E. Lin

28. Two-way Factorial Design
Factors:
-Factor A (teaching method): 2 levels (oral vs. visual)
-Factor B (gender): 2 levels (female vs. male)
This is a two-by-two (2X2) design. A1 A2 S1 S2 S3 S4 S5 5
S11
S12
S13
S14
S15 5 B1 S6 S7 S8 S9
S10 5
S16
S17
S18
S19
S20 5 B2
E. Lin

29. Effect of Factor A (Teaching method)
Main Effect
Effect of Factor A (Teaching method) A1 A2 5 5 S1 S2 S3 S4 S5
S11
S12
S13
S14
S15 B1 5 5 S6 S7 S8 S9
S10
S16
S17
S18
S19
S20 B2
E. Lin

30. Effect of Factor B (Gender)
Main Effect
Effect of Factor B (Gender) A1 A2 5 5 S1 S2 S3 S4 S5
S11
S12
S13
S14
S15 B1 5 5 S6 S7 S8 S9
S10
S16
S17
S18
S19
S20 B2
E. Lin

31. Effect of Interaction between Factors A and B
Interaction Effect
Effect of Interaction between Factors A and B A1 A2 5 5 S1 S2 S3 S4 S5
S11
S12
S13
S14
S15 B1 5 5 S6 S7 S8 S9
S10
S16
S17
S18
S19
S20 B2
E. Lin

32. No interaction Interaction
E. Lin

33. Single-Subject Design
Synonym:
Single-case design
Single-system design
Time series experimentation
Definition: an experimental design involving the systematic collection of repeated measurements of a behavioural response over time, usually at frequent and regular intervals
E. Lin

34. Single-Subject Design: Length of Phases
Two choices:
Equal phase lengths: preset a short period time to minimize maturation, motivational changes over prolonged periods
Unequal phase lengths: extend baseline or intervention phases until stability is achieved
E. Lin

35. Single-Subject Design: Structure
Repeated measurement
Two design phases:
Baseline phase: period prior to treatment
Intervention phase: Period during treatment.
(Portney & Watkins, 2000)
E. Lin

36. Single-Subject Design: Assumption
Baseline data reflect the ongoing effects of background
variables, such as daily activities, other treatments, and
personal characteristics, on the target behaviours.
Therefore, when treatment is initiated, changes from
baseline to the intervention phase should be attributable to
intervention.
(Portney & Watkins, 2000)
E. Lin

37. Single-Subject Design: Length of Phases
Why not 1 or 2 sessions only?
Stability: a minimum of 3 to 4 data points in each phase (the greater the number of data points, the more obvious trends will become)
Why not 100 sessions?
Efficiency: also to avoid maturation, history, and other confounding factors.
(Portney & Watkins, 2000)
E. Lin

38. Single-Subject Design: Measuring the Target Behaviour
Frequency: the number of occurrence of a certain behaviour within a fixed time interval or a fixed number of trials
Duration: how long the target behaviour lasts
Magnitude: some form of instrumentation that provides a quantitative score
E. Lin

39. Single-Subject Design
Why use it?
Practical: fewer subjects are required.
Emphasis on individual performance: allowing for differentiation between subjects who respond favourably to treatment from those who are not affected by treatment.
E. Lin

40. Types of Single-Subject Design
Withdrawal design
A-B-A design
A-B-A-B design
Multiple treatment designs
A-B-C-B design
Interactive design: A-B-BC-B-BC
Alternating treatment design
Multiple baseline designs
E. Lin

41. Withdrawal Design: A-B-A Design
(Portney & Watkins, 2000)
E. Lin

42. Withdrawal Design: A-B-A-B Design
(Portney & Watkins, 2000)
E. Lin

43. Multiple Treatment Design:
A-B-C-B Design
(Portney & Watkins, 2000)
E. Lin

44. Interactive Design: A-B-BC-B-BC Design (Portney & Watkins, 2000)
E. Lin

45. Alternating Treatment Design
(Portney & Watkins, 2000)
E. Lin

46. Multiple Baseline Designs
Multiple baseline design across subjects
Multiple baseline design across conditions
Multiple baseline design across behaviours
(Portney & Watkins, 2000)
E. Lin

47. Split-Half
Method
(Portney & Watkins, 2000)
E. Lin

48. Split-Half Method (continued)
(Portney & Watkins, 2000)
E. Lin

49. Two Standard Deviation Method
(Portney & Watkins, 2000)
E. Lin

50. References
Hegde, M. N. (2003). Clinical Research in Communicative Disorders:
Principles and Strategies (3rd Edition). Austin, TX: Pro-ed.
Jadad, A. (1998). Randomised Controlled Trials. London: BMJ Books.
Portney, L. G. & Watkins, M. P. (2000). Foundations of Clinical
Research: Application to Practice (2nd Edition). Upper Saddle
River, NJ: Prentice Hall Health.
Schiavetti N., & Metz, D. (2002). Evaluating Research in Communicative
Disorders (4th Edition). Sydney: Allyn & Bacon.
E. Lin