1. Expanding Application of Clinical Significance for Outcome Measures: BAI and R-DAS Zack Hamingson, MA, Matt Multach, Jermaine Dictado, Kenichi Shimokawa, PhD
The Family Institute, Northwestern University
Abstract
Sample
Results
Defining clinically significant change for clients’ psychological functioning has become increasingly important in the context of evidence based practice and managed care settings.
The current research addresses issues and improves upon the Jacobson & Truax method (JT method; 1991), a popular model for defining such change, by applying a normative comparison and meta-analytic approach instead of using vaguely defined functional and dysfunctional populations.
We used general and clinical samples to define clinical cut off scores and to determine the Reliable Change Index (RCI) for the Beck Anxiety Inventory (BAI) and the Revised Dyadic Adjustment Scale (R-DAS).
General community: No exclusion criteria applied to sample
Clinical: BAI- Diagnosis or clinical determination of a principal DSM III or IV diagnosis of one or more anxiety disorders
Functional: Individuals in the sample were screened out for clinical symptoms
Measure
Sample Types
RCI
Cutoff Score
D from general mean
BAI
General vs. Clinical
4.3
8.37a
.51
Functional vs. Clinical
6.6
12.36b
.02
R-DAS
9.0
47.56a
.46
Methods
Initial online search of existing literature yielded a pool of 3567 (BAI) and 127 (R-DAS) articles, 232 (BAI) and 10 (R-DAS) were retained.
Clinical cut off scores and RCI scores (Jacobson & Truax, 1991) were meta-analytically derived for various general population and clinical samples using the random effects model.
To test the distinctness of each normative group, for each pair of groups a t test was performed and a d effect size calculated. In addition, a d statistic was calculated for each clinical sample to its corresponding general community and functional samples to evaluate cut off scores in relation to general population.
The RCI was calculated for each pair of groups. Clinical cut off scores based on the point of overlap between two distributions in comparison was also calculated.
Discussion
Use of general population and functional samples as a reference for clinical cutoff scores resulted in different clinical cut off scores.
Cutoff based on the clinical and functional samples as in the JT method resulted in a cut off score that was skewed to the healthy direction of the distribution, resulting in almost half the general population classified as being in the clinical range.
Cutoff based on the normative comparison provides clinical cut off in terms of empirically based distance from the general population mean, which makes more intuitive sense.
Introduction
Results
The JT method calculates the RCI and clinical cut off scores.
The RCI quantifies the amount of change required for the change to be greater than the instruments’ standard error of measurement.
Cutoff b
(JT cutoff)
Cutoff a
Seggar, Lambert, and Hansen (2002) conducted a meta-analytic study of the literature to establish cut off scores for the Beck Depression Inventory (BDI; Beck, Ward, Mendelson, Mock, & Erbaugh, 1961; Beck, Rush, Shaw, & Emery, 1979).
References
Beck, A. T., & Beamesderfer, A. (1974). Assessment of depression: The Depression Inventory. In P. Pichot (Ed.), Psychological measurements in psychopharmacology: Modern problems in pharmacopsychiatry (Vol. 7, pp ). Basel, Switzerland: Karger.
Beck, A. T., Epstein, N., Brown, G., & Steer, R. A. (1988). An inventory for measuring clinical anxiety: Psychometric properties. Journal of Consulting and Clinical Psychology, 56, Beck, A. T., Ward, C. H., Mendelson, M., Mock, J., & Erbaugh, J. (1961). An inventory for measuring depression. Archives of General Psychiatry, 4,
Busby, D. M., Christensen, C., Crane, D.R. & Larson, J.H. (1995). A revision of the Dyadic Adjustment Scale for use with distressed and nondistressed couples: Construct hierarchy and multidimensional scales. Journal of Marital and Family Therapy, 21:
Jacobson, N. S. & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59,
Hollon, S. D. & Flick, S. N. (1988). On the meaning and methods of clinical significance. Behavior Assessment, 10,
Lambert, M. J., Hansen, N. B., Bauer, S. (2008). Assessing the clinical significance of outcome results. In A. M. Nezu, C. M. Nezu (Eds), Evidence-based outcome research: A practical guide to conducting randomized controlled trials for psychosocial interventions. New York: Oxford University Press.
Seggar, L. B., Lambert, M. J., & Hansen, N. B. (2002). Assessing clinical significance: Application to the Beck Depression Inventory. Behavior Therapy, 33,
Spanier, G. B. (1976). Measuring dyadic adjustment: New scales for assessing the quality of marriage and similar dyads. Journal of Marriage and the Family, 38, Tingey, R., Lambert, M. J., Burlingame, G. M., & Hansen, N. B. (1996). Assessign clinical significance: Proposed extensions in method. Psychotherapy Research, 6,
Wampold, B. E., Jenson, W. R. (1986). Clinical significance revisited. Behavior Therapy, 17,
Lambert, Hansen, and Bauer (2008) discussed difficulties with defining appropriate normative groups when evaluating specific clinical samples’ outcomes.
Hollon and Flick (1988) argued for the use of normative comparison instead of using two arbitrarily defined groups..
Functional
General
Population
Clinical
Dysfunctional 

2. Body font: Helvetica Font size: 9 Group Mean SD 18.23 .581 18.11 .549
Group
Mean SD
Tx Setting
Inpatient & outpatient
18.23
.581
Inpatient & outpatient*
18.11
.549
Outpatient
18.16
.583
17.81
.582
Inpatient
2.35
Inpatient*
18.81
1.96
Diagnosis
GAD*
14.47
2.01
GAD
15.44
2.22
OCD
17.33
2.85
OCD*
17.86
2.12
PD*
24.04
2.02 PD
25.08
2.20
Social Phobia
14.67
1.64