Test-Retest Reliability of the Work Disability Functional Assessment Battery (WD-FAB) Dr. Leighton Chan, MD, MPH Chief, Rehabilitation Medicine Department National Institutes of Health Clinical Center NI H

Disclosure I have no potential conflict of interest to report

Social Security Administration-National Institutes of Health Collaboration SSA- NIH Objectives –Use Item Response Theory (IRT) Computerized Adaptive Testing (CAT) to create a Work Disability Functional Assessment Battery (WD-FAB) NIH awarded a contract in 2009 to Boston University to develop what is now the WD-FAB

What is WD-FAB? An individualized assessment of functional activity that measures self-reported functional ability Includes 313 items across eight scales of functional activity 4 physical domains: Basic Mobility; Upper Body Function; Fine Motor Function; Community Mobility 4 mental health domains: Mood & Emotions; Cognition & Communication; Resilience/Adaptability; Social Interaction Highly efficient-~15-20 minutes to complete

WD-FAB Reliability Studies Test-retest studies – Examined the initial physical functioning and mental health FAB domains Validity Studies – Construct, divergent validity – Ceiling, floor effects – Respondent burden

2014 Test Re-test Reliability Study… Reliability in this context is the measure’s ability to produce consistent scores for a respondent at different points in time Inclusion criteria –21-66 years old –Includes a sample of individuals who self-reported inability to work due to a permanent disability FAB was administered twice, 7-10 days apart –Participants also indicated whether their physical or mental health had improved, worsened, or stayed the same over the past week Administered the FAB to a sample of 376 adults living in US & a sample of 316 adults who reported being work disabled –Both adult samples were drawn from a large internet opt-in survey pool, by YouGov, Inc.

Results Intra-class correlation coefficient using a 2-way mixed model: ICC range:General PopulationWork Disabled Physical Function SEM (Standard Error of Measurement) MDC90 (Minimal detectable change) Behavioral Health SEM MDC

Test Retest Conclusions WD-FAB demonstrates adequate test- retest reliability –Reliability of the scales is good (>0.7) slightly higher better in PF scales –MDC: Needs to be improved in BH domains- may have been enhanced through replenishment –Second Test-retest study is underway

Additional psychometric tests of FAB… Validity –Concurrent/Divergent validity assessed by examining correlations between the FAB and scores on legacy measures of each domain –Also assessed data quality, efficiency of CAT administration and measurement accuracy –A sample of individuals self-reporting inability to work due to permanent disability completed the FAB and additional assessments to measure either physical or mental health

Legacy instruments Physical function Physical component summary (PCS) of the Veteran’s Rand-36 (similar to SF-36) Patient Reported Outcomes Measurement Information System (PROMIS) physical function short form Mental health function Mental component summary (MSC)of the VR-36 Behavior and symptom identification scale (Basis)-24

Physical Function Correlations MCSPCSPPF10CMBPUBFUEFM VR-36 MCS1 VR-36 PCS-0.25 PROMIS Physical Function 10-Item Short Form (PPF10) Changing and Maintainting Body Position (CMBP).12*0.42ᶧ0.65ᶧ Upper Body Function (UBF).21*0.43ᶧ0.69ᶧ.63 Upper Extremity Fine Motor (UEFM).24*0.23ᶧ0.54ᶧ Whole Body Mobility (WBM).15*0.55ᶧ0.7ᶧ Notes. Sample sizes varied from 417 to 497 owing to differences in the proportion of missing data across scales. All correlations are significant at P<.05. *Discriminant validity correlations. ᶧ Convergent validity correlations.

Mental Health Correlations VR-36BASIS-24BH MCSPCSDEPRELATEHARMEMOTPSYCHSELF-EMOODBC VR-36 MCS1. VR-36 PCS-.15 BASIS-24: Depression/Functioning (DEP) BASIS-24: Relationships (RELATE) BASIS-24: Self-Harm (HARM) BASIS-24: Emotional Lability (EMOT) BASIS-24: Psychosis (PSYCH) Self-Efficacy (SELF-E)0.46*0.06ᶧ-0.46*-0.41*-0.3*-0.35*-0.32* Mood and Emotions (MOOD)0.67*0.21ᶧ-0.74*-0.42*-0.47*-0.54*-0.39*.49 Behavioral Control (BC)0.32*0.07ᶧ-0.36*-0.28*-0.35*-0.59*-0.43*.43 Social Interactions (SOCIAL)0.56*0.31ᶧ-0.63*-0.39*-0.28*-0.31*-0.24* Notes. Sample sizes varied from 466 to 476 owing to differences in the proportion of missing data across scales. All correlations are significant at P<.05 except for correlations of <.10. BASIS-24, 24-Item Behavior and Symptom Identification Scale *Discriminant validity correlations. ᶧ Convergent validity correlations.

FAB Validity and Reliability Studies Key findings: –The Physical Function and Behavioral Health domains demonstrated good test-retest reliability in adults with work-disability and general adult samples –Studies revealed minimal missing data, substantial score variation, absence of clustering at the floor and ceiling –Low respondent burden (6.5 min to complete each test) –Measurement accuracy was very high for the physical functioning domain; behavioral health measures demonstrated more variability –Concurrent validity correlations for 2 FAB domains with legacy measures were moderate to strong

QUESTIONS?