Presentation is loading. Please wait.

Presentation is loading. Please wait.

Rohlings Interpretive Method: How Can a Flexible Battery Perform Like a Fixed Battery Martin L. Rohling, Ph.D. Associate Professor Department of Psychology.

Similar presentations


Presentation on theme: "Rohlings Interpretive Method: How Can a Flexible Battery Perform Like a Fixed Battery Martin L. Rohling, Ph.D. Associate Professor Department of Psychology."— Presentation transcript:

1 Rohlings Interpretive Method: How Can a Flexible Battery Perform Like a Fixed Battery Martin L. Rohling, Ph.D. Associate Professor Department of Psychology University of South Alabama

2 January 13, 2007Rohling - CCPN Orlando, FL 2 Clinical vs. Mechanical Diagnosis Much research has been conducted since Meehl (1954) found clinical judgment to be less accurate than mechanical or actuarial judgement e.g., Dawes, Faust, & Meehl (1989); Filskov (1981); Garb (1989); Garb (1994); Garb (1998); Grove et al. (2000); Sawyer (1966); and Wedding & Faust (1989) Such results influential in causing NPs to turn to different versions of the HRB (Russell, 1998). Batteries have been defined as the method by which one can avoid the clinical errors highlighted by Meehl an others, using actuarial rules for diagnosis (Russell, 1995; Russell et al., 2005).

3 January 13, 2007Rohling - CCPN Orlando, FL 3 Rohlings Interpretive Method (RIM): Development History Conducted several meta-analysis with Dr. Laurence Binder at the Portland, OR – VA The last of these focused on the residual cognitive effects of mild head injury. Binder, Rohling, & Larrabee (1997) Binder et al. grouped effect sizes (ES) into domains of neuropsychological functioning based on factor analytic studies. e.g., Leonberger, Nicks, Larrabee, & Goldfader (1992)

4 January 13, 2007Rohling - CCPN Orlando, FL 4 RIM Generated from Meta-Analytic Procedures Meta-analysis (MA) combines effect sizes (ES) across samples assuming that they all sample the population M for the particular effect of interest. Common method ES calculation is a standardized mean difference score (e.g., Glass delta). delta = difference between con. & exp. groups Ms divided by con. groups SD. delta analogous to Z score - linear equivalent of T score used in clinical neuropsychology

5 January 13, 2007Rohling - CCPN Orlando, FL 5 RIM Generated from Meta-Analytic Procedures Binder et al. (1997) combined ESs generated from various tests into cognitive domains. Why not similarly combine ESs, or T scores, from a single patient into cognitive domains in the same way that it is accomplished in MA. Each test score is treated as a ES that reflects the individuals ability within a domain. ES can be combined based on homogeneity of variance, so as to avoid combining apples and oranges.

6 January 13, 2007Rohling - CCPN Orlando, FL 6 Introduction to the RIM Analysis Flexible battery (multiple measure) use: Is the most frequently cited model of assessment among neuropsychologists. Only 7% of neuropsychologists use a fixed battery (Rabin et al, 2006, ACN). Regarding the suitability, practicality, and usefulness of any fixed battery: We know of no batteries that fully satisfy these criteria. (Lezak, Howieson & Loring 2004, Neuropsych. Assess., 4 th ed, p 648.)

7 January 13, 2007Rohling - CCPN Orlando, FL 7 Advantages of Flexible Battery Dynamic & responsive to clinicians needs Covers 1 or many domains Flexible, can be adapted for each patient Can oversample domains Well suited for hypothesis-driven approach

8 January 13, 2007Rohling - CCPN Orlando, FL 8 Potential Problems with a Flexible Battery Inflated error rates Multicollinearity Weighting decision problems Unknown veracity/reliability of sets of tasks Human judgment errors

9 January 13, 2007Rohling - CCPN Orlando, FL 9 Human Judgment Errors (Wedding & Faust, 1989, ACN) Hindsight bias Confirmatory bias Overreliance on salient data Under-utilization of base rates Failure to take into account co-variation

10 January 13, 2007Rohling - CCPN Orlando, FL 10 Potential Benefits with Rohlings Interpretive Method (RIM) Judgment errors can threaten reliability & validity of multiple measure test batteries. RIM was designed to reduce these effects. Based on meta-analytic techniques. Uses a linear combination of scores placed on a common metric.

11 January 13, 2007Rohling - CCPN Orlando, FL 11 Potential Benefits of RIM A strategy that produces summary results analogous to those generated in a fixed- battery approach (e.g., HII, GNDS, AIR). Takes advantage of psychometric properties of same metric data, e.g., T Scores.

12 January 13, 2007Rohling - CCPN Orlando, FL 12 Todays Presentation - Intent Present a set of procedures that allows for a quantitatively-based comparison of an overall battery of measures. Non-specific to battery measures themselves. Can be used for any individual patient. Demonstrate importance and practicality of use of established statistical indices. (e.g., alpha, beta, effect size).

13 January 13, 2007Rohling - CCPN Orlando, FL 13 Todays Intent (contd) Present a data format for any set of measures to be inspected at: Global level (OTBM) Domain level (DTBM) Test measure level (ITBM) Present a series of calculations to assist in the generation of these indices. Present Steps in conjunction with clinical judgment from an informed position.

14 January 13, 2007Rohling - CCPN Orlando, FL 14 Common RIM Domains of Functioning Symptom Validity (SV) Tests Emotional / Personality (EP) Measures Meta-Cognition, Pain, or other self-ratings Estimated Premorbid General Ability (EPGA) Test Battery Means Overall (OTBM), Domain (DTBM), & Instrument (ITBM) Cognitive Domains: VC, PO, EF, AML, VML, AW, PS Non-Cognitive Domains: PM, LA, SP

15 January 13, 2007Rohling - CCPN Orlando, FL 15 Sample RIM: Summary Table

16 January 13, 2007Rohling - CCPN Orlando, FL 16 Sample RIM: Graphic Display

17 January 13, 2007Rohling - CCPN Orlando, FL 17 Brief of RIM Steps: There are 24 steps to the RIM process 17 calculation steps: Advice on design of the battery Calculation of summary statistics Generation of graphic displays 7 interpretative steps. Detail a systematic procedure for use of the statistical summary table and graphic displays to: Assess and verify summary data. Identify strengths/limitations of current data. Obtain a reliable diagnosis. Develop tx plans based on sound judgments. We briefly review each step in just a moment.

18 January 13, 2007Rohling - CCPN Orlando, FL 18 Support for the RIM Process Rational support/reasoning: Reduce clinical judgment errors. The RIM is a Process, not a program. Rather, the RIM is a way of formalizing thinking & interpretation of individual case data. This is operationalizing what many flexible battery clinicians are already doing in their head.

19 January 13, 2007Rohling - CCPN Orlando, FL 19 Support for the RIM Process: Specific Advantages Psychometric properties at level with fixed, co- normed batteries, without their limitations. Flexibility of test selection. Flexibility of theoretical view of cognition (domain structure)

20 January 13, 2007Rohling - CCPN Orlando, FL 20 Support for the RIM Process: Specific Advantages Quantitatively support your conclusions and interpretations Statistical evaluation Measure of confidence in findings Measure of limitations of findings Ability to present data at different levels of interpretation Greater defensibility

21 January 13, 2007Rohling - CCPN Orlando, FL 21 The RIM has a Set of Procedure or Specific Steps

22 January 13, 2007Rohling - CCPN Orlando, FL 22 RIM Steps 1-4: Summary Data 1) Design & administer battery. Use well standardized recently normed tests. 2) Estimate premorbid general ability. Use Reading (WTAR), Regression (OPIE-III), & academic records (rank, SAT, ACT). 3) Convert test scores to a common metric. We recommend T scores, but z or SS OK too. 4) Assign scores to domains. Factor analysis to support assignment (Tulsky et al., 2003)

23 January 13, 2007Rohling - CCPN Orlando, FL 23 RIM Steps 5-8: Summary Data 5) Calculate domain M, sd, & n. 6) Calculate test battery means (TBM). Overall TBM – All scores, large N & high power. Domain TBM – Avoids domain over weighting. (e.g., attention & memory). Instrument TBM – One score per norm sample. 7) Calculate p for heterogeneity. Have you put apples & oranges together? 8) Determine categories of impairment. Recommend using of Heaton et al. (2003).

24 January 13, 2007Rohling - CCPN Orlando, FL 24 RIM Steps 9-12: Summary Data 9) Determine % of test impaired. Analogous to Halstead Impairment Index # scores below cutoff / total # of of scores 10) Calculate ES for all summary stats. Use Cohens d = (Me – Mc) / SD pooled 11) Calculate confidence interval for stats. 90% CI = x SEM 12) Upper limit of performance for impair. Look for overlap between 90% CI of EPGA (lower) & Summary Stats (upper)

25 January 13, 2007Rohling - CCPN Orlando, FL 25 RIM Steps 13-17: Summary Data 13) Conduct one-sample t tests. Use EPGA as reference point 14) Conduct a between-subjects ANOVA. Looking for strengths & weaknesses 15) Conduct power analyses. Only needed for those NS differences 16) Sort scores for visual inspection. 17) Graphically display summary statistics.

26 January 13, 2007Rohling - CCPN Orlando, FL 26 RIM Steps 18-20: Interpretation 18) Assess battery validity. Examine the Symptom Validity scores. Caution in accepting low power results. Look at heterogeneity of summary stats. 1. Normative sample unrepresentative of patient. 2. Scores assigned to wrong domain. 3. Inconsistent performance on construct measures. 19) Examine influence of psychopathology. Examine scores for heterogeneity. 20) Check OTBM, DTBM, & ITBM impaired.

27 January 13, 2007Rohling - CCPN Orlando, FL 27 RIM Steps 21-24: Interpretation 21) Examine strengths/weaknesses looking at: 1. Confidence intervals overlap. 2. Results from one-sample t tests. 3. Results of ANOVA. 4. %TI show differences otherwise not evident. Determine if pattern existed premorbidly. 22) Examine non-cognitive domains. Psychomotor, Lang/Aphasia, Sensory Percept 23) Explore Type II errors –need more tests? 24) Examine sorted T-scores Look for patterns missed by summary stats.

28 January 13, 2007Rohling - CCPN Orlando, FL 28 RIM Sample Case 1: Obvious TBI Age: 37 Handed: Left Race: Euro-American Sex: Female Ed: 14 years Occup: Nursing Marital: Sep. 10 yrs Living: Camper in parents backyard Reason for Referral: TBI in head-on boat accident. Propeller hit pt in right parietal-occipital lobe (LOC = 7 days; GCS = 3). Eval. to determine capacity for medical & financial decisions, parenting skills, occupational prognosis, & disability status. Significant emotional, behavioral, occupational, and social problems pre-TBI.

29 January 13, 2007Rohling - CCPN Orlando, FL 29 RIM Sample Case 1: Obvious TBI

30 January 13, 2007Rohling - CCPN Orlando, FL 30 RIM Sample Case 1: Obvious TBI

31 January 13, 2007Rohling - CCPN Orlando, FL 31 TBI Dose Response Curves Dikmen ESsMeyers T Scores

32 January 13, 2007Rohling - CCPN Orlando, FL 32 Combined Dikmen & Meyers Estimates: ES, T, & Difference LOCnEST Mean Diff. (EPGA – OTBM) Trauma Control G1: < 1hr G2: 1-24 hr G3: 1-6 day G4: 7-13 day G5: day G6: > 28 day

33 January 13, 2007Rohling - CCPN Orlando, FL 33 Return to Work Study: OTBMs for 4 Groups of TBI Survivors GroupnMSDES Disabled Unemployed Below Previous At Previous

34 January 13, 2007Rohling - CCPN Orlando, FL 34 RIM Sample Case 1: Obvious TBI Normal Distribution of T Scores

35 January 13, 2007Rohling - CCPN Orlando, FL 35 RIM Sample Case 2: Subtle Diabetes Reason for Referral: 2 yrs dangerous work habits. Eval to see if atrial fib & Type II diabetes impairs cognition. Hospitalized TIA-like Sx. Admitted to problems for 20 yrs, cardiac dysrhythmia & bradycardia, pacemaker, blood sugar difficult to manage, & family Hx of heart disease & diabetes. Age: 55 Handed: Right Race: Euro-American Sex: Male Ed: 13 years Occup: Mechanic Marital: Married 20 yr Living: at home w/wife

36 January 13, 2007Rohling - CCPN Orlando, FL 36 RIM Sample Case 2: Subtle Diabetes

37 January 13, 2007Rohling - CCPN Orlando, FL 37 RIM Sample Case 2: Subtle Diabetes

38 January 13, 2007Rohling - CCPN Orlando, FL 38 RIM Sample Case 2: Subtle Diabetes Normal Distribution of T Scores

39 January 13, 2007Rohling - CCPN Orlando, FL 39 RIM Critiques: Concern 1 The method of calculating the standard deviations (SDs) for summary statistics and domain scores is incorrect. Since many of the remaining steps of the RIM depend on the use of these SDs, this error is magnified in the subsequent steps. SDs statistically can not exceed 9.99 and are more likely to be around 6.4

40 January 13, 2007Rohling - CCPN Orlando, FL 40 Response 1: RIM Ms 4 Datasets

41 January 13, 2007Rohling - CCPN Orlando, FL 41 Inter-Individual Ms & SDs NMnSD Dataset 1 Psych Pts WAIS-R WAIS Dataset 2 (Green) Dataset 3 (Meyers) 1, Dataset 4 (HRB) Total 4 Samples 3,

42 January 13, 2007Rohling - CCPN Orlando, FL 42 Response 1: RIM SDs 4 Datasets

43 January 13, 2007Rohling - CCPN Orlando, FL 43 Intra-Individual Ms & SDs NMnSD % > 9.99 Dataset 1 Psych Pts WAIS-R % WAIS % Dataset 2 (Green) % Dataset 3 (Meyers) 1, % Dataset 4 (HRB) % Total 4 Samples 3, %

44 January 13, 2007Rohling - CCPN Orlando, FL 44 RIM Critiques: Concern 2 More false-positives then clinical judgment. Palmer et al. (2004) expressed concern that We failed to distinguish statistical from clinical significance. This failure is a critical error that precludes the prudent clinician from using the RIM.

45 January 13, 2007Rohling - CCPN Orlando, FL 45 Response 2: RIM vs. Manual Detecting Differences – Overall % % of Total SsMANUAL METHOD RIM t TESTVIQ-PIQ: NSVIQ-PIQ: Sig. Marginal Ms VIQ-PIQ: NS 54%23%78% VIQ-PIQ: Sig. 1%21%22% Marginal Ms 55%45%100%

46 January 13, 2007Rohling - CCPN Orlando, FL 46 Response 2: RIM vs. Manual Detecting Differences – ESs Means (SDs)MANUAL METHOD RIM t TESTVIQ-PIQ: NSVIQ-PIQ: Sig. Marginal Ms VIQ-PIQ: NS.38 (.30).80 (.41).50 (.39) VIQ-PIQ: Sig (.82)1.70 (.86)1.69 (.85) Marginal Ms.40 (.37)1.22 (.80).90 (.71)

47 January 13, 2007Rohling - CCPN Orlando, FL 47 Response 2: RIM vs. Manual Detecting Differences Scores Means (SDs)MANUAL METHOD RIM t TESTVIQ-PIQ: NSVIQ-PIQ: Sig. Marginal Ms VIQ-PIQ: NS 3.9 (2.5)13.2 (3.7) 6.7 (5.2) VIQ-PIQ: Sig. 6.7 (0.8)19.0 (6.5) 16.9 (8.5) Marginal Ms 4.0 (2.5)15.9 (6.0) 9.3 (7.4)

48 January 13, 2007Rohling - CCPN Orlando, FL 48 RIM Critiques: Concern 3 Clinicians who use the RIM will: Idiosyncratically assign scores to cognitive domains. This will result in low inter-rater reliability in analysis & diagnosis.

49 January 13, 2007Rohling - CCPN Orlando, FL 49 RIM Critiques: Concern 4 Scores on domains are unit weighted, which introduces error. Willson & Reynolds (2004) said scores load on multiple domains. Assignment to domains & weights depend on: Battery of tests administered. Patients whose test scores are being examined.

50 January 13, 2007Rohling - CCPN Orlando, FL 50 Response 4: Cross-Valid. Unit Wts Conducted 4 multiple reg. on 457 pts WAIS-R. Split sample in ½ - assess shrinkage. Regressed patients verbal subtests onto PIQ. Generated ideal weights for the 1 st ½ of sample. Used wts to predict PIQs in the 2nd ½ of sample. Pre-PIQs regressed on actual PIQs 2 nd ½ sample. Also, generated weights for the 2 nd ½ of sample. Use Pre-PIQs regress on actual PIQs 1 st ½ sample. Repeated, except performance subtests predict VIQ split sample ½ & generate same statistics as before.

51 January 13, 2007Rohling - CCPN Orlando, FL 51 Response 4: Cross-Valid. Unit Wts Purpose of these procedures: How much variance in wts. is sample specific. Amount of shrinkage using cross-validated wts. Shrinkage error compared to error introduced by using unit wts vs. ideal wts. Results: 98% of the variance accounted for with unit wts. Compared to ideal weights. Support use of unit wts. Rather than ideal wts. See, Dawes, R. M. (1979).

52 January 13, 2007Rohling - CCPN Orlando, FL 52 RIM Critiques: Concern 5 Multiple measures used to generate composite scores: Results in less accurate estimates of the cognitive domains.

53 January 13, 2007Rohling - CCPN Orlando, FL 53 Response 5: Estimate FSIQ Using Scaled Score Meanss

54 January 13, 2007Rohling - CCPN Orlando, FL 54 RIM Critiques: Concern 6 A general ability factor is used to represent premorbid functioning for all domains. This not supported by the literature. This results in inaccurate conclusions regarding degree of impairment suffered by a patient in each cognitive domains assessed.

55 January 13, 2007Rohling - CCPN Orlando, FL 55 Domain Means Correlations Domain Means Correlations All were Significant ( p <.001 ) Premorbid OTBM DTBM Attention/Working Mem Proc. Spd/Mental Flex Verbal Reasoning Visual Reasoning Verbal Memory Visual Memory Dom Motor/Sensory Non-Dom Motor/Sens

56 January 13, 2007Rohling - CCPN Orlando, FL 56 RIM Critiques: Concern 7 Norms used come from samples that are of undocumented comparability. Furthermore, even when norms used were generated from different but comparable samples, their format prohibits ready comparisons.

57 January 13, 2007Rohling - CCPN Orlando, FL 57 Response 7: Split-Half Reliability Analyze Dataset 2: OTBMs from 42 DVs Individuals data split into two sets 21 test variables for each OTBM (1 & 2) 2 independent OTBMs created for pt. Split DVs intentionally-separated so no normative sample included both OTBMs

58 January 13, 2007Rohling - CCPN Orlando, FL 58 Response 7: Split-Half Reliability Results r =.81, 66% of variance accounted Slope of the regression line was.82 (SE =.027) Intercept 9.2 (SE = 1.20). Mean OTBM-1 = 45.0 (sd = 7.3) Mean OTBM-2 = 43.6 (sd = 7.2) Results simulate worse case scenario. used an entirely different set of norms. Est. test-retest r for OTBM 42 DVs increased r from.81 to.88 using the Spearman-Brown correction).

59 January 13, 2007Rohling - CCPN Orlando, FL 59 Response 7: Split-Half Reliability No overlap in normative samples. Worst-case condition, generally administer instruments (e.g., WAIS-III) with OTBMs generated from co-normed variables. Meyers & Rohling test-retest reliability of.86. When different norms used, often gave same instruments (e.g., AVLT or RCFT) No instrument used OTBM-1 included OTBM-2 Heaton et al.s (2001) - schizophrenic pts. Obtained a test-retest reliability of.97. Comparing 2 identical batteries, not worst-case.

60 January 13, 2007Rohling - CCPN Orlando, FL 60 RIM Critiques: Concern 8 The RIM will result in an undue inflation of clinicians confidence. Such overconfidence results in more error in a interpretation, not less.

61 January 13, 2007Rohling - CCPN Orlando, FL 61 RIM vs. Tulsky et al. (2003): Case 1 WAIS & WMS BatteryFull Battery Index or Domain Scores Disc. Model RIM Model # # S EPGA 1 (WTAR) U Overall TBM 2 (FSIQ) (112) I 70 M Domain TBM M Inst TBM 2 (FSIQ/GMI) I 14 1 Verbal Comp (VCI) 120 * 118 * Percept Organ (POI) 121 * 117 * Attent/Work Mem (WMI) I* 9 4 Process Speed (PSI) 93 ** 95 ** 286 I* 5 5 Aud Mem & Learn (AMI) 97 ** 4 81 I***** 18 6 Vis Mem & Learn (VMI) Executive Function (EF) I* 15 8 Psycho-Motor (PM)

62 January 13, 2007Rohling - CCPN Orlando, FL 62 RIM vs. Tulsky et al. (2003): Case 2 WAIS & WMS BatteryFull Battery Index or Domain Scores Disc. Model RIM Model # # S EPGA 1 (WTAR) U Overall TBM 2 (FSIQ) (119) I 53 M Domain TBM I 7 M Inst TBM 2 (FSIQ/GMI) I 11 1 Verbal Comp (VCI) Percept Organ (POI) Attent/Work Mem (WMI) Process Speed (PSI) Aud Mem & Learn (AMI) I 9 6 Vis Mem & Learn (VMI) I 10 7 Executive Function (EF) I 14 8 Psycho-Motor (PM)

63 January 13, 2007Rohling - CCPN Orlando, FL 63 Summary of the Rohling Interpretive Method of Statistical Analysis of Individual Neuropsychological Test Data

64 January 13, 2007Rohling - CCPN Orlando, FL 64 Summary of RIM Steps 24 total steps to the process 17 calculation steps Battery Design Calculation of summary statistics Generation of graphic displays 7 interpretative steps Use of summary table and graphic displays to: Assess and verify summary data Identify strengths/limitations of current data Obtain a reliable diagnosis Develop tx plans based on clinical judgments.

65 January 13, 2007Rohling - CCPN Orlando, FL 65 Summary of RIM Advantages Formulize thinking interpretation of data: Operationalize what you already do. Reduce judgment errors thru RIM Process. Take advantage of psychometric properties at level with fixed, co-normed batteries. Allows flexibility of test selection. Allows flexibility of theoretical view of cognition (e.g., domain structure)

66 January 13, 2007Rohling - CCPN Orlando, FL 66 Summary of RIM Advantages contd Gives Quantitative support for your conclusions and interpretations Statistical evaluation Measure of confidence in findings Measure of limitations of findings Ability to present data at different levels of interpretation Equals greater defensibility

67 January 13, 2007Rohling - CCPN Orlando, FL 67 Our RIM Cautions/Concerns Does not replace clinical judgment, rather, informs clinical judgment. This still means CJ errors are possible. Susceptibility T-Scores to distrib. deviance Process, not program Pre-morbid ability estimates Domain selection, test placement

68 January 13, 2007Rohling - CCPN Orlando, FL 68 RIM is Not Alone Out There! Dawn Flanagan, Ph.D., at St. Johns University in New York independently developed a similar method The Cattell-Horn-Carroll (CHC) Cross Battery Approach. Second edition of Essentials of Cross-Battery Assessment (Flanagan, Ortiz, & Alfonso, in press) is due out in March, which explains her method, along with co-authors Some of her work can also be found on the website by Dumont-Willis.

69 Published Research Findings Using the RIM 1) RIM vs. HRB 2) Variance Accounted for by SVT 3) Effect of Depression on NP Results 4) Prediction of Employment after TBI

70 January 13, 2007Rohling - CCPN Orlando, FL 70 RIM of HRB: OTBM vs. HII Heaton et al.s (1991) HRB norms for OTBM T Score (M=50, sd=10) OTBM r with HII = -.79 (p <.0001) 62% variance account. Over predicts low Under predicts high

71 January 13, 2007Rohling - CCPN Orlando, FL 71 RIM of HRB: OTBM vs. GNDS OTBM r with GNDS = % variance acc. OTBM neither under nor over predicts across range of GNDS Intercept impairment is T Score = 46.0 Reitan & Wolfson (1993) (GNDS = 29)

72 January 13, 2007Rohling - CCPN Orlando, FL 72 RIM of HRB: OTBMs Relationship to Global Indices INDICES OF FUNCTION Correlation Coefficient Halstead Impairment Index.79 Average Impairment Rating.90 Global Neuro. Deficit Scale.87 RIM: Domain TBM.99 RIM: Instrument TBM.95 RIM: % Tests Impaired.96

73 January 13, 2007Rohling - CCPN Orlando, FL 73 RIM of HRB: Diagnostic Classification Using the HII BR 65% Sens.Spec.PPVNPV % Corr. HII % AIR % GNDS % OTBM % ITBM % %TI %

74 January 13, 2007Rohling - CCPN Orlando, FL 74 RIM of HRB: Cross-Validation of RIM using HRB in 2 Samples Regressed Dikmen & Meyers TBI data Generated a predicted HII for pts in OK dataset. Correlation actual & predicted HII =.95 Sen =.60, Spec =.77, PPV =.78, NPV =.59 Overall % Correct Classification = 71% Predicted HII from MNBs OTBM & got a more accurate indicator of impairment than actual HII

75 January 13, 2007Rohling - CCPN Orlando, FL 75 Factor Loadings of Domain Scores Genuine NormalGenuine NeuroExag Normal Obj Perf Self- Report Obj Perf Self- Report Obj Perf Self- Report NPT SVT MCI PSX Eigen % Var. 33%46%30%42%51%29%

76 January 13, 2007Rohling - CCPN Orlando, FL 76 M & SDs of Composite Z scores GenuineExaggerate NormalNeurologicNormalNeurologic Neuropsych Test Scores.33 (.62).19 (.64) -.60 (.80) -.79 (.65) Symptom Validity.51 (.38).50 (.30) (.94) -.50 (.52) Memory Complaints.14 (.93).41 (.92) -.62 (.93).49 (.54) Psychiatric Symptoms.10 (.95).46 (.96) -.39 (.85).19 (.99)

77 January 13, 2007Rohling - CCPN Orlando, FL 77 Mean Z score on Objective Tests Small diff. between Gen. Normal & Gen. Neuro. on NPT No diff. between Exag. Normal & Exag. Neuro on NPT Deficits for Exag. Neuro were more modest than for Exag. Normals on SVT Interaction between Validity & Neuro Status.

78 January 13, 2007Rohling - CCPN Orlando, FL 78 Mean Z score Self-Report No diff. between Gen. Neuro. & Exag. Neuro on Memory Complaints No diff. between Gen. & Exag. Neuro on Psych. symptoms Deficits for Exag. Normal on Psych. symptoms & Memory Complaints, the latter is larger Interaction between Validity & Neuro Status.

79 January 13, 2007Rohling - CCPN Orlando, FL 79 Depression Study: Reference Rohling, M. L., Green, P., Allen, L. M., & Iverson, G. L. (2002). Depressive symptoms and neurocognitive test scores in patients passing symptom validity tests. Archives of Clinical Neuropsychology, 17,

80 January 13, 2007Rohling - CCPN Orlando, FL 80 Mood Group Assignment Patients classified into 2 subgroups From entire sample, 420 passed all SVTs Sample split based on BDI Low-Depressed 25%ile on BDI (< 10) n = 178, M = 6 (3) High-Depressed 75%ile on BDI (> 25) n = 187, M = 31 (6)

81 January 13, 2007Rohling - CCPN Orlando, FL 81 Depression Study Participants All 365 patients referred for evaluation for compensation-related purposes All diagnostic groups included 53% Head injury referrals 22% Medical referrals 14% Psychiatric referrals 11% Other neurological Age = 42 (11); Ed = 13 (3); Sex = 64% males; Non-English = 18%; Handedness = 9% Left

82 January 13, 2007Rohling - CCPN Orlando, FL 82 Results Mood & Validity Status 107 (30%) 68 (19%) 159 (44%) 27 (7%) SVT Status Mood BDI NonDep 25%ile Depress 75%ile GenuineExaggerating 175 (48%) 186 (52%) 266 (74%) 95 (26%)

83 January 13, 2007Rohling - CCPN Orlando, FL 83 Results: Sample Split by Validity

84 January 13, 2007Rohling - CCPN Orlando, FL 84 Exaggerating patients accounted for 39% of High-Dep group 14% of Low-Dep group Mood & Effort used as IVs and Cognition DV Effect for effort, no effect for mood However, when Memory Complaints DV Effects for both effort and mood Also, when other Emotion/Personality DV Effects for both effort and mood Effect of Mood Depends on Effort

85 January 13, 2007Rohling - CCPN Orlando, FL 85 Effect of Mood Depends on Effort When both mood groups were included in regression analysis, as predicted: Memory ratings related to mood (r =.60; p <.0001) Mood not correlated with cognition (r =.10; p >.10) Memory ratings not related to cognition (r =.13, p =.06)

86 January 13, 2007Rohling - CCPN Orlando, FL 86 Mood Replication Gervais pain sample findings (n = 177) Exaggerating patients accounted for 55% of High-Dep; 33% of Low-Dep group Memory ratings related to mood (r =.55) Mood not correlated with cognition (r =.06) Memory ratings related to cognition (r =.15) Group means correlated with Greens.94 all patient (High-D, Low-D, Gen, & Exag).

87 January 13, 2007Rohling - CCPN Orlando, FL 87 Effect if Pain on OTBM

88 January 13, 2007Rohling - CCPN Orlando, FL 88 Effect if Pain on OTBM

89 January 13, 2007Rohling - CCPN Orlando, FL 89 Return to Work after Injury Three main hypotheses using MNB-RIM OTBM will predict return to work level Cognitive domain that will be most predictive will be executive function Adding the Patient Competency Rating Scale will improve work prediction PCRS is by Prigatano (1985)

90 January 13, 2007Rohling - CCPN Orlando, FL 90 Return to Work: ANOVA of OTBM GroupnMSDES Disabled Unemployed Below Previous At Previous

91 January 13, 2007Rohling - CCPN Orlando, FL 91 Logistic Regression Using OTBM Predicted ObservedDisable Un- employ Below Prev AtPrev % Corr Disabled % Unemployed % Below Previous % At Previous %

92 January 13, 2007Rohling - CCPN Orlando, FL 92 Return to Work: Summary OTBM differences between groups Disabled /Unemployed not able to separate Below/At Previous not able to separate Collapsed groups result in 71% correct above base rate of 52% correct

93 January 13, 2007Rohling - CCPN Orlando, FL 93 Return to Work: Domain Analysis Executive function not the most predictive Most of variance carried by Perceptual Organization & Working Memory Using Cognitive Domains OTBM increases % Correct from 71% to 74% Incremental validity of PCRS very low. 7% of the variance

94 January 13, 2007Rohling - CCPN Orlando, FL 94 Return to Work: Domain Analysis By including premorbid variables, increases diagnostic accuracy; most helpful being: Premorbid IQ, level of occupation, & education Including acute measures increases accuracy; most helpful being: LOC group Time since injury

95 January 13, 2007Rohling - CCPN Orlando, FL 95 Depression Study Conclusions Memory complaints not synonymous with impairment in compensation sample Findings replicated Effort accounts for more variance in self- ratings of cognition & objective performance than mood Findings replicated

96 January 13, 2007Rohling - CCPN Orlando, FL 96 Whats wrong with this patient-1? (Key: RCPS)

97 January 13, 2007Rohling - CCPN Orlando, FL 97 Whats wrong with this patient-1? (Key: RCPS)

98 January 13, 2007Rohling - CCPN Orlando, FL 98 Whats wrong with this patient-2? (Key: JSVD)

99 January 13, 2007Rohling - CCPN Orlando, FL 99 Whats wrong with this patient-2? (Key: JSVD)

100 January 13, 2007Rohling - CCPN Orlando, FL 100 Whats wrong with this patient-3? (Key: NPAD)

101 January 13, 2007Rohling - CCPN Orlando, FL 101 Whats wrong with this patient-3? (Key: NPAD)

102 January 13, 2007Rohling - CCPN Orlando, FL 102 Whats wrong with this patient-4? (Key: SMAA)

103 January 13, 2007Rohling - CCPN Orlando, FL 103 Whats wrong with this patient-4? (Key: SMAA)

104 January 13, 2007Rohling - CCPN Orlando, FL 104 Rohlings Interpretive Method: Use of Meta-Analytic Procedures for Single Case Data Analysis Martin L. Rohling Questions & Comments Welcome!

105 January 13, 2007Rohling - CCPN Orlando, FL 105 CT/MRI Data Participant Demographic Information Variable Sample Sizes (N = 124) Gender Male 82 Female 42 Ethnicity Caucasian119 Other 5

106 January 13, 2007Rohling - CCPN Orlando, FL 106 CT/MRI Diagnostic Groups Sample Size MVA/TBI47 Blow to Head32 LCVA24 RCVA21

107 January 13, 2007Rohling - CCPN Orlando, FL 107 CT/MRI 1. All were Right Handed. 2. All were followed by Dr. Meyers through hospitalization and rehabilitation. 3. None were involved in litigation. 4. All passed internal validity checks.

108 January 13, 2007Rohling - CCPN Orlando, FL 108 CT/MRI CT/MRI Location Left Frontal59 Left Parietal37 Left Temporal34 Left Occipital 6 Right Frontal40 Right Parietal42 Right Temporal31 Right Occipital 3

109 January 13, 2007Rohling - CCPN Orlando, FL 109 CT/MRI All were given MNB CT/MRI data coded for injury reported on MRI/CT at the time of injury Present = 1 Absent = 0

110 January 13, 2007Rohling - CCPN Orlando, FL 110

111 January 13, 2007Rohling - CCPN Orlando, FL 111

112 January 13, 2007Rohling - CCPN Orlando, FL 112 CT/MRI Independent Sample 1-tailed t-test on each lobe On CT/MRI report Present = 1 Absent = 0

113 January 13, 2007Rohling - CCPN Orlando, FL 113 CT/MRI Data

114 January 13, 2007Rohling - CCPN Orlando, FL 114 Brain Regions Involved in the Performance of WAIS-III Arithmetic

115 January 13, 2007Rohling - CCPN Orlando, FL 115 Brain Regions Involved in the Performance of the Boston Naming Test

116 January 13, 2007Rohling - CCPN Orlando, FL 116 Brain Regions Involved in the Performance of the Rey-CFT Copy

117 January 13, 2007Rohling - CCPN Orlando, FL 117 Brain Regions Involved in the Performance of the AVLT Total Score

118 January 13, 2007Rohling - CCPN Orlando, FL 118 CT/MRI NP tests generally behaved as expected A more Systemic or Domain like approach better at explaining results Construct of Executive Function not supported.

119 January 13, 2007Rohling - CCPN Orlando, FL 119 Domains used by the MNB Attention/Working Memory: Digit Span Forced Choice Animal Naming Sentence Rep AVLT 1 Processing Speed/Mental Flexibility: Digit Symbol Dichotic Both Trails A Trails B

120 January 13, 2007Rohling - CCPN Orlando, FL 120 Domains used by the MNB Verbal Reasoning Similarities Arithmetic Information COWA Dichotic Left Dichotic Right Boston Naming Token Test Visual Reasoning Picture Completion Block Design JOL Category RCFT Copy

121 January 13, 2007Rohling - CCPN Orlando, FL 121 Domains used by the MNB Verbal Memory AVLT Total AVLT Immediate AVLT Delayed AVLT Recognition Visual Memory RCFT Immediate RCFT Delayed RCFT Recognition

122 January 13, 2007Rohling - CCPN Orlando, FL 122 Domains used by the MNB Motor and Sensory Finger Tapping Dominant Hand Finger Tapping Non-Dominant Hand Finger Localization Dominant Hand Finger Localization Non-Dominant Hand

123 January 13, 2007Rohling - CCPN Orlando, FL 123

124 January 13, 2007Rohling - CCPN Orlando, FL 124

125 January 13, 2007Rohling - CCPN Orlando, FL 125

126 January 13, 2007Rohling - CCPN Orlando, FL 126

127 January 13, 2007Rohling - CCPN Orlando, FL 127 Commonality of Reduced O2

128 January 13, 2007Rohling - CCPN Orlando, FL 128 Domain Consistency N = 936 Passed all validity checks No missing data Not involved in litigation Calculated Domain Ms Regression used to predict Domain Ms using all on other Domain Ms

129 January 13, 2007Rohling - CCPN Orlando, FL 129 Domain Means Correlations – Premorbid OTBM DTBM Attent/Work Mem – Pro Spd/Mental Flex Verbal Reason Visual Reason Verbal Memory Visual Memory Dom Motor/Sensory Nond Motor/Sensory All were Significant p <.001

130 January 13, 2007Rohling - CCPN Orlando, FL 130 Domain Ms Correlations (cont.) Premorbid OTBM DTBM Attent/Work Mem ProcSpd/Ment Flex Verbal Reasoning Visual Reasoning Verbal Memory Visual Memory Dom Motor/Sen Nond Motor/Sen All were Significant p <.001

131 January 13, 2007Rohling - CCPN Orlando, FL 131 Domains Regression Equations Attention & Working Memory (Verbal Reasoning) *.315 (Verbal Memory) *.273 (Processing Speed) *.193 Constant =

132 January 13, 2007Rohling - CCPN Orlando, FL 132 Domains Regression Equations Processing Speed/ Mental Flexibility Verbal Reasoning *.401 Visual Reasoning *.284 Attention & Working Memory *.230 Constant = 2.434

133 January 13, 2007Rohling - CCPN Orlando, FL 133 Domains Regression Equations Verbal Reasoning Processing Speed *.361 Attention & Working Memory *.354 Visual Reasoning *.243 Constant = 2.5

134 January 13, 2007Rohling - CCPN Orlando, FL 134 Domains Regression Equations Visual Reasoning Visual Memory *.322 Processing Speed/Mental Flexibility *.213 Verbal Reasoning *.208 Constant =

135 January 13, 2007Rohling - CCPN Orlando, FL 135 Domains Regression Equations Verbal Memory Attention & Working Memory *.738 Visual Memory *.388 Constant =

136 January 13, 2007Rohling - CCPN Orlando, FL 136 Domains Regression Equations Visual Memory Visual Reasoning *.698 Verbal Memory *.311 Processing Speed *.0909 Constant =

137 January 13, 2007Rohling - CCPN Orlando, FL 137 Regression Adjusted SE RR 2 of the Estimate Attent/Working Memory Processing Speed Verbal Reasoning Visual Reasoning Verbal Memory Visual Memory

138 January 13, 2007Rohling - CCPN Orlando, FL 138 Review Took a battery of well known tests Developed Norms Identified Validity, Reliability, Sensitivity and Specificity. Internal Validity Checks and Internal Consistency


Download ppt "Rohlings Interpretive Method: How Can a Flexible Battery Perform Like a Fixed Battery Martin L. Rohling, Ph.D. Associate Professor Department of Psychology."

Similar presentations


Ads by Google