1. Cricket Mitchell, PhD CIMH Evaluation Consultant CiMH Palette of Measures Evaluation Training: How I Think Questionnaire (HIT)

2. Palette of Measures Evaluation: What You Will Need (slide 1 of 2) Palette of Measures Data Entry Shell v2 (Excel file) –Developed by CiMH and customized for each participating agency –Holds all data for clients served thru the Palette of Measures project –Demographics, service delivery information, pre- and post- outcome measure data Palette of Measures Data Dictionary v2 (Word document) –A guide for using the associated data entry shell –Defines each column in the excel file 2

3. Palette of Measures Evaluation: What You Will Need (slide 2 of 2) Outcome measures from the two-pronged approach –General Outcome Measure –Target-Specific Outcome Measure(s) –For example… How I Think Questionnaire © (HIT) –Copyrighted and published by Research Press Must be purchased by interested Palette of Measures project participating agencies 3

4. Overview of Training Brief Overview of Palette of Measures evaluation protocol How I Think Questionnaire © : Target-specific measure for disruptive behavior –Administration –Scoring –Clinical Utility Instructions for Palette of Measures data entry and data submissions –Data entry: HIT –Data entry: Demographics & Services –Data submissions to CiMH 4

5. 5 Brief Overview of Palette of Measures Evaluation Protocol

6. Outcome Assessment Palette of Measures providers will track outcomes using data from pre- and post- administrations of standardized measures of functioning Pre- and Post- a “dose” of treatment / an intervention interval –General measure of youth mental health functioning (e.g., YOQ/YOQ-SR, CANS, Ohio Scales) –Target-specific measure linked to focus of treatment/intervention (e.g., AQ, ECBI, PHQ-9, PTSD-RI, RCADS) Providers may choose to administer mid-course assessments as well –e.g., at 3-month intervals in usual care 6

7. A note about the use of standardized assessment measures… (slide 1 of 2) Assessment is the beginning of developing a relationship with the child and family –Demonstrates a desire to know what the child and family are experiencing –By incorporating standardized assessment measures of functioning, the efficiency and thoroughness of assessment is enhanced 7

8. A note about the use of standardized assessment measures… (slide 2 of 2) Using standardized assessment measures of functioning… –Assists in initial clinical impressions –Provides valuable information to guide treatment/interventions –Assesses sufficiency of treatment delivered –Demonstrates treatment-related improvements in child functioning 8

9. 9 How I Think Questionnaire © (HIT)

10. CiMH HIT Training Information on the administration, scoring, and clinical utility of the HIT was obtained from the measure’s Manual, written by Alvaro Q. Barriga, John C. Gibbs, Granville Bud Potter and Albert K. Liau, and published by Research Press 10

11. How I Think Questionnaire © (HIT) Description Target-specific measure for disruptive behavior Measures youth tendencies to engage in self-serving cognitive distortions, or thinking errors –Youth self-report for ages 12-19 5-15 minutes to complete 11

12. How I Think Questionnaire © (HIT) Description Written at a 4 th grade reading level Valid and reliable Copyrighted and published by Research Press –After purchase of the manual (~$2) the cost per use ~$1.20 **20% discount if purchased through CIMH –http://www.researchpress.com/product/item/5136/http://www.researchpress.com/product/item/5136/ 12

13. HIT © Description 54 items 6-point Likert scale response options –Disagree strongly (1) –Disagree (2) –Disagree slightly (3) –Agree slightly (4) –Agree (5) –Agree strongly (6) Eight Subscale Scores –Self-Centered (SC) –Blaming Others (BO) –Minimizing/Mislabeling (MM) –Assuming the Worst (AW) –Opposition-Defiance (OD) –Physical Aggression (PA) –Lying (L) –Stealing (S) Two Summary Scale Scores –Overt –Covert HIT Total Score 13

14. Example: Items from the HIT © People should try to work on their problems. If I see something I like, I take it. When I get mad, I don’t care who gets hurt. Everybody lies, it’s no big deal. You should get what you need, even if it means someone has to get hurt. 14

15. HIT Administration Administer pre- and post- a dose of treatment, or an intervention interval, focused on disruptive behavior –HIT completed by client –Some agencies may choose to administer mid-treatment assessments as well 15

16. Let’s take a look at the HIT... (not distributed due to copyright laws) 16

17. How I Think Questionnaire © Scoring Use the HIT Questionnaire Computational Form –Only one is provided with each HIT manual, this form will need to be copied to score each HIT For each item, transfer the value corresponding to the circled response into box under the appropriate scale (item # in parentheses) –Values for each item response are on slide 13 –Each item loads onto one of the nine scales Determine each scale’s Mean score –Sum all responses and divide by the number of items in that scale Determine the Mean Summary Scores (e.g. Overt Scale, Covert Scale, Overall HIT Score) 17

18. HIT Scoring Missing Data (items that are left blank) –If 5 or more items are missing, consider the questionnaire invalid –If an item is left blank, it is not coded and will not contribute to the scale score in which it falls Note that this will change the denominator (divisor) in determining the scale score, which is the mean, or average of all scale items Two Responses to One Question –If a youth has circled two responses to an item, take the higher value of the two 18

19. HIT Scoring Use the HIT Questionnaire Profile Form to plot a youth’s scores –As with the HIT Questionnaire Computational Form, only one is provided with each HIT manual, this form will need to be copied to score each HIT The Profile Form shows youth scores that are above the clinical cutpoint, are borderline-clinical, or are in the nonclinical range 19

20. Clinical Utility of the HIT HIT Total Score/Overall HIT Score –Possible scores range from 1.0 to 6.0 –Clinical cutpoint 2.98 or higher –Total Scores are reflective of self-serving cognitive distortions, or thinking errors, reported by youth –High scores reflect more cognitive distortions that are likely related to antisocial behaviors; whereas, low scores reflect more normative, non-clinical thought processes It is important to examine the profile of elevations across all summary scales and subscales 20

21. Clinical Utility of the HIT HIT Subscale Scores –Possible scores range from 1.0 to 6.0 –Clinical cutpoints vary (see slide 26) –Each subscale is named for its item content e.g., the Blaming Others subscale contains items that assess youth tendencies to blame others for their actions, the Lying subscale contains items that assess youth condoning and engaging in lying –Higher scores reflect a higher endorsement of that thought process or behavior 21

22. Clinical Utility of the HIT HIT Subscale Scores –Examining the profile of cognitive distortion scales should provide a method for directly assessing the particular cognitive style displayed by a youth –Permits clinicians to individualize treatment approaches more effectively e.g., youth with elevated Self-Centered scores are likely to require cognitive interventions that address a strong egocentric bias 22

23. Clinical Utility of the HIT Overt Summary Scale Opposition-Defiance (OD) and Physical Aggression (PA) –The Overt Scale reflects a predilection for antisocial behavior that typically involves confrontation of a victim Covert Summary Scale Lying (L) and Stealing (S) –The Covert Scale reflects a preference for antisocial behavior that is primarily nonconfrontational 23

24. Clinical Utility of the HIT Treatment planning will likely vary for youth with high Overt Scale scores vs. youth with high Covert Scale scores –Numerous studies have found differences in the etiological factors, symptom presentation, and long-term prognosis between youth who display overt aggression and youth who engage in delinquency that is more covert 24

25. Clinical Utility of the How I Think Questionnaire © Assesses a variety of specific aspects of self- serving youth thinking errors that are likely to contribute to aggression and delinquent behavior –e.g., blaming others, assuming the worst, minimizing and mislabeling Informs which aspects of thinking errors and associated antisocial behavior youth have the greatest need for intervention Comparisons of pre/post scores demonstrate treatment-related outcomes in maladaptive thought processes 25

26. Summary of HIT Score Ranges and Clinical Cutpoints 26 HIT Score (range 1-6) Clinical Cutpoint HIT Score (range 1-6) Clinical Cutpoint Self-Centered (SC)3.16Lying (L)3.39 Blaming Others (BO)3.08Stealing (S)2.57 Minimizing/Mislabeling (MM)2.96 Assuming the Worst (AW)2.92Overt Summary Score3.06 Opposition-Defiance (OD)3.20Covert Summary Score2.96 Physical Aggression (PA)3.01HIT Total Score2.98

27. 27 Palette of Measures Data Entry and Data Submissions

28. Palette of Measures Data Entry Shell There is a separate spreadsheet in the excel workbook (aka database) for each type of data: –Demographics & Services –Pre- General outcome measure –Post- General outcome measure –Pre- Target-specific measure(s) –Post- Target-specific outcome measure(s) Specific outcome measure spreadsheets included in each agency’s database varies across Palette of Measures project participants 28

29. Palette of Measures Data Entry Shell In addition to the spreadsheets that hold data... –There is an Instructions spreadsheet Basic data entry instructions Contact information for T.A. (Cricket Mitchell) –There is a Data Lists spreadsheet at the end of the workbook that you will not use Data Lists populate the pull-down menus in other spreadsheets 29

30. Palette of Measures Data Entry: HIT 30 There is a separate spreadsheet for Pre-HIT data, Post-HIT data, and Mid-HIT data In each spreadsheet, there is a separate field for each of the HIT Scale Scores, Subscale Scores, and Total Scores In the event of missing data, leave the fields blank/empty. Do not enter text into any of the fields. Mid-HITDate of Assessm ent Self-Report (ages 12-19) How I Think Questionnaire Mean Total, Scale, and Subscale Scores Client ID# Assessm entInterval HIT TotM HIT- OVM HIT- COV M HIT- SC M HIT- BO M HIT- MM M HIT- AW M HIT- ODM HIT- PA M HIT- LY M HIT- ST M HIT- AR M

31. Palette of Measures Data Entry: HIT 31 For agencies who will conduct mid-treatment assessments, indicate the Assessment Interval in the Mid-HIT spreadsheet by selecting from the available pull-down menu e.g., 1 st mid-treatment assessment, 2 nd mid-treatment assessment Mid- HITDate of Assess- ment Self-Report (ages 12-19) How I Think Questionnaire Mean Total, Scale, and Subscale Scores Client ID# Assess- ment Interval HIT Tot M HIT- OV M HIT- COV M HIT- SC M HIT- BO M HIT- MM M HIT- AW M HIT- OD M HIT- PA M HIT- LY M HIT- ST M HIT- AR M

32. But, before you enter any outcome data, you’ll enter Demographics & Service Delivery Information... But, before you enter any outcome data, you’ll enter Demographics & Service Delivery Information... 32

33. Palette of Measures Data Entry: Demographics & Services (1 of 2) 33 Use a unique identifier for Client ID# Categorical variables will have pull-down menus from which you’ll select an option (e.g., gender, ethnicity, language) Dates should be entered as xx/xx/xxxx Axis I diagnoses s/b the numeric DSM-IV code Therapist ID is optional Client Information: Primary(DSM-IV code) Client ID#DOBGenderEthnicityLanguagePrimary Axis ISecondary Axis ITherapist ID

34. Palette of Measures Data Entry: Demographics & Services (2 of 2) 34 Select Focus from pull-down menu (e.g., anxiety, depression) The shell will hold data for up to 4 foci, or treatment targets Enter Date of First Session The remaining fields are to be completed at the end of treatment targeting this particular focus (e.g., Date of Last, # Sessions) 2 levels of “Completed?” Treatment targeting this particular focus Overall service delivery Focus 1 of Treatment Date of Total #Completed (if Focus 1 not completed)Completed (if Services not completed) Focus1 First Session Last Session of SessionsFocus 1?ReasonServices?Reason

35. Palette of Measures Data Submissions (slide 1 of 3) Data submissions to CiMH will occur twice a year throughout the duration of the project –The end of each May (reflecting all clients served from the initiation of the project through the end of that April) –The end of each December (reflecting all clients served from the initiation of the project through the end of that November) –Note that this is the anticipated schedule; actual data submission dates may vary slightly An email notice will be sent to Palette of Measures site leads approximately one month in advance of each data submission deadline 35

36. Data Submissions (slide 2 of 3) Providers may choose from among the following methods for submitting their Palette of Measures Excel databases to CiMH: –Use YouSendIt, or another secure web-based transfer site, to submit data electronically YouSendIt (www.YouSendIt.com) is a vendor that supports the secure transfer of electronic data (encrypted and password- protected)www.YouSendIt.com –Mail a password-protected CD to CiMH and submit the password separately (via email or phone) –Email an encrypted, password-protected file(s) to CiMH and submit the password separately (via email or phone) 36

37. Data Submissions (slide 3 of 3) After data are submitted, sites continue to enter new data into the same database –Always reflects an ongoing, historical record of clients served through the Palette of Measures project Every effort is made to distribute reports within two months of each data submission –Aggregate and site/agency-specific reports 37

38. 38 Questions

39. 39 The End Contact Information Cricket Mitchell, PhD Email: cmitchell@cimh.orgcmitchell@cimh.org Cell phone: 858-220-6355