1. Cricket Mitchell, PhD CIMH Evaluation Consultant CiMH Palette of Measures Evaluation Training: Aggression Questionnaire (AQ)

2. Palette of Measures Evaluation: What You Will Need (slide 1 of 2) Palette of Measures Data Entry Shell v2 (Excel file) –Developed by CiMH and customized for each participating agency –Holds all data for clients served thru the Palette of Measures project –Demographics, service delivery information, pre- and post- outcome measure data Palette of Measures Data Dictionary v2 (Word document) –A guide for using the associated data entry shell –Defines each column in the excel file 2

3. Palette of Measures Evaluation: What You Will Need (slide 2 of 2) Outcome measures from the two-pronged approach –General Outcome Measure –Target-Specific Outcome Measure(s) –For example… Aggression Questionnaire © (AQ) –Copyrighted and published by Western Psychological Services (WPS) Must be purchased by interested Palette of Measures project participating agencies 3

4. Overview of Training Brief Overview of Palette of Measures evaluation protocol Aggression Questionnaire © : Target-specific measure for disruptive behavior –Administration –Scoring –Clinical Utility Instructions for Palette of Measures data entry and data submissions –Data entry: AQ –Data entry: Demographics & Services –Data submissions to CiMH 4

5. 5 Brief Overview of Palette of Measures Evaluation Protocol

6. Outcome Assessment Palette of Measures providers will track outcomes using data from pre- and post- administrations of standardized measures of functioning Pre- and Post- a “dose” of treatment / an intervention interval –General measure of youth mental health functioning (e.g., YOQ/YOQ-SR, CANS, Ohio Scales) –Target-specific measure linked to focus of treatment/intervention (e.g., AQ, ECBI, PHQ-9, PTSD-RI, RCADS) Providers may choose to administer mid-course assessments as well –e.g., at 3-month intervals in usual care 6

7. A note about the use of standardized assessment measures… (slide 1 of 2) Assessment is the beginning of developing a relationship with the child and family –Demonstrates a desire to know what the child and family are experiencing –By incorporating standardized assessment measures of functioning, the efficiency and thoroughness of assessment is enhanced 7

8. A note about the use of standardized assessment measures… (slide 2 of 2) Using standardized assessment measures of functioning… –Assists in initial clinical impressions –Provides valuable information to guide treatment/interventions –Assesses sufficiency of treatment delivered –Demonstrates treatment-related improvements in child functioning 8

9. 9 Aggression Questionnaire © (AQ)

10. CiMH AQ Training Information on the administration, scoring, and clinical utility of the AQ was obtained from the measure’s Manual, written by Arnold H. Buss, Ph.D., and W.L. Warrant, Ph.D., and published by Western Psychological Services (WPS) 10

11. Aggression Questionnaire © (AQ) Description Target-specific measure for disruptive behavior Measures youth tendencies to respond to situations in an aggressive manner –Youth self-report ages 9-18 –Can also be used as adult self-report for ages 19+ (separate norms) 10 minutes to complete 11

12. Aggression Questionnaire © (AQ) Description Written at a 3 rd grade reading level Valid and reliable Copyrighted and published by Western Psychological Services (WPS) –After purchase of the manual (~$62) the cost per use ~$1.80 –http://portal.wpspublish.com/portal/page?_pageid =53,70400&_dad=portal&_schema=PORTALhttp://portal.wpspublish.com/portal/page?_pageid =53,70400&_dad=portal&_schema=PORTAL 12

13. AQ © Description 34 items 5-point Likert scale response options –Not at all like me (1) –A little like me (2) –Somewhat like me (3) –Very much like me (4) –Completely like me (5) Five Scale Scores –Physical Aggression (PHY) –Verbal Aggression (VER) –Anger (ANG) –Hostility (HOS) –Indirect Aggression (IND) Total Score 13

14. Example: Items from the AQ © My friends say that I argue a lot. I may hit someone if he or she provokes me. I have threatened people I know. I wonder what people want when they are nice to me. I have become so mad I have broken things. 14

15. AQ Administration Administer pre- and post- a dose of treatment, or an intervention interval, focused on disruptive behavior –AQ completed by client –Some agencies may choose to administer mid-treatment assessments as well 15

16. Let’s take a look at the AQ... (not distributed due to copyright laws) 16

17. AQ Scoring Tear off the perforated edge to open packet –The youth’s circled responses will show on the inside, and will reflect the value associated with the response First, calculate the Inconsistent Responding Index (INC) –For 12 pairs of items, record the value of each response in the appropriate box –If the difference between the pair of scores is greater than 1, check the box off to the right Sum the checked boxes to obtain the INC Score INC Scores of 5 or greater indicate inconsistency in responding, and scores should be interpreted with caution 17

18. AQ Scoring Next, go the AQ Scoring Worksheet and transfer the value of each circled response into the box at the right-hand side –Each item loads onto one of the five scales (e.g., PHY, VER, ANG) Sum all responses in each scale to determine Scale Scores Sum the Scale Scores to determine Total Score Raw scores can be plotted on the Profile Sheet, which will show how the youth’s responses compare to typical youth responses 18

19. AQ Scoring Missing Data (items that are left blank) –If 5 or more items are missing, consider the questionnaire invalid OR 2 items from any one Scale –Substitute the median item response for the missing item The median item response is based on the standardization sample and represents the response that divided the sample in half The median response is bolded on the AQ Scoring Worksheet 19

20. Clinical Utility of the AQ Total Score –Possible scores range from 34 to 170 –Clinical cutpoint 110 or higher –Total Scores are reflective of the general level of anger and aggression reported by youth –High scores could reflect youth with relatively few, but extremely intense angry or aggressive episodes; as well as youth with less intense anger or aggression, but across relatively chronic circumstances It is important to examine elevations on subscales to understand the profile of the youth’s experiences 20

21. Clinical Utility of the AQ Physical Aggression (PHY) Scale –Possible scores range from 8 to 40 –Clinical cutpoints differ by gender 30 or higher for Males 26 or higher for Females –Assesses the use of physical force when expressing anger or aggression –High scores indicate that the youth is reporting a lack of ability to control urges toward physical aggression 21

22. Clinical Utility of the AQ Verbal Aggression (VER) Scale –Possible scores range from 5 to 25 –Clinical cutpoint 18 or higher –Assesses the use of quarrelsome and hostile speech –High scores indicate that the youth is aware of a tendency to be more argumentative than most people Often aroused to anger by situations perceived as unfair 22

23. Clinical Utility of the AQ Anger (ANG) Scale –Possible scores range from 7 to 35 –Clinical cutpoint 24 or higher –Assesses the use of self-control and aspects of arousal related to anger –High scores are often associated with irritability, frustration, emotional lability, and temperamental gesturing 23

24. Clinical Utility of the AQ Hostility (HOS) Scale –Possible scores range from 8 to 40 –Clinical cutpoint 26 or higher –Assesses attitudes of bitterness, social alienation, and paranoia –High scores on the Hostility Scale magnify the implications of elevations on other scales because the youth is in a state of social alienation, and thus is not well able to take into account the needs or feelings of others 24

25. Clinical Utility of the AQ Indirect Aggression (IND) Scale –Possible scores range from 6 to 30 –Clinical cutpoint 20 or higher –Assesses the tendency to express anger in actions that avoid direct confrontation –High scores may be reflective of youth experiencing chronic frustration and disrupted peer relationships, largely as a result of an inability to satisfactorily resolve circumstances that lead to anger 25

26. Clinical Utility of the Aggression Questionnaire © Assesses a variety of specific aspects of youth self-reported anger and aggression –e.g., physical, verbal, indirect Informs which aspects of aggressive tendencies youth have the greatest need for intervention Comparisons of pre/post scores demonstrate treatment-related outcomes in specific aspects of anger and aggression 26

27. Summary of AQ Score Ranges and Clinical Cutpoints 27 AQ Scale Range of possible scores Clinical Cutpoint Physical Aggression (PHY)8 to 40 Males: 30 Females: 26 Verbal Aggression (VER)5 to 2518 Anger (ANG)7 to 3524 Hostility (HOS)8 to 4026 Indirect Aggression (IND)6 to 3520 Total Score34 to 170110

28. 28 Palette of Measures Data Entry and Data Submissions

29. Palette of Measures Data Entry Shell There is a separate spreadsheet in the excel workbook (aka database) for each type of data: –Demographics & Services –Pre- General outcome measure –Post- General outcome measure –Pre- Target-specific measure(s) –Post- Target-specific outcome measure(s) Specific outcome measure spreadsheets included in each agency’s database varies across Palette of Measures project participants 29

30. Palette of Measures Data Entry Shell In addition to the spreadsheets that hold data... –There is an Instructions spreadsheet Basic data entry instructions Contact information for T.A. (Cricket Mitchell) –There is a Data Lists spreadsheet at the end of the workbook that you will not use Data Lists populate the pull-down menus in other spreadsheets 30

31. Palette of Measures Data Entry: AQ 31 There is a separate spreadsheet for Pre-AQ data, Post-AQ data, and Mid-AQ data In each spreadsheet, there is a separate field for each of the AQ Scale Scores and Total Scores In the event of missing data, leave the fields blank/empty. Do not enter text into any of the fields. Pre-AQ Self-Report (ages 9+) Date ofAggression Questionnaire Raw Scores Client ID#AssessmentAQTot1AQ-PA1AQ-VA1AQ-A1AQ-H1AQ-IN1

32. Palette of Measures Data Entry: AQ 32 For agencies who will conduct mid-treatment assessments, indicate the Assessment Interval in the Mid-AQ spreadsheet by selecting from the available pull-down menu e.g., 1 st mid-treatment assessment, 2 nd mid-treatment assessment Mid-AQ Self-Report (ages 9+) Date ofAssessment Aggression Questionnaire Raw Scores Client ID#AssessmentIntervalAQTotMAQ-PAMAQ-VAMAQ-AMAQ-HMAQ-INM

33. But, before you enter any outcome data, you’ll enter Demographics & Service Delivery Information... But, before you enter any outcome data, you’ll enter Demographics & Service Delivery Information... 33

34. Palette of Measures Data Entry: Demographics & Services (1 of 2) 34 Use a unique identifier for Client ID# Categorical variables will have pull-down menus from which you’ll select an option (e.g., gender, ethnicity, language) Dates should be entered as xx/xx/xxxx Axis I diagnoses s/b the numeric DSM-IV code Therapist ID is optional Client Information: Primary(DSM-IV code) Client ID#DOBGenderEthnicityLanguagePrimary Axis ISecondary Axis ITherapist ID

35. Palette of Measures Data Entry: Demographics & Services (2 of 2) 35 Select Focus from pull-down menu (e.g., anxiety, depression) The shell will hold data for up to 4 foci, or treatment targets Enter Date of First Session The remaining fields are to be completed at the end of treatment targeting this particular focus (e.g., Date of Last, # Sessions) 2 levels of “Completed?” Treatment targeting this particular focus Overall service delivery Focus 1 of Treatment Date of Total #Completed (if Focus 1 not completed)Completed (if Services not completed) Focus1 First Session Last Session of SessionsFocus 1?ReasonServices?Reason

36. Palette of Measures Data Submissions (slide 1 of 3) Data submissions to CiMH will occur twice a year throughout the duration of the project –The end of each May (reflecting all clients served from the initiation of the project through the end of that April) –The end of each December (reflecting all clients served from the initiation of the project through the end of that November) –Note that this is the anticipated schedule; actual data submission dates may vary slightly An email notice will be sent to Palette of Measures site leads approximately one month in advance of each data submission deadline 36

37. Data Submissions (slide 2 of 3) Providers may choose from among the following methods for submitting their Palette of Measures Excel databases to CiMH: –Use YouSendIt, or another secure web-based transfer site, to submit data electronically YouSendIt (www.YouSendIt.com) is a vendor that supports the secure transfer of electronic data (encrypted and password- protected)www.YouSendIt.com –Mail a password-protected CD to CiMH and submit the password separately (via email or phone) –Email an encrypted, password-protected file(s) to CiMH and submit the password separately (via email or phone) 37

38. Data Submissions (slide 3 of 3) After data are submitted, sites continue to enter new data into the same database –Always reflects an ongoing, historical record of clients served through the Palette of Measures project Every effort is made to distribute reports within two months of each data submission –Aggregate and site/agency-specific reports 38

39. 39 Questions

40. 40 The End Contact Information Cricket Mitchell, PhD Email: cmitchell@cimh.orgcmitchell@cimh.org Cell phone: 858-220-6355