Cricket Mitchell, PhD CIMH Evaluation Consultant CiMH Palette of Measures Evaluation Training: Youth Outcome Questionnaires (YOQs)

Palette of Measures Evaluation: What You Will Need (slide 1 of 2) Palette of Measures Data Entry Shell v2 (Excel file) –Developed by CiMH and customized for each participating agency –Holds all data for clients served thru the Palette of Measures project –Demographics, service delivery information, pre- and post- outcome measure data Palette of Measures Data Dictionary v2 (Word document) –A guide for using the associated data entry shell –Defines each column in the excel file 2

Palette of Measures Evaluation: What You Will Need (slide 2 of 2) Outcome measures from the two-pronged approach –General Outcome Measure –Target-Specific Outcome Measure(s) –For example… Youth Outcome Questionnaire © (YOQ) and Youth Outcome Questionnaire – Self Report © (YOQ-SR) –Provided to agencies participating in the Palette of Measures project under CiMH’s statewide license 3

Overview of Training Brief Overview of Palette of Measures evaluation protocol Youth Outcome Questionnaires © (YOQ & YOQ-SR): General outcome measures of youth mental health functioning –Administration –Scoring –Clinical Utility Instructions for Palette of Measures data entry and data submissions –Data entry: YOQ/YOQ-SR –Data entry: Demographics & Services –Data submissions to CiMH 4

5 Brief Overview of Palette of Measures Evaluation Protocol

Outcome Assessment Palette of Measures providers will track outcomes using data from pre- and post- administrations of standardized measures of functioning Pre- and Post- a “dose” of treatment / an intervention interval –General measure of youth mental health functioning (e.g., YOQ/YOQ-SR, CANS, Ohio Scales) –Target-specific measure linked to focus of treatment/intervention (e.g., AQ, ECBI, PHQ-9, PTSD-RI, RCADS) Providers may choose to administer mid-course assessments as well –e.g., at 3-month intervals in usual care 6

A note about the use of standardized assessment measures… (slide 1 of 2) Assessment is the beginning of developing a relationship with the child and family –Demonstrates a desire to know what the child and family are experiencing –By incorporating standardized assessment measures of functioning, the efficiency and thoroughness of assessment is enhanced 7

A note about the use of standardized assessment measures… (slide 2 of 2) Using standardized assessment measures of functioning… –Assists in initial clinical impressions –Provides valuable information to guide treatment/interventions –Assesses sufficiency of treatment delivered –Demonstrates treatment-related improvements in child functioning 8

9 Youth Outcome Questionnaires © (YOQ & YOQ-SR)

CiMH YOQ & YOQ-SR Training Information on the administration, scoring, and clinical utility of the YOQ & YOQ-SR was obtained from each measure’s respective Administration and Scoring Manual published by OQ Measures, LLC 10

YOQ & YOQ-SR Description General measure of functioning Assesses the global mental health functioning of children –Parent/caregiver report for children ages 4-18 –Self-report for adolescents –Adult self-report is also available for 19+ Available in English and Spanish 11

YOQ & YOQ-SR Description 5-15 minutes to complete Sensitive to clinical change in short periods of time –Regardless of treatment modality, diagnosis, or discipline of the treating professional Valid and reliable Provided to partner agencies under CiMH’s statewide license –Partnering agencies sign a User’s Agreement and return it to CIMH 12

YOQ & YOQ-SR Description Parallel versions of the same measure 64 items 5-point Likert scale response options* –Never or Almost Never –Rarely –Sometimes –Frequently –Almost Always or Always *response values vary by item Six Scale Scores –Intrapersonal Distress (ID) –Somatic (S) –Interpersonal Relations (IR) –Social Problems (SP) –Behavioral Dysfunction (BD) –Critical Items (CI) Total Score 13

YOQ & YOQ-SR Administration Administer pre- and post- a dose of treatment, or an intervention interval –YOQ-SR completed by adolescent client –YOQ completed by mother/mother figure or father/father figure (when available) Both can be completed for clinical purposes; however, only one should be entered for outcome tracking purposes –Select parent/caregiver who spends most time with client 14

Let’s take a look at the YOQ-SR... 15

YOQ & YOQ-SR Administration “… during the past 7 days.” Ask parents and youth to fill out the questionnaires as honestly as possible –Informants can easily be influenced by the attitude of the person administering the scale –Let them know that this questionnaire will help you, as a clinician, better understand how the child is doing overall Ask parents and youth to complete all items 16

YOQ & YOQ-SR Scoring For each item, transfer the value corresponding to the selected response into the box at the right-hand side of the page titled, ‘For Office Use Only’ –Each item loads onto one of the six scales (e.g., ID, SP, IR) –Note that some items have negative response option values Sum the items in each scale on Side1 –Add the numbers in all boxes under the heading ID and enter that subtotal into the ID box at the bottom of the page –Repeat for each scale Sum the items in each scale on Side 2 Transfer the subtotals from Side 1 to Side 2 Sum the subtotals to determine Scale Scores Sum the Scale Scores to determine Total Score –Note that it is possible to have negative values for scores 17

YOQ & YOQ-SR Scoring Missing Data (items that are left blank) –If 5 or more items are missing, consider the questionnaire invalid –Substitute a mean item response for the missing item Determine in which scale the missing item belongs Add up the other items in that scale, and determine their average Substitute the average score for the missing response 18

Clinical Utility of the YOQ & YOQ-SR Total Score –Possible scores range from -16 to 240 –Clinical cutpoints 46 or higher on the YOQ 47 or higher on the YOQ-SR –Lower scores indicate more normative, non- clinical, aspects of general mental health functioning Elevations on certain scales indicate areas of specific distress for the child/youth 19

Clinical Utility of the YOQ & YOQ-SR Intrapersonal Distress (ID) Scale –Possible scores range from -4 to 68 –Clinical cutpoints 16 or higher on the YOQ 17 or higher on the YOQ-SR –Assesses the amount of emotional distress in the child/youth, including anxiety, depression, fearfulness, hopelessness, and self-harm –High scores indicate a considerable degree of intrapersonal distress in the child/youth 20

Clinical Utility of the YOQ & YOQ-SR Somatic (S) Scale –Possible scores range from 0 to 32 –Clinical cutpoints 5 or higher on the YOQ 6 or higher on the YOQ-SR –Indicates change in somatic distress or physical complaints –High scores indicate the parent/caregiver is aware of, or the youth is experiencing, a high number of somatic symptoms; while low scores indicate either absence or unawareness of them 21

Clinical Utility of the YOQ & YOQ-SR Interpersonal Relations (IR) Scale –Possible scores range from -6 to 34 –Clinical cutpoints 4 or higher on the YOQ 3 or higher on the YOQ-SR –Assesses issues relevant to the child/youth’s relationship with parents, other adults, and peers –High scores indicate significant interpersonal difficulty; while low scores reflect a cooperative, pleasant interpersonal demeanor 22

Clinical Utility of the YOQ & YOQ-SR Social Problems (SP) Scale –Possible scores range from -2 to 68 –Clinical cutpoints 3 or higher on the YOQ & YOQ-SR –Assesses problems that are socially related including aggression and delinquency –A feature of these items is that they are slow to change; whereas, content tapped by many of the other scales often changes over a period of time as a result of treatment intervention 23

Clinical Utility of the YOQ & YOQ-SR Behavioral Dysfunction (BD) Scale –Possible scores range from -4 to 40 –Clinical cutpoints 12 or higher on the YOQ 11 or higher on the YOQ-SR –Assesses inattention, hyperactivity, impulsivity, concentration, ability to organize tasks, and ability to handle frustration 24

Clinical Utility of the YOQ & YOQ-SR Critical Items (CI) Scale –Possible scores range from 0 to 36 –Clinical cutpoints 5 or higher on the YOQ 6 or higher on the YOQ-SR –Assesses areas such as paranoia, obsessive-compulsive behaviors, hallucination, delusions, suicide, mania, and eating disorders 25

Clinical Utility of the YOQ & YOQ-SR Assesses a variety of specific areas of difficulty in child/youth mental health functioning Assists in initial clinical impressions Provides valuable information to guide treatment/interventions Comparisons of pre/post scores reveal areas of clinical improvement as well as areas of potential unmet need 26

Summary of YOQ/YOQ-SR Score Ranges and Clinical Cutpoints 27 YOQ/YOQ-SR Scale Range of possible scores Clinical Cutpoint for YOQ Clinical Cutpoint for YOQ-SR Intrapersonal Distress (ID) -4 to Somatic (S) 0 to 3256 Interpersonal Relations (IR) -6 to 3443 Social Problems (SP) -2 to 3033 Behavioral Dysfunction (BD) -4 to Critical Items (CI) 0 to 3656 Total Score -16 to

28 Palette of Measures Data Entry and Data Submissions

Palette of Measures Data Entry Shell There is a separate spreadsheet in the excel workbook (aka database) for each type of data: –Demographics & Services –Pre- General outcome measure –Post- General outcome measure –Pre- Target-specific measure(s) –Post- Target-specific outcome measure(s) Specific outcome measure spreadsheets included in each agency’s database varies across Palette of Measures project participants 29

Palette of Measures Data Entry Shell In addition to the spreadsheets that hold data... –There is an Instructions spreadsheet Basic data entry instructions Contact information for T.A. (Cricket Mitchell) –There is a Data Lists spreadsheet at the end of the workbook that you will not use Data Lists populate the pull-down menus in other spreadsheets 30

Palette of Measures Data Entry: YOQ/YOQ-SR 31 There is a separate spreadsheet for Pre-YOQ data, Post-YOQ data, and Mid-YOQ data In each spreadsheet, there is a separate field for each of the YOQ and YOQ-SR Scale Scores and Total Scores In the event of missing data, leave the fields blank/empty. Do not enter text into any of the fields. Pre-YOQs PARENT/CAREGIVER REPORT: (ages 4-18) Date ofYOQ (Scale Scores and Total Score) Client ID#AssessmentYOQ1-IDYOQ1-SYOQ1-IRYOQ1-SPYOQ1-BDYOQ1-CIYOQ1-Total …

Palette of Measures Data Entry: YOQ/YOQ-SR 32 For agencies who will conduct mid-treatment assessments, indicate the Assessment Interval in the Mid-YOQs spreadsheet by selecting from the available pull-down menu e.g., 1 st mid-treatment assessment, 2 nd mid-treatment assessment Mid-YOQs Date ofAssessmentYOQ (Scale Scores and Total Score) Client ID#AssessmentInterval YOQM- ID YOQM- S YOQM- IR YOQM- SP YOQM- BD YOQM- CI YOQM- Total

But, before you enter any outcome data, you’ll enter Demographics & Service Delivery Information... But, before you enter any outcome data, you’ll enter Demographics & Service Delivery Information... 33

Palette of Measures Data Entry: Demographics & Services (1 of 2) 34 Use a unique identifier for Client ID# Categorical variables will have pull-down menus from which you’ll select an option (e.g., gender, ethnicity, language) Dates should be entered as xx/xx/xxxx Axis I diagnoses s/b the numeric DSM-IV code Therapist ID is optional Client Information: Primary(DSM-IV code) Client ID#DOBGenderEthnicityLanguagePrimary Axis ISecondary Axis ITherapist ID

Palette of Measures Data Entry: Demographics & Services (2 of 2) 35 Select Focus from pull-down menu (e.g., anxiety, depression) The shell will hold data for up to 4 foci, or treatment targets Enter Date of First Session The remaining fields are to be completed at the end of treatment targeting this particular focus (e.g., Date of Last, # Sessions) 2 levels of “Completed?” Treatment targeting this particular focus Overall service delivery Focus 1 of Treatment Date of Total #Completed (if Focus 1 not completed)Completed (if Services not completed) Focus1 First Session Last Session of SessionsFocus 1?ReasonServices?Reason

Palette of Measures Data Submissions (slide 1 of 3) Data submissions to CiMH will occur twice a year throughout the duration of the project –The end of each May (reflecting all clients served from the initiation of the project through the end of that April) –The end of each December (reflecting all clients served from the initiation of the project through the end of that November) –Note that this is the anticipated schedule; actual data submission dates may vary slightly An notice will be sent to Palette of Measures site leads approximately one month in advance of each data submission deadline 36

Data Submissions (slide 2 of 3) Providers may choose from among the following methods for submitting their Palette of Measures Excel databases to CiMH: –Use YouSendIt, or another secure web-based transfer site, to submit data electronically YouSendIt ( is a vendor that supports the secure transfer of electronic data (encrypted and password- protected) –Mail a password-protected CD to CiMH and submit the password separately (via or phone) – an encrypted, password-protected file(s) to CiMH and submit the password separately (via or phone) 37

Data Submissions (slide 3 of 3) After data are submitted, sites continue to enter new data into the same database –Always reflects an ongoing, historical record of clients served through the Palette of Measures project Every effort is made to distribute reports within two months of each data submission –Aggregate and site/agency-specific reports 38

39 Questions

40 The End Contact Information Cricket Mitchell, PhD Cell phone: