1. SPF SIG State-Level Evaluation COMMUNITY LEVEL INSTRUMENT (CLI): PART 2

2.  SAMHSA/CSAP cross-site evaluation overview  Purpose of the CLI: Parts 1 & 2  Data collection required for the CLI: Part 2  Roles and responsibilities  Timing and submission deadlines  Resources available WEBINAR AGENDA

3.  All communities funded by the SPF SIG are required to submit cross-site evaluation data.  Quick Cross-site evaluation terms in cross-site evaluation tools:  “Grantee” (State of AK who received the SPF SIG)  “Subrecipients” (6 funded communities in Alaska) SAMHSA CROSS-SITE EVALUATION INSTRUMENTS

4.  CLI: Part 1 –progress through the Strategic Prevention Framework  CLI: Part 2 –information about the specific interventions being implemented (adaptations, # reached, demographics)  PLI – participant change in alcohol consumption resulting from individual level interventions  Completed annually throughout the SPF SIG (Steps 1-5)  Completed quarterly for each intervention once implementation begins (Step 4)  Completed by participants of individual level interventions at pre, post, & follow-up (Step 4) SAMHSA CROSS-SITE EVALUATION INSTRUMENTS

5. 5 PURPOSE OF COLLECTING CLI: PART 1 AND 2 DATA  Assess  community implementation of the SPF process  changes in communities’ prevention infrastructure  types of interventions implemented  challenges to the implementation of the SPF process and alcohol prevention interventions  overall impact of the SPF SIG initiative  Identify  characteristics of community-level interventions  approaches that are most effective in producing population- level and participant-level outcomes.

6. Complete Q163 in CLI:Part 1 for each intervention being implemented in your Strategic Plan (We will individually work with you on this): 1.Intervention Name 2.Strategy Name (if a multi-component intervention) 3.CSAP Strategy Type  Alternative Drug-free Activities  Prevention Education  Problem Identification and Referral  Information Dissemination  Environmental Strategies  Community-Based Processes Complete CLI:Part 2 for each strategy identified in Q163 HOW ARE CLI: PART 1 AND 2 INSTRUMENTS LINKED TOGETHER?

7. 7 THINGS TO KEEP IN MIND  Interventions/ Strategies are planned activities intended to change alcohol related behavior (not general capacity building)  Defining strategies accurately in Q163 is important because it determines which questions are asked in CLI Part II.  Detailed guidance, including examples, can be found in the document “Detailed Guidance for Completing Q163.” Again, we will work closely with you to complete Q163!!

8.  Uses a combination of multiple choice, fill in the blank, and open-ended questions to gain information about your implemented strategies.  Completed quarterly online for each strategy https://www.pmrts.samhsa.gov/pmrts/  Some information is best tracked on a regular basis to provide accurate number in quarterly reports  Completed by Project Directors and Evaluators  May take up to 3 hours to complete each quarter CLI: PART 2

9. Strategy Information  Consumption pattern the strategy targets  Intervening variables the strategy targets  Date strategy first funded  Whether or not strategy is evidence-based  Criteria to determine whether it was evidence-based  Areas served by strategy (city, zip code, estimated population)  Cost of strategy during reporting period  Percent of funding from SPF SIG funds COMPONENTS OF CLI: PART 2

10. Strategy Adaptations  Whether or not it is a new strategy  Whether or not any adaptations were made:  Target population  Curriculum or manual  Dosage  Duration  Setting  Cultural appropriateness  Whether or not any participant satisfaction was conducted on cultural appropriateness COMPONENTS OF CLI: PART 2

11. Strategy Outcomes  Whether or not any outcome data was collected during reporting period (attitudes, behaviors, knowledge)  Sampling strategy for outcome data collection  Whether any outcome data was analyzed during reporting period  Which CSAP National Outcome Measures were collected during reporting period COMPONENTS OF CLI: PART 2

12. Strategy Subforms (one is required for each strategy depending on which CSAP 6 category your strategy falls under):  Alternative Drug-free Activities  Community-Based Processes  Prevention Education  Information Dissemination  Problem Identification and Referral  Environmental Strategies All are a little bit different in their content COMPONENTS OF CLI: PART 2

13.  IOM Classification (Universal, Selective, Indicated) To be reported quarterly for each strategy:  Number of participants reached in total  Number of new participants reached  Demographics of new participants reached  Number of females/males reached  Number of individuals reached within age categories (0-11, 12-17, 18- 20, 21-25, 26+)  Number of individuals reached within ethnic categories (American Indian/AK Native, Asian, Black or African American, White, Multiracial, Other, Unknown) CLI: PART 2 STRATEGY SUBFORMS: PREVENTION EDUCATION, ALTERNATIVE DRUG-FREE ACTIVITIES, AND PROBLEM IDENTIFICATION/REFERRAL

14.  Unduplicated count of individuals reached during fiscal reporting period (since last October 1 st)  percentage of individuals who were reached since last October 1st  % female/male  % within age categories (0-11, 12-17, 18-20, 21-25, 26+)  % reached within ethnic categories (American Indian/AK Native, Asian, Black or African American, White, Multiracial, Other, Unknown) Percentages should equal 100% CLI: PART 2 STRATEGY SUBFORMS: COMMUNITY BASED PROCESSES, ENVIRONMENTAL STRATEGIES, AND INFORMATION DISSEMINATION SUBFORMS

15. Some examples…… Social Marketing Environmental strategies  # of radio ads created  # of radio ads aired Enforcement Environmental Strategies  # of compliance checks conducted  # of sobriety checkpoints established OTHER QUESTIONS FOR ENVIRONMENTAL STRATEGIES

16. Reporting Period Subrecipient Data Entry State Evaluator Review & Feedback Final Subrecipient Edits Final Submission by State Evaluator Due Dates 1.1.13 - 3.30.13 4.1.134.15.134.25.134.30.13 4.1.13 - 6.30.13 7.1.137.15.137.25.137.30.13 7.1.13 - 9.30.13 10.1.1310.15.1310.25.1310.30.13 10.1.13 - 12.31.13 1.3.141.15.141.25.141.30.14 SUBMISSION DEADLINES FOR CLI: PART 2 …..Continue with same schedule throughout life of grant

17. 17  Local assistance by state evaluators  Submission deadlines schedule  Guidance documents  1) Question by Question (QxQ) guides for CLI Parts I and II  2) Detailed Guidance document for Q163  3) Checklist for Reviewing the CLI Parts I and II  Frequently Asked Questions  Federal Cross-site evaluation team collects grantee questions and disseminates FAQ document to grantees regularly.  CLI Response Report  Communities can print reports showing what responses they have given in previous reporting periods. WHAT ASSISTANCE IS AVAILABLE?

18. 18 WHAT ASSISTANCE IS AVAILABLE TO COMMUNITIES? (CONT’D)  On-line Instrument Training Video  Interactive videos demonstrating navigation through the web-based MRT system. http://kitusers.kithost.net/support/diticsupport/MRT/SPFSIGInstrument sGLICLIPLITrainingVideos/tabid/1114/Default.aspx http://kitusers.kithost.net/support/diticsupport/MRT/SPFSIGInstrument sGLICLIPLITrainingVideos/tabid/1114/Default.aspx  Help Line  If you experience technical difficulties in using the web-based system, contact the DITIC Help Desk by email or phone 1-888-DITIC-4-U (1-888-348-4248) Monday - Friday, 9:00 am - 8:00 pm EST