1. NJ - 1 Performance Measurement Reporting Development Services Group, Inc. Don Johnson For more information contact Development Services Group, Inc. 7315 Wisconsin Avenue, Suite 800E, Bethesda, MD 20814 Phone: 301-951-0056 Fax: 301-951-3324 Toll-free: 1-877-5-TITLE-5 www.dsgonline.com

2. NJ - 2 Why Use Performance Measurement?  New requirement from OJJDP, similar to other federal funding sources – comply with GRPA  Gives national standards for a move toward accountability  Provides an additional program monitoring tool for PPB

3. Priority for Evidence-Based Programs Priority for Evidence-Based Programs and Increased Accountability  Title II of the JJDP Act of 2002 states, “designated state agency will, to the extent practicable, give priority in funding to programs and activities that are based on rigorous, systematic, and objective research that is scientifically based.”  Title II also states that the state agency shall “not expend funds to carry out a program if such program, during the preceding 2-year period, fails to demonstrate, before the expiration of such 2- year period, that such program achieved substantial success in achieving the goals specified.”

4. Priority for Evidence-Based Programs Priority for Evidence-Based Programs and Increased Accountability  Title V of the JJDP Act of 2002 states the Administrator shall give priority to “developing data-driven prevention plans, employing evidence-based prevention strategies, and conducting program evaluations to determine impact and effectiveness.”

5. What Is Performance Measurement? Performance measurement is a system of tracking progress in accomplishing specific goals, objectives, activities and outcomes. Performance measurement:  Is directly related to program goals and objectives  Measures progress quantitatively – outputs and outcomes  Is not exhaustive of all possible measures  Provides a “temperature” reading—it may not tell you everything you want to know but provides a quick and reliable gauge of selected results NJ - 5

6. Evaluation is a more formal process, often done by an experienced researcher outside the program. Performance Measurement is less formal and can be a component of a formal evaluation. Evaluation versus Performance Measurement NJ - 6

7. Title V Program Areas Funds for Title V Programs must be for:  At-risk youth to prevent them from entering the juvenile justice system, or  Early intervention programs for juveniles with first-time and non-serious offenses to keep them out of the juvenile justice system. Applications under Title V must propose to use Title V funds for these populations only. NJ - 7

8. Title V Program Areas  Select the Program Area that most closely aligns with your proposed Strategy/ Program based on the prioritized risk factors and problem indicators.  If funded, Juvenile Justice Commission staff will help verify the correct Program Area to use for Performance Reporting. There are 18 program areas, as listed in Section VIII of the NOAF, and described on the Title V web page http://dsgonline.com/ Program_Logic_Model/titlev_pm.htmhttp://dsgonline.com/ NJ-8

9. Title V Program Areas  Delinquency Prevention – directed at youth “at risk” of becoming delinquent. Program types: parent education, anger management, conflict resolution, youth development, improving social skills, etc.  School Programs – designed to prevent truancy, suspension and expulsion; including school safety programs.  Gangs – directed at reducing gang-related activities. Program Areas most likely to be selected NJ - 9

10. Outputs Versus Outcomes Outputs are products of program implementation/ activities (volume of work accomplished – numbers). Outcomes are benefits or changes that result from the program. Related to changes in Behavior, Attitudes, Skills or Knowledge (BASK). 2 types of outcomes:  Short-term Outcomes: those that occur during a youth’s involvement with the program or by the time of program completion.  Long-term Outcomes: those that occur 6 months to 1 year after a youth completes the program. NJ - 10

11. NJ - 11 Output and Outcome Quiz Individually Review the hand out Record your own answers At tables/In small groups Compare answers Be prepared to report out

12. NJ - 12 A graphic representation that clearly identifies and lays out the logical relationships between program objectives, activities, outputs, and outcomes. ObjectivesActivitiesOutputs Outcomes Short-term Long-term Logic Model

13. NJ - 13 Clarifies assumptions and relationships between program efforts and expected results Supports development of effective proposals Helps specify what to measure to verify progress and impact Guides assessment of underlying assumptions and self-correction Benefits of a Logic Model

14. NJ - 14

15. Sample Grid Chart: Delinquency Prevention Output Performance Measures NJ- 15

16. Sample Grid Chart: Delinquency Prevention Outcome Performance Measures NJ - 16

17. NJ - 17

18. NJ - 18

19. Logic Models and Grids These logic models and all of the logic models for each of the 18 Title V program areas can be downloaded from the OJJDP performance measurement web site at http://dsgonline.com/Program_Logic_Model /titlev_pm.htm. NJ - 17

20. Reporting Requirements From the applicable Program Area, successful applicants will report the following: Output Measures  It is required to report on all of the mandatory output measures, and at least two of the non- mandatory output measures. Outcome Measures  It is required to report on all of the mandatory short-term and long-term outcome measures, and at least two of the non-mandatory outcome measures (either short- or long-term). NJ - 20

21. 1. Amount of funds awarded (state reports) 2. Number of program youth served:  An unduplicated count of the number of youths served by the program during the reporting period. Definition of the number of youth served for a reporting period is the number of program youth carried over from previous reporting period, plus new admissions during the reporting period.  A spreadsheet tool is available on the Performance Measurement web site (hard copy in packet). Mandatory Output Measure NJ - 21

22. Number and percent of youth completing program requirements:  The number and percent of program youth who have successfully fulfilled all program obligations and requirements. Program requirements will vary by program but should be a predefined list of requirements or obligations that clients must meet prior to program completion. Program records are the preferred data source. Mandatory Short-term Outcome Measure NJ - 22

23. Number and percent of program youth exhibiting desired change in targeted behaviors (select one or more that apply to your program):  Substance use  School attendance  Antisocial behavior  Family relationships  Pregnancies Mandatory Outcome Measure Short-term and Long-term NJ - 23

24. Logic Model Example NJ - 24

25. Logic Model Exercise NJ - 25

26. NJ - 26 Elements of the data collection plan  MEASURES: Specify observable and measurable characteristics. Include all performance measures  DEFINITION: Specify what you will use to describe or represent each measure (indicators)  FREQUENCY OF COLLECTION: Specify regular intervals when data will be collected Write Data Collection Plan

27. NJ - 27 Elements of the data collection plan  RESPONSIBLE FOR COLLECTION: Specify who will collect the data  INSTRUMENT/DATA SOURCE: Specify the data you will use to measure each indicator  LOCATION OF DATA: Specify where data will come from  HOW PROCESSED OR RETRIEVED: Data collection method to be used – MOU Required? Write Data Collection Plan (cont’d)

28. Data Collection Example NJ - 28

29. Data Collection Form Exercise NJ - 29

30. NJ - 30 Performance Measurement Information Additional information on OJJDP Performance Measures for Title V available at: www.dsgonline.com/performance_measures.htm www.dsgonline.com/performance_measures.htm

31. NJ - 31 1. Identify an evidence-based program to be implemented to address your problem 2. Incorporate performance measurement into your proposal Select a program area (corresponding to your proposed program/strategy) Construct program logic model Review mandatory output and outcome measures Select non-mandatory output and outcome measures Develop data collection plan Next Steps