Presentation is loading. Please wait.

Presentation is loading. Please wait.

Balancing Evaluation and Program Needs REAP Symposium 2009.

Similar presentations


Presentation on theme: "Balancing Evaluation and Program Needs REAP Symposium 2009."— Presentation transcript:

1 Balancing Evaluation and Program Needs REAP Symposium 2009

2 Agenda Introduction EYA Organizational Overview Performance and Quality Improvement Division PQI Charge from the EYA Leadership Team Method: Cross-functional Teams Areas of Focus Balanced Scorecard/ Key Performance Indicators Master Quality Plan and Toolkit Totally Integrated Electronic Record Report Functionality Integrating Processes Q and A “It is one of the most beautiful compensations of this life that no person can sincerely try to help another without helping oneself.” ~Ralph Waldo Emerson

3 January 27, A Proven History Pioneering “Firsts” 1968: First provider of an outdoor therapeutic program in Florida. 1969: First provider of an outdoor therapeutic program for Girls in the Southeast. 1982: First Private Provider of State Training School in Southeast. 1985: Funded/Implemented Demonstration Project for ReEntry in Florida. 1991: Funded and Implemented Early Intervention Demonstration Project (Hi-Five). 1994: One of First Private Providers of High Risk Programming for Girls in Florida. 1996: External Review of Data and Programs and Outcomes by Clemson and Harvard. 1996: First Outdoor Therapeutic Programs to be Accredited by COA. 2001: Largest private provider of conditional release programming in Florida. 2007: Achieves agency-wide COA accreditation. 2008: EYA Becomes Lead Agency Provider.

4 Geographical Reach Currently operating in 9 states About 40 programs nationwide Currently in TX, LA, FL, TN, GA, NH, VT, RI, and NC Currently targeting six new states for expansion

5 Eckerd Youth Alternatives Mission: Develop and share programs that promote the well-being of children and serve at-risk youth and their families. Vision: Ensure that each child has the opportunity to succeed.

6 We base our concepts upon a belief in God and the uniqueness and inherent worth of each individual. We believe that youth always come first. We strive to assist youth in developing their potential to achieve balanced physical, intellectual, emotional, and spiritual strength. We believe that youth and their families are our partners in treatment. We seek to build a culturally diverse organization that honors the cultural diversity of the youth, families and communities we serve. We embrace innovation, flexibility, reflection, and continuous improvement. We thrive in a culture of learning, personal development and professional growth. We value all members of our staff and strive to provide the highest level of care and support to each. We hold ourselves accountable for achieving superior outcomes. We strive for excellence in all of our endeavors. EYA Values

7 Continuum of Services for Youth and Families

8 January 27,  Safety  Quality Assurance  Performance Improvement & Planning  Evidence-Based Practices & Outcomes Areas of Focus QUALITY STAFF  Targeted Recruitment  Professional Development  Retention FINANCIAL  Fiscal Strength & Discipline  Accurate Analysis & Forecasting  Capital Allocation & Cash Management EXTERNAL RELATIONS  Donor Cultivation  Communications & Publications  Alumni Relations GROWTH AND TRANSFORMATION  Mission-Driven Reinvestment  Business ViabilityAssessment  Marketplace Analysis  Targeted Retooling & Expansion

9 Performance & Quality Improvement As a performance-based organization, we clearly define, collect, validate, analyze and communicate our outcomes to make informed decisions based on evidence

10 Cross-functional Teams Balanced Scorecard Master Quality Improvement Plan with Toolkit Data Collection (Totally Integrated Electronic Record) and Report Functionality

11 Balanced Scorecard Team The Balance Scorecard™ is a performance management tool that tracks the most important goals and initiatives at an organization. It measures financial and operational activities to ensure that they are aligned with the organizational mission, vision and strategies. Summary Benefit Ensures consistency and thoughtfullness in the methodologies used to measure outcomes and provides feedback to all levels of the organization. Tasks Prepare list of the different scorecards Select the measurement criteria Identify data source, instrument, collection, analysis, reporting for each measurement Create communication tool Effort: Medium Impact: High

12 Balanced Scorecard Basics Developed by Robert Kaplan and David Norton (1990)Developed by Robert Kaplan and David Norton (1990) Based on research exploring organizational performance measurementBased on research exploring organizational performance measurement Communication ToolCommunication Tool Measurement SystemMeasurement System Strategic Management SystemStrategic Management System

13 Balanced Scorecard Perspectives Customer: Who are our customers? What do our customers expect or demand from us? What is our value proposition in serving them?Customer: Who are our customers? What do our customers expect or demand from us? What is our value proposition in serving them? Internal Process: What must we excel at in order to continue to add value for our customers? What are the processes we have/need to best execute our strategies?Internal Process: What must we excel at in order to continue to add value for our customers? What are the processes we have/need to best execute our strategies? Employee Learning: What is the level/content of employee skills needed to meet our mission/value? What information systems needs to support these processes? What organizational climate (culture) supports this success?Employee Learning: What is the level/content of employee skills needed to meet our mission/value? What information systems needs to support these processes? What organizational climate (culture) supports this success? Financial: What are the financial perspectives that would help us understand the effectiveness and efficiency of our financial processes?Financial: What are the financial perspectives that would help us understand the effectiveness and efficiency of our financial processes?

14 Balanced Scorecard Perspectives Balanced Scorecard CurrentDepartments Customer/ClientQualityOperations External Relations Employee Learning Staff Human Resources Internal Process GrowthDevelopment PQI Education Facilities Services Admissions FinancialFinancial Treasury/Risk Mgt Accounting

15 Performance Measure Criteria Linked to strategyLinked to strategy Easily understoodEasily understood Link in a chain of cause and effectLink in a chain of cause and effect Updated frequentlyUpdated frequently AccessibleAccessible Measurable (averages, avoiding initiatives)Measurable (averages, avoiding initiatives) Is it driving the “right” behavior?Is it driving the “right” behavior?

16 Evaluation Plan

17 Sample Evaluation Plan

18 Balanced Scorecard Examples Customer: 1) Percentage of youth completing an EYA program increasing mental health functioning by at least 10% 2) Percentage of youth completing an EYA program reporting continued positive wellbeing impact (defined by education, mental health, behavioral, including recidivism, employment, and living status.) Internal Process: 1) Number of instances of TIER being “off-line” during normal working hours 2) Cost of fundraising expenses incurred by EYA in a fiscal year will be at least $1/$1. Employee Learning: 1) Voluntary employee turnover will be maintained at 50% or less. 2) Percentage of participants’ managers finding an increase in productivity, initiative, confidence, or measure directly related to the subject of the training. Financial: 1) Overall administrative overhead is less than 10%. 2) Total revenue minus total expense is at least 3% of total revenue.

19 Master QI Plan Team Summary Benefit Effort: High Impact: High A comprehensive document that establishes organizational strategies for identifying critical areas, assessing, developing and implementing quality assurance objectives and procedures in a consistent and methodical way. Allows us to deliver top quality care to our clients, provide excellent, reliable data and information to our internal and external customers and measures outcomes for continuous quality improvement. Assess current systems and tools Cross functional needs assessment Define requirements, resources and deliverables Implementation strategies that include training and ongoing support. Tasks

20 Data and Reporting Team Effort: Medium Impact: Medium Summary Benefit Several key tools are used to track and report the quality monitoring and response information, including PBViews and TIER computer systems. This initiative will re-deploy these systems and train the individuals who will be using the systems. Appropriate tools to capture and report large amounts of data is critical in the monitoring of a diverse group of programs in multiple locations. Having effective tools will minimize the time required by staff and leadership to report and analyze quality data. Deploy software with clear expectations on who and when data will be entered Engage vendor to re-deploy TIER system Create a training plan for all users of the different systems to ensure they are comfortable using the system Tasks

21 Business Users Summary Information Transactional Details Transactional Details Management Portal/Intranet Everyone

22 PBViews Sample Maps 18 Canned Reports Briefing Books Internet Publishing Performance “View” BIRT/e.Spreadsheet BIRT Report Studio

23 Annual Descriptive Summary

24 Youth Outcomes Satisfaction Survey measures perceptions of accountability, consequences, education, empathy, group, home/transition, staff 35% youth and 14% parent respondent rate – not in parity with EYA demographics Respondents are only from residential programs – primarily NC 64% of youth completing an EYA program reported positive satisfaction of services 65% of parents of youth completing an EYA program reported that their child made positive life changes 62% of parents reported that their relationship with their child improved

25 Youth Outcomes WJII: Measures math (calculation, math fluency, applied problems) and reading (letter-word identification, reading fluency, passage comprehension) Average length of stay ranges from months Youth receive a minimum of 250 instructional minutes of math and 250 instructional minutes of reading/week 90% youth respondent rate (Pre-Post Test) At intake, 53% were below grade level for reading and 79% were below grade level for math 82% of youth completing an EYA program improved reading levels an average of 1.6 grade levels 78% of youth completing an EYA program improved math levels an average of 1.2 grade levels

26 Youth Outcomes Matson’s Evaluation of Social Skills for Youngsters 78% youth respondent rate (Pre-Post Test) Average length of stay ranges from months Demographics are in parity with EYA population 42% of youth completing an EYA program improved social skills by an average of 15%. Less than 20% of youth in ReEntry or community supervision programs scored in the normal range for “appropriate” social skills. In this population, aggression and overconfidence scored high as well. More than 80% of you in private academies and day schools scored in the normal range for “appropriate” social skills. In this population, impulsivity scored high as well.

27 Youth Outcomes ASEBA Youth Self Report measures mental health functioning Average length of stay ranges from months 82% youth respondent rate (Pre-Post Test) Demographics are in parity with EYA population 47% of youth completing an EYA program scored in the normal range at the beginning of the program and 72% scored in the normal range at the end of the program Other than the commonality of clinical scores in rule breaking and externalizing behaviors, youth varied greatly in other constructs

28 Integrating into Practice Safety: evaluating frequency and areas of vulnerability, incident cost metrics, promoting effective strategies & impact, cross-functional teams, organizational safety program QI: educating organization, formalizing processes and systems, creating communication strategies with accountability, updating policies/procedures, refining Internal Program Reviews PI: fine tuning performance outcomes, evaluating current methodologies and solidifying data collection, web-enabling instruments, analyzing data, interpreting and reporting results, collaborative research and data sharing,

29 January 27, Internal Program Reviews Tool Development Scheduling & Team Selection Pre-review Conference Program Review Post Review Conference Written Report Enhancement Plan/Follow-UP Evaluation of Findings Reporting Findings Internal Review Process

30 Next Steps Collaboration/Partnering with other providers and researchersCollaboration/Partnering with other providers and researchers Sharing methodologies and approachesSharing methodologies and approaches Understanding the impact of outcomes and how it influences current issues and trainingUnderstanding the impact of outcomes and how it influences current issues and training Disseminating information about programs effectiveness and innovationsDisseminating information about programs effectiveness and innovations Designing protocols based on best evidenceDesigning protocols based on best evidence Identify model programs and replicating successIdentify model programs and replicating success

31 Q and A Opportunity is missed by most because it is dressed in overalls and looks like work. ~Thomas Edison Maurie Lung, LMHC, LMFT Eckerd Youth Alternatives, Inc./Life Adventures Counseling & Consulting PO Box 10907, St. Petersburg, FL Greg Holeman, Chief Quality Officer Eckerd Youth Alternatives 100 N. Starcrest Dr., Clearwater, FL X437


Download ppt "Balancing Evaluation and Program Needs REAP Symposium 2009."

Similar presentations


Ads by Google