1. Data efficiency in effective communication 6 th Annual Summit Vocational Rehabilitation Program Evaluation and Quality Assurance Providence RI September 16-17, 2013

2. Presented by: Susan Stoddard, Ph.D., President, InfoUse; RTAC Scott Sabella, Ph.D., Director, Region 8 TACE Michael Quinn, Program Evaluation Coordinator, Alabama Department of Rehabilitation Services Elizabeth Moody, Strategic Planning Coordinator, Florida Division of Vocational Rehabilitation

3. The expanding data challenge Expansion of data availability Better internal data systems New data analysis and display tools Advances in data technology Significant investment in data and systems How to use data effectively and efficiently? How to communicate progress and results?

4. Needed: best data to support… Reporting Evaluating Planning Deciding Auditing And of course assuring service quality, outcomes, and continuous improvement In all of this we rely on our data.

5. Danger – Information/need mismatch Lots of information: 911 and agency MIS; financial transactions ; vendor data; census/ACS and other population information. Many internal and external information needs: reporting to RSA and other funding sources; general reporting to the public; internal monitoring and performance assessment, etc. Costly, inefficient How to link these data to agency information needs to support communication and continuous improvement?

6. Major data source options (existing) Agency data (RSA-911, RSA-2, MIS, case management) Public data (census, ACS, disability population, state labor market data, IDEA data) Fiscal reports Case file reviews Consumer satisfaction surveys Monitoring reports State audits

7. Data Source Options (new) Needs assessments (training, client, CRP, stakeholders) Surveys (clients, counselors, PWD) Focus groups Community hearings Key informant interviews Targeted evaluations Targeted case file reviews

8. Examples Examples of linking the data to agency information needs to support communication and continuous improvement – Alabama – Florida – Utah – CSNAs: Michigan, Iowa, Ohio – Working with SRC

9. Alabama VRS Examples of effective data communication using maps

10. Alabama VRS annual report

11. Alabama tailored briefing for legislative district

12. Examples of effective data communication Florida VR

13. External communicati on method Personalized VR performance and state population data Return on Investment (ROI) Legislative Profiles

14. Available via Sharepoint Interactive graphics Magic Kingdom themed training Florida VR Performance Report

15. Survey administered every 2 years Employee input drives decision- making Linkage is evident Employee Climate Survey + Strategic Plan

16. Utah State Office of Rehabilitation (USOR) Economic Impact Study Written Benefits Analysis Study

17. USOR Economic Impact Study Relatively simple formula (increase in tax revenue + decrease in public assistance + decrease in Medicaid payments)/state dollars invested Multiple uses – Information to agency leadership and staff on impact – State legislature advocacy – State report cards to U.S. Senators and Congressman – National advocacy

18. USOR Written Benefits Analysis Study Are Benefits Planning Assistance and Outreach services associated with improved earnings and employment outcomes Yes, those who received benefits planning services had higher wages than the control group Yes, those who received benefits planning services was more likely to have a successful closure and more likely to be employed in the quarter after closure Utilization Advocacy for the program Decision making Used to support funding benefits planner positions

19. CSNAexamples CSNA – Look beyond the “every three year” study… – To what extent is the CSNA integrated into the continuous feedback and MIS systems? – Ways to make the CSNA investment serve agency planning and communication needs – What new opportunities need to be created? – CSNA information and formatting… some possibilities

20. Michigan CSNA: Outcomes and Attrition

21. Michigan: Employment rates

22. Iowa: Outcome rate by impairment, 2007-2010

23. Iowa: Unique population story

24. Iowa: Lowest growth rate in US, 1900-2009

25. 5 th in % population 65 and over, 2008

26. Iowa Caseload comparisons

27. Ohio decreasing referrals, applicants eligibility

28. Ohio funding context for declines

29. Ohio: County needs/service gaps Transition

30. Working with your SRC Providing input on outreach to PWD Providing input on needs for PWD Indentifying key informants Developing surveys and communication strategies Reviewing and analyzing information Prioritizing needs Making recommendations for strategic plan Making recommendations for the state plan

31. Parkinson’s Law of Triviality (Bike Shedding) “The time spent on any item on an agenda will be in inverse proportion to the sum involved” Committee deliberations regarding building an atomic reactor Committee deliberations regarding building a bike shed.

32. Bike Shedding and SRC Example Promise grant initiatives ($1,000,000) Attrition rate ($100,000) Transportation project ($100,000) What to serve for lunch at a working lunch meeting ($600) What is the point?

33. Example: Framing Information for Your SRC? Intent: Are you trying to get their input or get items through? How much invested in helping them understand complicated issue Preservation of language for certain information Examples related to daily life

34. The Important VR Cycles State fiscal year Federal fiscal year State legislative cycle CSNA 3 year cycle State plan submission Federal reporting (RSA-911, RSA-2) Federal monitoring

35. Windows of Opportunity Discussion What are the windows of opportunity for these various cycles? – Preparing for legislative sessions – Federal reporting (program, fiscal) – CSNA – State plan – Federal monitoring

36. Conclusion: 8 principles to remember Know your available data source options and how to use them Make use of existing data Use information for multiple purposes Data quality: avoid GIGO Make sure there is a link in process from data to decision making Make sure the data gets to the people who can use it, communicated in a form they understand Share, borrow and adapt good ideas Support a continuous improvement system

37. Contact sustoddard@gmail.com Scott.Sabella@unco.ecu