1. The CRP Cost Study Evaluating the true cost of CRP services
Andrew K. Clemons
State of Washington Division of Vocational Rehabilitation
Laurie Ford
University of Washington Center for Continuing Education in Rehabilitation
Dr. David Vandergoot
WorkLife Resources

2. The CRP Cost Study: Evaluating the true cost of CRP services
Welcome
Overview
Cost Study partnerships, objectives, and background
DVR’s CRP contracts and fee structures
Launching the CRP Cost Study
Project management and data collection
Research design and methodology
Analysis and findings
Impacts, next steps, and Q & A
Visit CCER online to download the full report.
9th Annual Summit on Performance Management in Vocational Rehabilitation, 2016

3. The CRP Cost Study Research partnership Objective
State of Washington Division of Vocational Rehabilitation (DVR)
University of Washington Center for Continuing Education in Rehabilitation (CCER)
Dr. David Vandergoot
Washington’s Community Rehabilitation Programs (CRPs)
Objective
To determine if DVR’s contract fees adequately and responsibly compensate CRPs for actual direct and indirect service costs

4. Washington DVR & CRP Partnerships
DVR & its CRP partners have an established history of close collaboration at local and state levels
DVR invests significantly in CRP services
Community Based Assessments, Job Placement & Job Retention, Intensive Training (i.e. Supported Employment), Off-site Psychosocial Job Support
Services are offered at three intensity levels based on individualized determinations of consumer needs
DVR’s CRP Contract utilizes an outcomes-based milestone fee structure

5. Washington DVR & CRP Partnerships
DVR’s CRP partners represent a wide range of organizations and service delivery models
Community-based programs
Industry-based programs
Many of DVR’s CRP partners also have contracts and close relationships with mental health and developmental disabilities programs
Significant percentages of DVR consumers are jointly served by Washington’s One-stop Centers (WIOA 1b & III, TAA, UI) & TANF and other assistance services

6. CRP Contracts & Fees
2006
Outcome-based model
2008
Milestones
Intake
Outcome
2010
Activity
2006 – 2008
An outcomes-based model is implemented, replacing a fee-for-service model
2008 – 2010
A milestone fee structure is implemented, including two milestone payments: an intake fee and an outcome fee
An activity fee is added to the milestone structure to help CRPs manage costs incurred during service delivery

7. CRP Contracts & Fees CRP Cost Study
2012
Milestones
Intake
Activity
Outcome
2014
2015
CRP Cost Study
2012 – 2014
Small percentage fee increase is implemented based on reasonable estimates of cost increases
2014 – present
No changes; CRPs continue to express concern that DVR fees do not adequately cover costs
2015 – 2016
CRP Cost Study is launched to evaluate CRPs’ costs of serving DVR consumers

8. Impetus: CRP Perspectives
Between 2010 & 2015, CRPs expressed concern that DVR fees did not adequately cover actual costs.
CRPs reported
2010 addition of activity fee was insufficient to cover mid- service costs
2012 rate increases were insufficient to cover actual costs and increasing costs of living in Western Washington
Recruitment, training, and retention of qualified CRP staff was adversely impacted by insufficient fees

9. Impetus: DVR Perspectives
Between 2010 & 2015, DVR shared concerns regarding the adequacy of DVR fees and the potential impacts on DVR consumers and CRP partnerships.
DVR observed
Increasing frequencies of Level 3 services, while consumer demographics remained consistent
Increasing frequencies of renegotiated levels, as CRPs sought higher-level service payments, occurring while consumers participated in contracted CRP services
Increasing dialogue, at state and local levels, regarding DVR’s fee structure and CRPs’ costs

10. Launching the CRP Cost Study
In response, DVR sought objective third-party analyses of CRPs’ costs, using CRPs’ actual cost records, and the extent to which DVR’s fees adequately and responsibly covered these costs.
DVR conducted policy and literature reviews and drafted a statement of work which described CRP service types, direct and indirect cost definitions, and areas of inquiry
DVR engaged its partners to launch the study
The Center for Continuing Education in Rehabilitation (CCER)
Dr. David Vandergoot
The Community Employment Alliance (CEA)
Washington’s Community Rehabilitation Programs

11. Steering Committee
CCER coordinated a tripartite steering committee of CRPs, DVR representatives, and the research team.
CRP Cost Study Steering Committee
CRP Directors & Leadership
DVR Leadership & Management
Cost Study Research Team

12. Project Management & Data Collection
The CRP Cost Study: Evaluating the true cost of CRP services
Project Management & Data Collection
CRP Recruitment
Announcement letter from DVR Director to CRP Leadership
Follow-up s from CCER
Invitation through the CRP organization (Community Employment Alliance)
Personal contacts by Steering Committee members
Our role was to keep the project moving forward and keep track of data submission etc.
CCER well known to Washington CRPs and also a neutral party – no investment in how the data turned out
9th Annual Summit on Performance Management in Vocational Rehabilitation, 2016

13. Project Management & Data Collection
The CRP Cost Study: Evaluating the true cost of CRP services
Project Management & Data Collection
CRP Participation
63 CRPs agreed to participate
Wide range in organizations, size, and regions
Included 36 of 39 Washington counties
Size ranged from single person to multi-service, multi-location agencies with hundreds of staff
9th Annual Summit on Performance Management in Vocational Rehabilitation, 2016

14. Project Management & Data Collection
The CRP Cost Study: Evaluating the true cost of CRP services
Project Management & Data Collection
As data collection began, 17 of the 63 CRPs declined to participate. While some CRPs could not provide needed data, others had concerns regarding the study itself.
Workload or organizational changes
Skepticism about impact of study results
Concern that retrospective study will be inaccurate
Very few DVR plans/staff
Director doesn’t want to share requested information
Too much work to collect data
Don’t have accurate records
1. Most CRPs have high turnover and lots of staff changes
9th Annual Summit on Performance Management in Vocational Rehabilitation, 2016

15. Project Management & Data Collection
Participation Incentives
DVR offered a $ 1,500 stipend for each data collection tool completed ($ 3,000 total)
CCER facilitated payment upon receipt of data collection tools
However, these incentives did not significantly affect CRP participation

16. Project Management & Data Collection
The CRP Cost Study: Evaluating the true cost of CRP services
Project Management & Data Collection
CRP Cost Study Steering Committee
5 CRP representatives and project staff
Roles:
Review and contribute to design and methodology
Assist in CRP recruitment
Test data collection tools
Four in-person meetings as well as conference calls and communications
One CRP representative was from a MH center
DD was invited to participate but chose not to.
9th Annual Summit on Performance Management in Vocational Rehabilitation, 2016

17. Summary of Data Elements & Analysis
The CRP Cost Study: Evaluating the true cost of CRP services
Summary of Data Elements & Analysis
Table listing eighteen data elements, the source, the purpose, and how the data will be analyzed.
For example:
A: # of consumers served by each CRP during the study year, from DVR administrative data
J: CRP wages/hour, benefits/hour, taxes/hour for staff providing services to DVR consumers, provided by CRP
M: Average and median total costs for each service and service level when an outcome is achieved, when no outcome is achieved, and total, calculated by contractor from CRP data (hours of service, hourly personnel cost, travel costs, indirect costs)
Helpful to make sure we were collecting all of the data needed to answer the questions specified in the scope of work
9th Annual Summit on Performance Management in Vocational Rehabilitation, 2016

18. Project Management & Data Collection
The CRP Cost Study: Evaluating the true cost of CRP services
Project Management & Data Collection
Project Management
Tracking completion of data collection tools, vendor registrations, stipend payments
Review of data collection tools, redaction of confidential data, coordination with researcher
Ongoing timeline and deliverables management
Technical assistance for participating CRPs
What agency activities to include in calculation of indirect rate (e.g. residential, commercial)
Registering as a vendor with UW
Use study year or accounting year for indirect
Spreadsheet irregularities (missing formulas, duplicated data)
Which consumers should be included in total (e.g. those that receive only I & R)
Multiple geographic location – should all costs be included?
Can negotiated indirect rate be used
Don’t have good data for some services/consumers
Include all hours of service or only those provided during study year
How to calculate % of FTE serving VR customers
Fringe benefit rate in an owner-operated for-profit business
Whole agency vs just employment team
How to include costs paid by donated funds
9th Annual Summit on Performance Management in Vocational Rehabilitation, 2016

19. Unforeseen Challenges
The CRP Cost Study: Evaluating the true cost of CRP services
Unforeseen Challenges
DVR tracks records by case numbers; CRPs keep records by name.
DVR prepared prepopulated collection tools with case numbers, so CRPs could input data
Steering committee members caught this in their review/testing of the tools
Consumer names had to be added to already completed collection tools for each CRP, delaying distribution
CRPs were asked to complete the Direct Cost Calculator, then redact the consumer names before submitting
9th Annual Summit on Performance Management in Vocational Rehabilitation, 2016

20. Statistical Approach Sampling Strategy
Unit of Analysis – service provided to an individual consumer
Sampling Strategy
6,063 consumers of DVR were served during Study Year
A 20% sample was determined to provide a reliable estimate of population values
A 20% sample was randomly drawn from the eligible consumer population of each participating CRP
If the resulting sample from a CRP was less than ten, additional consumers were randomly chosen to result in a sample of ten

21. Statistical Approach Study Sample
Unit of Analysis – service provided to an individual consumer
Study Sample
For CRPs serving a total of less than 10, all consumers were selected
A total of 1,132 consumers were eventually selected (≈ 19% of population)
Seventeen CRPs dropped out after the fact, reducing sample to (≈15.5%)
Data were actually returned on 875 consumers (≈14.4% and response rate of 90%)

22. Analyses Comparison of Cost of Providing Service to Actual Fee Paid
Percentage Difference between Costs Incurred and Fees Paid (Recovered Costs)
Calculation of Hourly Cost for Each Service (Mean and Median)
Estimation of Relationship between CRP Number of Staff and Costs
Estimation of Relationship between CRP Number of Locations and Costs
Estimation of Relationship between CRP Number of Counties and Outcomes
Estimation of Percent of Total Revenue of CRPs from DVR during Study Year
Preliminary Analysis of Closure Outcomes and Services by cost of Service Provided, days from eligibility to plan development, and total days served

23. Findings: Comparing Costs & Fees
Table 1: Provider and Fee-Based Statistics
Provider-Based Statistics
Fee-Based Statistics
Total Service Costs N
Provider Based Mean Direct (Costs)
Total Cost with Indirect, Fringe and Other Direct (rate of 64.26%)
Total Served Mean
(Fee Paid)
Total Served Median
Vocational Evaluation 6
$317.73
$521.91
$1,154.17
$1,150.00
Job Placement 1 24
$681.22
$1,118.97
$541.25
$300.00
Job Placement 2
198
$1,067.68
$1,753.77
$706.29
$600.00
Job Placement 3
134
$885.55
$1,454.61
$1,039.01
$900.00

24. Findings: Comparing Costs & Fees
Table 1: Provider and Fee-Based Statistics
Provider-Based Statistics
Fee-Based Statistics
Total Service Costs N
Provider Based Mean Direct (Costs)
Total Cost with Indirect, Fringe and Other Direct
(rate of 64.26%)
Total Served Mean
(Fee Paid)
Total Served Median
Intensive Training 1 11
$704.39
$1,157.03
$818.18
$1,125.00
Intensive Training 2 12
$683.46
$1,122.66
$1,812.50
$2,250.00
Intensive Training 3 27
$1,371.79
$2,253.30
$2,472.22
$3,375.00
Community Based Assessment 1 32
$726.29
$1,193.01
$538.59
$732.50
Community Based Assessment 2
164
$628.62
$1,032.58
$982.32
$300.00
Community Based Assessment 3
139
$739.52
$1,214.73
$1,336.40
$450.00

25. Findings: Comparing Costs & Fees
Table 1: Provider and Fee-Based Statistics
Provider-Based Statistics
Fee-Based Statistics
Service N
Provider Based Mean Direct
(Costs)
Total Cost with Indirect, Fringe and Other Direct
(rate of 64.26%)
Total Served Mean
(Fee Paid)
Total Served Median
Trial Work 2 3
$583.25
$958.05
$1,333.33
$1,700.00
Trial Work 3 7
$975.34
$1,602.09
$1,422.86
$450.00
Offsite Psychosocial SE 2 1
$189.17
$310.73
$2,250.00
Offsite Psychosocial 2
$220.75
$362.60
Job Retention 1 34
$578.41
$950.09
$708.09
$225.00
Job Retention 2 51
$576.57
$947.08
$1,488.60
$765.00
Job Retention 3 28
$969.21
$1,592.03
$2,515.18
$3,037.50

26. Highlights of Cost-Fee Analysis
Indirect, fringe benefits and other direct costs not reimbursable by DVR averaged 64% of direct costs
Overall, across all services, fees and total costs compared favorably
There was a wide variation of difference between costs and fees when looking at each individual service

27. Extent of Recovered Costs
Table 2: Differences between Provider Costs and Fees
Total Service Costs
Mean Cost Difference
Percent of
Costs Recovered from Fees
Vocational Evaluation
$632.26
221%
Job Placement 1
-$577.72
48%
Job Placement 2
-$1,047.48
40%
Job Placement 3
-$415.60
71%
Intensive Training 1
-$338.85
Intensive Training 2
$689.84
161%
Intensive Training 3
$218.92
110%
Community Based Assessment 1
-$654.41
45%
Community Based Assessment 2
-$50.26
95%
Community Based Assessment 3
$121.68

28. Extent of Recovered Costs
Table 2: Differences between Provider Costs and Fees
Total Service Costs
Mean Cost Difference
Percent of
Costs Recovered from Fees
Trial Work 2
$375.28
139%
Trial Work 3
-$179.23
89%
Offsite Psychosocial SE 2
$1,939.27
724%
Offsite Psychosocial Non SE 2
$87.40
124%
Job Retention 1
-$242.00
75%
Job Retention 2
$541.52
157%
Job Retention 3
$923.15
158%

29. Interpreting Results Nine of 17 services did demonstrate cost recovery
Exactly 100% recovered costs - All CRP costs were reimbursed
Above 100% recovered costs - DVR fees exceeded CRP costs
Below 100% recovered costs - DVR fees did not cover CRP costs
Nine of 17 services did demonstrate cost recovery
(110% to 724%)
Eight of 17 did not demonstrate cost recovery
(40% to 95%)

30. Interpreting Results
Job Placement services tended to have the lowest cost recovery
Job Retention services had the highest cost recovery
Samples for some services too small to achieve reliable estimates

31. Table 3: Hourly Average Costs
Hourly Service Costs
Table 3: Hourly Average Costs
Service
Hourly Average Cost
Median Average Hourly Cost
Average Hours of Services
Median Hours of Services
Vocational Evaluation
$48.18
$48.72
10.5 8
Job Placement 1
$27.68
$28.91
67.6
49.2
Job Placement 2
$32.00
$31.82
105.5
78.2
Job Placement 3
$30.43
$31.73
75.1
60.7
Intensive Training 1
$28.10
$32.33
83.5
59.4
Intensive Training 2
$28.20
$25.55
71.6
87.8
Intensive Training 3
$32.14
$34.38
215.8
231.7
Community Based Assessment 1
$21.92
$30.74
109.7 94
Community Based Assessment 2
$33.04
$31.74
60.4
55.5
Community Based Assessment 3
$30.04
$29.50
66.2
53.6

32. Table 3: Hourly Average Costs
Hourly Service Costs
Table 3: Hourly Average Costs
Service
Hourly Average Cost
Median Average Hourly Cost
Average Hours of Services
Median Hours of Services
Trial Work 2
$32.55
$41.56
29.4 25
Trial Work 3
$27.44
$22.87
58.4
66.4
Offsite Psychosocial SE 2
$33.59
Offsite Psychosocial Non SE 2
$32.96
Job Retention 1
$37.52
$36.62
47.3 41
Job Retention 2
$34.80
$31.31
70.5
49.5
Job Retention 3
$40.87
$45.31
101.3
92.4

33. Highlights of Analysis
Vocational Evaluation & Job Retention services were the most expensive per hour
Intensive Training (i.e. Supported Employment) services tended to be the least expensive per hour
However, most services were approximately $30-$34 per hour
Intensive Training (i.e. Supported Employment) tended to exhibit the greatest number of service hours provided

34. CRP Staff Size & Costs Analysis of Variance at four levels* 1 employee
2-5 employees
6-10 employees
>10 employees
*Only significant relationships reported (ɑ≤ .10)

35. CRP Staff Size & Costs
Job Placement services were most expensive for CRPs with >10 employees
Community Based Assessment services were most expensive for CRPs with only one employee and >10 employees
Job Retention services were most expensive for CRPs with only one employee
Overall, CRP services were most expensive for CRPs with only one employee or >10

36. CRP Locations & Costs Analysis of Variance at four levels* 1 location
*Only significant relationships reported (ɑ≤ .10)

37. CRP Locations & Costs
Job Placement services costs were lowest for CRPs with 2-5 locations
Community Based Assessment services were lowest for CRPs with >5 locations
Job Retention services were lowest for CRPs with >5 locations
Overall, the highest costs were CRPs with one location and the lowest costs were those with >5 locations

38. DVR fees & CRPs’ Revenues
The average number of authorizations paid to CRPs during the study year was 193
The average revenue from DVR was $210,700 while the average overall revenue was $1,996,833
The average percent of revenue CRPs derived from DVR was 27%
The median overall revenue is $1,009,990, indicating that most CRP’s revenues are well below the average

39. Closure Outcomes & Costs
The highest cost level is associated with the Open Cases group. The differences are significant.
Closure Outcome and Total CRP Service Costs N
Mean
Median
Std. Deviation F
Sig.
Other
179
$656
$553
$570
4.803
.008
Rehab
101
$788
$536
$864
Open Cases
591
$876
$624
$899
Total
871
$820
$593
$842

40. Closure Outcomes & Costs
The same pattern occurs with the Open Cases group having the highest fees paid and the Other group the lowest by far. The difference is highly significant.
Closure Outcome and Total DVR Fees Paid N
Mean
Median SD F
Sig.
Other
179
$2054
$2000
$1449
72.24
.000
Rehab
101
$4556
$3930
$2820
Open Cases
589
$5372
$4320
$3663
Total
869
$4594
$3500
$3490

41. Highlights of Analysis
DVR Case Service Timelines
Days from application to determination of eligibility was not significant
Days from plan development until closure was not significant
Days from eligibility to plan development was significant

42. Impact & Next Steps CRP Cost Study Phase Two activities
Third-party recommended percentage adjustments to DVR fee structure
DVR cost projections by percentage ranges
Implement new fee schedule
Cost analyses by geographic regions
Eastern & Western Washington
Metropolitan and rural areas
Cost analyses by CRP service outcome
DVR administrative data only, proxy CRP service completion using payment amounts benchmarked to milestones
Using CRP Cost Study data, determine impact of service completions on CRPs’ encumbered costs and outcomes

43. DVR-CRP Workgroup
The CRP Cost Study strengthened DVR-CRP partnerships and laid the foundation for collaborative activities outlined in The DVR State Plan.
Review the current contract model to determine needed improvements or modifications.
Develop an evaluation tool to assess CRP performance and service quality.
Complete, publish, and disseminate the CRP Cost Study… and use results to inform future contracts and fee structures.
Develop and provide training for DSHS/DVR and CRP staff to promote effective collaboration, CRP staff retention, and shared best-practice service methodologies.
Disseminate best practice guidance to DSHS/DVR staff regarding communications and collaboration with local CRPs.

44. Q & A

45. Visit CCER online to download The CRP Cost Study full report.
Thank you
Andrew K. Clemons
State of Washington Division of Vocational Rehabilitation
Laurie Ford
University of Washington Center for Continuing Education in Rehabilitation
Dr. David Vandergoot
WorkLife Resources
Visit CCER online to download The CRP Cost Study full report.