1. Dana Keener, Ph.D. ICF Macro 2009 AEA Annual Meeting November 12, 2009
Evaluating the Added Value of Offering Technical Assistance Following a Training Program
Dana Keener, Ph.D.
ICF Macro
2009 AEA Annual Meeting
November 12, 2009

2. Overview Background/context of the study What is technical assistance?
Intervention description
Methods
Results
Conclusions

3. Study Context
CDC’s Rape Prevention and Education (RPE) Grant Program in the Division of Violence Prevention
Four regional trainings offered in summer to sexual violence prevention professionals
RPE wanted to explore the added value of offering TA after a traditional training session

4. What is Technical Assistance?
What  Intentional, individualized, tangible help, aid, or assistance
For whom  provided to an individual, organization, or community
Why  for the purpose of increasing knowledge, skills, and abilities towards the successful achievement of a particular end- goal or product

5. Four Dimensions of Technical Assistance in the Literature
Relationship quality
Individualization
Proactive design
Sufficient dosage

6. The Intervention
Purpose: to promote the application of effective training practices among sexual violence prevention professionals
Two components:
A full-day training component
A subsequent telephone-administered technical assistance component

7. Desired Outcomes of Intervention
Satisfaction with the training and TA
Motivation to use effective training practices
Knowledge and self-perceived ability to use effective training practices
Application of effective training practices

8. Training Component 6 hour in-person training Repeated in 4 locations
Included PowerPoint, stories, worksheets, exercises, and group discussion
Small group sizes
Atlanta = 4 participants
Chicago = 3 participants
San Diego = 9 participants
Hartford = 11 participants

9. Technical Assistance Component
Conducted via scheduled telephone conferences within 6 months of training
Two different intensity levels
Designed to extend and reinforce the training component
No new material presented
Focused on application of training content

10. Technical Assistance Component
1-60 min group call
Scheduled 90 days after training
n=14
3-60 min group calls
1-60 min individual call
Scheduled every days after training
n=13
Low-Intensity TA
(Chicago and Hartford)
High-Intensity TA
(Atlanta and San Diego)

11. Research Design and Methods
Longitudinal, quasi-experimental
Data collected via surveys and observations

12. Participants 27 participants in total
Self-selected into the training session
Professional, well-educated
Involved in training others as part of their jobs

13. Group Assignment Based on training city
Chicago and Hartford  low-intensity TA
Atlanta and San Diego  high-intensity TA
No systematic differences observed between groups at pre-test

14. Timing of Data Collection
Points of Measurement
Timing of Data Collection
Measure
Method
Before training session
Pre-Training Survey
Self-report paper/pencil
During training session
Training Observation Tool
Evaluator completed paper/pencil
Immediately after the training session
Post-Training Survey
Following each TA call
Post-TA Survey
Web survey
Six months after the training session
Six Month Follow-Up Survey

15. Response Rates Pre-Training  100% Post-Training  100%
6 Month Follow-up  96%

16. Measures Demographics/Participant Characteristics
Organizational Characteristics
Satisfaction with Training and TA
TA Engagement
Effective Training Practices
Attitudes/Motivation
Self-Perceived Ability (general and specific)
Knowledge (knowledge score)
Improvement
Application

17. RESULTS
Results

18. Training Outcomes Participants were highly satisfied w/the training
Participants reported significant increases in:
Perceived ability to plan, implement and evaluate training programs
Knowledge scores
Outcomes did not vary based on size of training group
Outcomes did not diminish 6 months after the training regardless of participation in TA

19. TA Engagement
Defined as participating in at least 50% of calls offered to them
15 engaged participants
9 from high-intensity group
6 from low-intensity group
Participants from the smallest two trainings were more likely to be engaged in the TA component
6 of 7 from smallest trainings were engaged (86%)
9 of 20 from largest trainings were engaged (45%)

20. Technical Assistance Outcomes
No changes in self-perceived ability was observed from post-training to 6 month follow-up
No differences found between high and low intensity groups

21. Technical Assistance Outcomes
Participants in the high-intensity TA group reported more improvement on training tasks than the low-intensity group
Engaged participants reported more improvement in training tasks than unengaged participants at six months
However, the engaged participants already had greater self-perceived abilities before the TA component

22. Technical Assistance Outcomes
Participants who were engaged in the TA fared better than participants who were not engaged at six months follow up.
BUT, most of the differences between engaged and unengaged participants preceded the TA intervention.
The results say more about who was inclined to participate in TA than they do about the impact of the TA component on the outcomes we measured.

23. One size DOES NOT FIT ALL!
Conclusions
Full-day, interactive, skills-based training with small groups (fewer than 12) can be very effective!
Brief TA that is one-size-fits-all is unlikely to contribute very much to outcomes, particularly if it follows training that was already effective.
Don’t throw in the towel on TA though! Need to test tailored TA programs under other conditions.
One size DOES NOT FIT ALL!

24. Recommendations for Future TA
Conduct a brief needs assessment among training participants to identify and select appropriate participants of a follow-up TA program.
Develop individualized TA goals and objectives for each participant.
Develop an individualized TA plan for each participant to meet their goals.
Schedule TA proactively, rather than waiting to be asked for assistance.