1. Selection Practices & Communication
Caryn Ward, PhD
OSEP SPDG Webinar
January 5th, 2017

2. Deepen understanding of best practices in Selection
Share examples and lessons learned from an SEA in putting the practices to use
Share examples and lessons learned for communication between linked teams and stakeholders

3. Implementation Drivers

4. Selection Best Practices Select for “tough to teach traits”
Set expectations for staff
Improve retention
Job descriptions
Interviewers understand the skills and abilities needed for position
Interview protocols are in place
Interview processes regularly reviewed
There is someone accountable for the recruitment and selection of staff who will carry out the program or practice
A specific person is responsible for coordinating the quality and timeliness of recruitment and selection processes for staff supporting the program or practice
-AND-
This person is able to execute the responsibilities related to his/her role in the selection process
Job descriptions are in place for staff positions that will carry out the program or practice
Job descriptions are clear about expectations for each position
Job descriptions explicitly align with the practices and competencies required for the program to be used competently
Interviewers understand the skills and abilities needed for the staff position
Interviewers have knowledge, skills, and abilities related to the staff position
Interviewers accurately assess applicant knowledge, skills, and abilities
Interview protocols are in place to assess candidates’ competencies for the staff positions that will carry out the program or practice
Job interview protocol includes all of the following: an assessment of core skills needed for position; specific procedures (e.g., vignette, role play) for assessing candidate capacity to perform key skills; specific procedures for assessing capacity to use feedback provided during the interview to improve; specific procedures to assess capacity to receive feedback professionally; Review of adherence to the interview protocol is documented
Ratings of applicant responses are recorded
 Interview processes are regularly reviewed
Interview processes are annually reviewed and revised as needed to improve the selection process
The annual review examines at least three of the following:
Interview results (e.g. protocol adherence; applicant responses); Pre-post training data for successful applicants; Turnover data; Fidelity data; Exit interview results

5. Example from an SEA: State Transformation Specialists
Context:
New Partnership formed with SEA to be an Active Intensive Scaling State
Requirement to select 2 State Transformation Specialists (STSs)
SISEP history with guiding states on STS selection process needed improvement as indicated by data (low longevity rates in STS held positions)
Overarching Critical Functions of STS Role:
Purposeful, multi-level communication to promote problem- solving and alignment
Leadership in developing the capacity to create, regenerate, and sustain State & Regional Implementation Teams.

6. Discussion poll:
What is your staff longevity rate for your respective SPDG projects at the SEA level?
Low
Middle
High
Hmm.. Not Sure

7. Opportunity for Implementation Informed Selection Process
Leadership Commitment
Position Description
Selection Criteria
Selection Process Developed
Interview Protocol
Debrief with Leadership for Improvement
State Leadership Team:
Chief Deputy Superintendent
Chief of Staff
Deputy Superintendents:
Educator, Student, & School Supports
P-20 & Student Transitions
Finance & Operations

8. SISEP Support & Example Position Description
Support Provided by SISEP:
Provided overview of Selection Driver with rationales
Provided Example Job Description for STS
Provided Handout describing STS roles and functions
Provided feedback on position description drafting
Provided guidance on development of selection process and interview protocol
Participated in performance interviews as coaching role for interviewers
Job Description Components:
Knowledge, Skills, & Abilities
Reporting Structure
Overall Purpose and Function
Detailed Function Descriptions with Tasks & % Time
See Example Position Description at SIGnetwork.org, Events Calendar, 1/5/2017.

9. Which of these would you consider as tough to teach traits?
Example Selection Criteria Generated by Leadership
Positive & Passionate
Exhibits resilience/perseverance
Exhibits adaptive skills – can determine when flexibility is needed
Creditability with a team
Effective consensus building skills
Effective facilitation skills
Values diverse perspectives
Familiarity with state government processes and functions
Instructional skills for training and coaching teams
Ability to hold teams accountability
Strong commitment to work
Effective diagnostic skills
Strong data analysis skills
Which of these would you consider as tough to teach traits?
See Selection Protocol at SIGnetwork.org, Events Calendar, 1/5/2017.

10. See Selection Protocol at SIGnetwork.org, Events Calendar, 1/5/2017.
Selection Process: Three Phase Interview Protocol
HR used screening criteria to identify applicants for interview protocol Results: 55 applicants
Leadership team reviewed all applications to determine 11 candidates to interview
15 Min Phone Interview
In Person Interview (Top 4-5)
Performance Tasks (Top 4-5)
See Selection Protocol at SIGnetwork.org, Events Calendar, 1/5/2017.

11. See Selection Protocol at SIGnetwork.org, Events Calendar, 1/5/2017.
Example Phone Interview Questions
See Selection Protocol at SIGnetwork.org, Events Calendar, 1/5/2017.

12. See Selection Protocol at SIGnetwork.org, Events Calendar, 1/5/2017.
Example In Person Interviews
See Selection Protocol at SIGnetwork.org, Events Calendar, 1/5/2017.

13. See Performance Tasks at SIGnetwork.org, Events Calendar, 1/5/2017.
Example Performance Tasks
See Performance Tasks at SIGnetwork.org, Events Calendar, 1/5/2017.

14. Discussion polls (Open Ended):
Part 1: How have you used performance tasks as part of the selection process for staff in your SPDG project?
Part 2: What have been facilitators and/or barriers to being able to use them?

15. See Memo Summary at SIGnetwork.org, Events Calendar, 1/5/2017.
Outcomes of Selection Process
Engaged state leadership in learning and application of an implementation practice from the start of capacity development
Recommended and hired 2 full time STSs
State leadership engaged in problem solving at the conclusion of the process to identify funding to support second position
Attention and support by Human Resources Department for System Change effort within State
Created plan with HR Director to look at how best practices of selection could be generalized to other practices and supports needed for staff to do so
See Memo Summary at SIGnetwork.org, Events Calendar, 1/5/2017.

16. Lessons Learned…. Funding for only one was secure
Get Started – We will figure it out for the second position
Obstacles to Use of Behavior Rehearsals:
“Civic Government doesn't allow us to do so”
“All interviews must follow the exact same protocol - thus providing feedback is not possible”
Found examples within SEA from one department to highlight use and capability to do so
Confirmed with HR Director both are allowed but do need to follow same protocol

17. Lessons Learned…. Rehearse the Behavior Rehearsals with interviewers
Feedback after 1st candidate performance task was needed on how to provide feedback to candidate and ask candidate to incorporate feedback and redo part of the task
Incorporate this process more explicitly into future interview protocols
Reporting Structure
Civic government system “level” of position dictates reporting structure
SISEP recommends STSs report to Chief Deputy and not be located within a specific department.

18. Communication & Learning: Enabling Context
Communication builds enabling context for our Active Implementation Formula.

19. Communication: Practice Policy Communication Cycle
Leadership Team
Sustainability
Policy Enables Practice
Practice Informs Policy
Implementation
Team
We want to have communication between direct service staff and the policy makers and administrators who support them.
Communication can be designed to provide organizational leaders and policy makers with information about implementation barriers and successes so that a more aligned system can be developed. Feedback from the practice level (Practice Informed Policy) engages and informs organizational leaders so that they can ensure that policy, procedures, resources, etc. enable innovative practices to occur in classrooms, centers and regions (Policy Enabled Practice) as intended.
Direct Service Staff
(c) Dean Fixsen and Karen Blase, 2004
(c) Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008 19 19

20. Communication: Linked Teams
Communication must be bi-directional between all linked teams in the system
Communication must be bi-directional between all linked teams in the system, as you see here.

21. Reflection
Do you currently have a process in place to communicate successes and problems among teams?
Poll (open)
If so, what are some of its strengths and weaknesses?

22. Communication: Who should be communicating?
Your team
Vertical team connections
Horizontal team connections
When you think about your work, there are vertical and horizontal teams that you can communicate with.
Vertical teams would be those above and below you in the cascade of support – for example, if you are a regional T/TA Provider, your vertical teams would be the Head Start programs you work with, and the regional office staff members.
Horizontal teams are those who are at the same level of the system as you – for example, other regional T/TA Staff.

23. Communication: Where are you?
Team
Team
Team
Team
Team
Teams that you are you supporting; teams that are supporting you. What are the vertical connections within our outside of the regional work you carry out in working with programs? Or, in terms of the programs you work with, who do they need to communicate with in a leadership or policy making role to advocate for policy changes when necessary? Also consider horizontally – would frequent communication be helpful with other programs or organizations they partner with to serve children and families in your program?
Team

24. Communication: How Often Should We Communicate?
Regularly scheduled
Formal process
Dedicated appointment
Opportunities to make changes
Quarterly
Monthly
Weekly
When you think about frequency of communication with the programs you support, are the opportunities to communicate regularly? Are these processes formalized. For example, who is responsible for gathering information that will be shared and collective feedback? Who does this feedback go to and when do Implementation team members at the programs you work with need to respond to the feedback? Having formalized communication opportunities allows for the chance to monitor improvement strategies and proactively provide the support needed to teachers and home visitors.

25. Communication: What should we talk about?
What’s working?
What’s not working?
What’s getting in our way?
How do we know?
DATA
What do implementation teams communicate about? Using data in a big and small way. Think about the data the programs you work with are collecting – what do the data tell them? How are things going? We also think about data as a big and little D. Data can include qualitative information such as feedback from teachers. Are multiple teachers struggling to engage children in a particular aspect of practice? As a TA provider, you can work with programs to identify strategies to routinely collect and use data to inform decisions and celebrate success.

26. Communication: Using Data and Information
Look
What data are we looking at today?
What are the data telling us?
How might we need to adjust or pivot our strategy based on this data?
How will we know these adjustments had the intended effect or benefit?
Who will be responsible for making this adjustment and in what timeframe?
Think
The look, think, act strategy is one way to review data and target improvement strategies. This is one way to structure agendas to review data.
Act

27. Monthly Regional Team Meeting
A System Example
Monthly Communication Process
Following
District Meetings
Debrief
Generate meeting documentation
Monthly Regional Team Meeting
Share progress, challenges and successes
Problem solve and plan for next District meetings
State Team
Meeting
Help problem solve challenges
Operationalize success

28. Communication in Action

29. Reflection
Do you currently have a process in place to communicate successes and problems among teams?
Poll (open)
How might a written communication protocol support the process or help to develop a process?

30. Communication Resources on AI Hub
Lesson 9: Linking Communication Protocols
Communication Protocol Worksheet

31. “I wanted a perfect ending
“I wanted a perfect ending. Now I've learned, the hard way, that some poems don't rhyme, and some stories don't have a clear beginning, middle, and end. Life is about not knowing, having to change, taking the moment and making the best of it, without knowing what's going to happen next.
Delicious Ambiguity.”
― Gilda Radner

32. For More Information Caryn Ward Caryn.ward@unc.edu
Frank Porter Graham Child Development Institute
University of North Carolina
Chapel Hill, NC
(c) Allison Metz, 2014

34. Citation and Copyright
This document is based on the work of the National Implementation Research Network (NIRN).
© Allison Metz, Leah Bartley, Jonathan Green, Laura Louison, Sandy Naoom, Barbara Sims, and Caryn Ward
This content is licensed under Creative Commons license CC BY-NC-ND, Attribution-NonCommercial-NoDerivs . You are free to share, copy, distribute and transmit the work under the following conditions: Attribution — You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work); Noncommercial — You may not use this work for commercial purposes; No Derivative Works — You may not alter, transform, or build upon this work. Any of the above conditions can be waived if you get permission from the copyright holder.
web:
The mission of the National Implementation Research Network (NIRN) is to contribute to the best practices and science of implementation, organization change, and system reinvention to improve outcomes across the spectrum of human services.