1. Measuring Collaboration at state, regional, and local levels
SIG Network Presentation
April 24, 2013
Measuring Collaboration at state, regional, and local levels
Pam William, Missouri Department of Elementary & Secondary Education
Pattie Noonan, University of Kansas
Amy Gaumer Erickson, University of Kansas

2. Collaboration
"In the long history of humankind (and animal kind, too) those who learned to collaborate and improvise most effectively have prevailed." Charles Darwin
“Collaboration is an unnatural act between two non-consenting adults.” Unknown

3. Levels of Collaboration
(Frey, Lohmeier, Lee & Tollefson, 2006)
Collaboration
Independent agencies function as one entity or system. Frequent communication with mutual trust. Consensus reached on all decisions.
Share ideas and resources frequently. Frequent and prioritized communication. All members have vote in decision-making.
Coalition
Sharing information and resources on a regular basis, defined roles, frequent communication and some shared decision-making.
Coordination
Cooperation
Providing information to each other, somewhat defined roles, independent decision-making and formal communication.
Networking
Characterized by awareness of an organization and low levels of communication for referral purposes.

4. Measuring Collaboration
STATE
Missouri Department of Elementary & Secondary Education (DESE) Survey
REGIONAL
Regional Professional Development Centers (RPDCs) Survey
DISTRICT/SCHOOL
School Implementation Scale - District & School Results
SCHOOL TEAMS
Team Functioning Scale – School & Team Results
INTERAGENCY
Missouri Interagency Transition Team (MITT) Social Network Analysis & Focus Group

5. State Department Collaboration
Annual Survey
Involvement in the SPDG
Change in knowledge
Perceived change in collaboration
Current level of collaboration through Social Networking Analysis

6. DESE Results 2011-12 Collaboration Coalition Coordinator Cooperation
Networking
No interaction
At all
Response
Total
Mean
Office of Quality Schools 1 2 3
2.33
Office of Special Education
5.00
Office of Early and Extended Learning 5
1.20
Office of Adult Learning and Rehabilitation Services 4
0.20
Office of Educator Quality
1.60
Office of College and Career Readiness
1.40
Office of Data System Management
2.80
Regional Professional Development Centers
3.80
MIM Schools
1.00
MIM Districts
Parent Centers (e.g., MPACT)
1.50
Representatives of Higher Education (e.g., MU, UMKC, KU evaluators)

7. Regional Professional Development Center Consultants Collaboration

8. RPDC Results 2011-12 Collaboration Coalition Coordinator Cooperation
Networking
No interaction
At all
Response
Total
Mean
Department of Elementary and Secondary Education 5 4 6 24 15 60
2.03
DESE Office of Special Education 3 12 8 27
1.45
DESE Office of Adult Learning and Rehabilitation Services 1 2 7 49 59
0.25
DESE Office of College and Career Readiness 26
1.07
Other Regional Professional Development Centers 13 11 58
2.95
MIM Schools
1.58
MIM Districts 10 57
1.44
Parent Centers (e.g., MPACT) 53
0.12
Representatives of Higher Education (e.g., MU, UMKC, KU evaluators) 14
1.27

9. District/School Collaboration
School Implementation Scale
Evaluates fidelity of implementation and the extent of implementation across a school and/or district.
Appeals to teachers, administrators, other certified staff, and noncertified staff.
All items are evidence-based and result in robust data.
Addresses tiered supports in both academics and behavior.
Is sensitive to change.
Is quick and easy to complete online. Results in a high response rate.
Supports school-level action planning and data-based decision-making.
Correlates closely with gains in academic achievement for students with disabilities.
Has high reliability (overall scale Alpha = .961)
Applicable to elementary, middle, and high schools.

10. One School’s Collaborative Environment Results
Percent of Staff

11. One School’s Family Engagement Indicators of Collaboration
Percent of Staff

12. School Team Collaboration
Team Functioning Scale
17 items
Reports developed for schools and collaborative teacher teams
Full Scale alpha: 0.948
Structure Subscale
alpha: 0.830
Communication Subscale
alpha: 0.939
Focus Subscale
alpha: 0.888

16. Social Network Analysis
Missouri Interagency Transition Team (MITT)

17. Social Networking Analysis
What is SNA?
Social network is to depict interconnections or associations among a set of actors or nodes. A core theoretical problem in network analysis is to detect and represent the relation structure and explain the variation in structural relations and their consequences.
Some Terminologies in SNA
--Actor (Node): can be persons, groups, organizations, etc.
--Relation: defined as a specific kind of contact, connection, or tie
between a pair of actors.
--Density: Average strength of all ties across all relations
--Nodal Indegree: the number of lines received by an actor from
others
--Nodal Outdegree: the number of lines sent by an actor to others

18. General Description of SNA
Social network analysis
Relations Matters!
---Individual characteristics only
explain half the story.
---Hypotheses can be phrased in
terms of correlations between
relations
So:
---The data of SNA is about relations
---It uses different statistical methods
Binary Socialmatrix
Friendship
Tom
Sue
Jim
Bob
--- 1
Valued Socialmatrix
Sender/Initiator
Reciever
Collaboration
Tom
Sue
Jim
Bob
--- 4 5 1 3

19. Social Network Analysis Methods
Ten Organizations comprised the network.
The intensity or the strength of interaction between pairs of actors is illustrated by a 5-point scale questionnaire with 0 represents No Interaction At All, 1 Networking, 2 Cooperation, 3 Coordination, 4 Coalition, and 5 Collaboration.
Social network analysis (SNA) was conducted with the program Ucinet 6.
By convention, for directed relations, the actors in the rows are initiators or senders and the actors in the columns are terminators or receivers of the relations, therefore the descriptive analysis were conducted from both senders or receivers perspectives.
Density was tested both before and after the training

20. Measuring Collaboration
(Frey, Lohmeier, Lee & Tollefson, 2006)
Collaboration
Independent agencies function as one entity or system. Frequent communication with mutual trust. Consensus reached on all decisions.
Share ideas and resources frequently. Frequent and prioritized communication. All members have vote in decision-making.
Coalition
Sharing information and resources on a regular basis, defined roles, frequent communication and some shared decision-making.
Coordination
Cooperation
Providing information to each other, somewhat defined roles, independent decision-making and formal communication.
Networking
Characterized by awareness of an organization and low levels of communication for referral purposes.

21. Results
Example--PTI receives an average tie strength of 1 (networking) before training and 2.05 (cooperation) after training

22. Results
The density after sustained implementation is 2.413, while the density in early implementation is The difference between the two is The p value is less than 0.001, indicating it is significant at level.

23. Tie Strength in Early Implementation

24. Tie Strength after Sustained Implementation

25. Side-by-side

26. Focus Group

27. Focus Group Method Topics:
90 minutes during the group’s regular meeting
10 participants
Recorded and transcribed
Trustworthiness through audits and code-recode cycles
Topics:
the origins, structure, and activities of the state transition team
collaborative processes and experiences

28. Focus Group Results Collaborative Characteristics
Variety of Partnerships
Relationships
Time Together
Shared Vision
Shared Leadership
Collaborative Activities
Site Visits
Joint Planning
Joint Training
Sharing Information

29. Variety of Partnerships
State Department of Education
Vocational Rehabilitation
Department of Corrections
Department of Mental Health
Parent Training & Information Center
Partnerships led to an “increased awareness that there are opportunities to collaborate on projects and initiatives that they might not have thought about before.”

30. Relationships
“Trust…makes someone a lot more likely to invest time and effort in something. We’ve all been on those committees where you know so-and-so’s never going to follow through…Well I don’t think that’s… the case with this group [because] we can count on each other to follow through with what we said we were going to do, and luckily and through our hard work, we’ve seen some results.”
Longevity of collaboration: not tied to a single initiative
“It…goes back to the old saying [that] it really does take a village to a raise a child, and…when it comes to transition, I think [the state transition team] is a village.”

31. Time Together
Site visits, conferences, training events, planning activities
“We keep showing up. We keep coming back to the table, and I think that’s one of the key things, because you can have partners fade out easily. That’s probably been a strength: longevity.”

32. Shared Vision Annual strategic planning
“Ultimately our purpose is to find ways to improve student outcomes for graduation and dropout and employability and transition planning.”

33. Shared Leadership
“Everybody feels empowered to jump in and commit to whatever they feel they can bring to the table.”
“Brainstorming and voting” to reach consensus
“When local teams see a state team that’s working together, it can be very inspirational to them…We’re modeling.”

34. Site Visits
Vocational training centers, career and technical education schools, model transition programs, women’s prison
“That was very eye-opening because it gave us the opportunity to see different pieces [which helped us see the] whole piece and how everybody fits into the process.”

35. Joint Planning Implementing statewide online career exploration tool
Collaboration with the National Dropout Prevention Center for Students with Disabilities (NDPC-SD)
Targeting state needs through reviewing multiple agencies’ data during annual retreats

36. Joint Training Attending conferences together
Inviting national speakers to joint trainings
Inviting staff from multiple agencies to trainings and state conferences
Multiple agencies presenting to community transition teams
“We all look at everything from our perspective [of] what we do for transition, and we don’t realize that there are whole parallel groups of people out there who are also working to accomplish the same thing…This has allowed us to say, ‘Okay, you’re doing that. Maybe we can do a joint training or maybe there are things we can share…,’ which makes us all stronger.”

37. Sharing Information
“I can point people in school districts or parents to other agencies, because I know more about what they do now than I did before and hopefully help them make good connections for their kids as well.
The group “has been very, very helpful…for passing on what we’ve learned to people in the field...and start doing the work at the local level.”