Integrating Measurable Outcomes into the Work of Teams Michael Fosmire LTF 6 April 7, 2006

Map to the talk Background – issues leading up to team development Background – issues leading up to team development Role of Outcomes in team management Role of Outcomes in team management Support for outcomes development Support for outcomes development Outcomes of outcomes Outcomes of outcomes Audience participation Audience participation

Organizational Challenges Driver of change: IT-Pub Svcs integration Driver of change: IT-Pub Svcs integration but also: but also: Empowerment of staff Empowerment of staff Communication across units Communication across units Justification of additional resources for the library Justification of additional resources for the library University-level requirements for assessment and reporting University-level requirements for assessment and reporting

Background Problems communicating between IT and Public Services units Problems communicating between IT and Public Services units 1999—Ad Hoc Team for Creating a User Centered Technical Environment created to address the challenge 1999—Ad Hoc Team for Creating a User Centered Technical Environment created to address the challenge Solution: Team based structures, not only for IT, but also purely Public Services functions (Reference, Instruction, Circulation, etc.) Solution: Team based structures, not only for IT, but also purely Public Services functions (Reference, Instruction, Circulation, etc.)

Org Chart (Old)

Org Chart (New) EACT unique: provides structure for teams/units—doesn’t ‘do’ anything

Effect on Organizational Structure Decisions can bubble up from teams Decisions can bubble up from teams Suggestions can ‘bubble down’ from managers Suggestions can ‘bubble down’ from managers Provides wider capacity for input from staff across the libraries Provides wider capacity for input from staff across the libraries

Ok, so what about outcomes? Major challenges: Major challenges: Instill structure for accountability in work of teams Instill structure for accountability in work of teams Help teams prioritize their work Help teams prioritize their work Solution: EACT – Developing outcome-based planning process (a posteriori realization) Solution: EACT – Developing outcome-based planning process (a posteriori realization)

EACT -- development Team started with very little installed knowledge base Team started with very little installed knowledge base No clear mandate for assessment No clear mandate for assessment Charge to Charge to ‘Create a culture of assessment’ ‘Create a culture of assessment’ Assist in assessments of teams and units Assist in assessments of teams and units

Evolution Saw need to: Saw need to: Get administrative buy-in that assessment was important Get administrative buy-in that assessment was important Integrate assessment planning into overall planning program of teams Integrate assessment planning into overall planning program of teams

Why Integrate Outcomes Moves focus from the project to the users Moves focus from the project to the users Helps staff to understand impact of their work Helps staff to understand impact of their work Reporting to the University level Reporting to the University level

Planning Document Identify challenges and opportunities facing the team in the coming year. Identify challenges and opportunities facing the team in the coming year. What are your goals that address the challenges/opportunities? What are your goals that address the challenges/opportunities? What are your measurable outcomes, i.e., how will you determine if you met your goals? What are your measurable outcomes, i.e., how will you determine if you met your goals? What resources are needed to achieve goals? (funding, borrowing personnel, speakers) What resources are needed to achieve goals? (funding, borrowing personnel, speakers) Special considerations–system wide impacts; coordination with other teams or units; communication/coordination with users Special considerations–system wide impacts; coordination with other teams or units; communication/coordination with users Asks the following questions:

Planning Document Outcomes Every team includes outcomes in their planning document Every team includes outcomes in their planning document Includes statement of how they will measure their outcomes Includes statement of how they will measure their outcomes Challenge: How do teams get the skills to create their outcomes?

EACT Support for Planning Team Leader/Member training at beginning of ‘team year’. (staff bring goals from the team) Team Leader/Member training at beginning of ‘team year’. (staff bring goals from the team) Training for all staff as part of Performance Management system Training for all staff as part of Performance Management system Individual consultations during planning process Individual consultations during planning process

Audience: who is your service designed for? Behavior: what performance or action is expected by the user?Condition: Terms under which assessment takes place Simplified Outcomes Worksheet Goal: Goal: Libraries Strategic Direction(s) Addressed: Libraries Strategic Direction(s) Addressed: Activities that support your Goal: Activities that support your Goal: Outcome Indicator Target Accomplishment Outcome Indicator Target Accomplishment Outcome Outcome Indicator Target Accomplishment Indicator Target Accomplishment Focus on Outcomes vs Outputs Use ABC Method (Modified from IMLS Outcomes Training)

Success? Outcomes of the planning process: Outcomes of the planning process: 8 teams had measurable outcomes 8 teams had measurable outcomes 3 teams had outputs (2 had no training) 3 teams had outputs (2 had no training) Consultations with EACT on assessment increased (about 50%) Consultations with EACT on assessment increased (about 50%)

Examples of Assessments Circulation/Reserves Circulation/Reserves “Secret Patron” assessment of student worker performance “Secret Patron” assessment of student worker performance Results provide feedback into their student worker training program Results provide feedback into their student worker training program C/R team devised questions, process, timeline C/R team devised questions, process, timeline EACT consulted and recruited ‘secret patrons’, submitted raw data C/R team for scoring EACT consulted and recruited ‘secret patrons’, submitted raw data C/R team for scoring Reference Services Team thinking of using method Reference Services Team thinking of using method

Serials/Binding (2) Tours of all the Libraries serials operations: Tours of all the Libraries serials operations: Led to changes in display, processing of serials Led to changes in display, processing of serials Best practices shared across the libraries Best practices shared across the libraries Assessed with survey of Assessed with survey of ‘what changes did you make?’ ‘what changes did you make?’ ‘how has it affected your workflow?’ ‘how has it affected your workflow?’ ‘what has been the effect of your changes?’ ‘what has been the effect of your changes?’

Results Teams can articulate outcomes Teams can articulate outcomes Teams are conducting assessments Teams are conducting assessments Staff comfort level with concepts, vocabulary has increased Staff comfort level with concepts, vocabulary has increased Functional teams have embraced assessment more than mixed teams Functional teams have embraced assessment more than mixed teams Still tendency to wait until project is over to devise assessment Still tendency to wait until project is over to devise assessment Lots of surveys Lots of surveys

Strengths of Team Approach to Assessment Assistance More depth of experience (more people involved) More depth of experience (more people involved) Revolving composition means dispersal of expertise across system Revolving composition means dispersal of expertise across system Team does own assessment plan, so stays grounded Team does own assessment plan, so stays grounded

Challenges of Team Approach to Assessment Assistance No direct accountability for assessment (consequence of not planning?) No direct accountability for assessment (consequence of not planning?) Synchronizing administrative needs/perspectives with team training Synchronizing administrative needs/perspectives with team training Assessing the assessments— managers have different levels of expertise, experience Assessing the assessments— managers have different levels of expertise, experience Team members do this part time Team members do this part time

Audience Participation What expectations do you have for your teams to do assessments? What expectations do you have for your teams to do assessments? What have been the biggest challenges for teams to assess their outcomes? What have been the biggest challenges for teams to assess their outcomes? How have you addressed those challenges organizationally? How have you addressed those challenges organizationally?

Thank you for your attention Michael Fosmire