1. Process Level Evaluation of Internal Productivity, Communication, and Collaboration SIGnetwork Grant Management PLC Anna Harms March 26, 2012

2. Agenda Review MiBLSi’s Value Added Work System (Logic Model) Three Levels of Evaluation Process Level Evaluation: What, Why and How Next Steps

3. Management/CoordinationManagement/Coordination Capital Resources PeopleMaterialsInformation Work Systems: Providing the MTSS practices and the supports for the these practices to take place successfully within schools and districts FinancialEvaluation Professional Learning Technical Assistance Political Regulatory/Legal Economy Geography Society/Culture Political Regulatory/Legal Economy Geography Society/Culture Environmental Factors Receivers Stakeholders /Funders Investments: Funding Visibility Political support Returns: Addressing critical issues (Discipline/Ethnicity) Addressing program directives (State Performance Plan) Consumers schools, districts, ISDs/RESAs Consumers schools, districts, ISDs/RESAs MiBLSi Value- Added Work System Valued MTSS Products/ Services Priority, Plan, Formative Assessment, Continuous Improvement, Risk Management Determine worth

4. MiBLSi Organizational Structure

5. Set up Evaluation Systems LevelDescriptionTools OrganizationalEvaluating impact of the RtI project Stakeholders Addressing critical issues (e.g., Discipline/Ethnicity) Addressing program directives (e.g., State Performance Plan) Consumers Valued results (e.g., Improved quality program, fidelity of implementation, successful student outcomes) Annual reports Measures of program quality Measures of fidelity of implementation Measures of student outcomes Survey of consumer satisfaction ProcessEvaluating operational productivity and efficiency of each unit within organization Project management tools FastTrack Basecamp, Manymoon Performer (worker) Evaluating worker productivity and efficiency Job models

6. POLL: Which level(s) does your project evaluation emphasize? Organizational Process Individual Worker All A Combination of the Above

7. Procedures WhatWhenWho Annual Master Plan is Developed Early in the school year (by mid August) Director, Assistant Directors

8. Annual Master Plan

9. Procedures WhatWhenWho Annual Master Plan is Developed Early in the school year (by mid August) Director, Assistant Directors Projects are entered into a FastTrack Gantt Chart Early in the school year (by mid August) Unit Coordinators

10. Sample Portion of FastTrack Schedule

11. Procedures WhatWhenWho Annual Master Plan is Developed Early in the school year (by mid August) Director, Assistant Directors Projects are entered into a FastTrack Gantt Chart Early in the school year (by mid August) Unit Coordinators Projects are entered into Manymoon project management software and specific tasks are assigned Early in the school year (by mid August) and ongoing (at least monthly) Unit Coordinators

12. Manymoon: Project Management

13. Manymoon

14. Tasks

15. Procedures WhatWhenWho Annual Master Plan is Developed Early in the school year (by mid August) Director, Assistant Directors Projects are entered into a FastTrack Gantt Chart Early in the school year (by mid August) Unit Coordinators Projects are entered into Manymoon project management software and specific tasks are assigned Early in the school year (by mid August) and ongoing (at least monthly) Unit Coordinators Provide a status update of work accomplished, project ability to complete future work, and find solutions to barriers QuarterlyUnit Coordinators, Program Team

16. Problem Solving with Unit Data Gather Unit Data / Problem Identification, Description Problem Analysis Plan Development Plan Implementation

17. Unit Status Update in GoogleDocs

18. Problem Identification / Description List the activities that were not accomplished on time. How close were we to the goal (days, weeks)? Does a type of task / project seem to be the problem? What is the impact of not having the task / problem completed on time (within the project and external to the project)?

19. Problem Analysis Were non-complete tasks / projects a priority to begin with? Was the goal realistic? Were resources properly allocated? Were unanticipated task demands added?

20. Plan Development Specific projects / tasks for which problem solving is needed Adjust timeline / due date Reallocate resources Remove other tasks Prioritize this work and demote other work

21. Plan Implementation Adjust tasks and timelines in FastTrack and Manymoon.

22. Procedures WhatWhenWho Annual Master Plan is Developed Early in the school year (by mid August) Director, Assistant Directors Projects are entered into a FastTrack Gantt Chart Early in the school year (by mid August) Unit Coordinators Projects are entered into Manymoon project management software and specific tasks are assigned Early in the school year (by mid August) and ongoing (at least monthly) Unit Coordinators Provide a status update of work accomplished, project ability to complete future work, and find solutions to barriers QuarterlyUnit Coordinators, Program Team Conduct interviews to gather feedback on collaboration and communication within and between units Quarter 3 (by end of June)External Evaluator

23. Interviews Scheduled in April External Evaluator will be conducting a combination of individual and group interviews Will summarize data and help us use the feedback to build stronger internal structures

24. Next Steps Early August: Pull together all data from 2011-12 Use data to inform 2012-13 Master Plan Reflect on the evaluation process and plan for adjustments that will allow for greater efficiency and systems to support ongoing data-based decision making

25. Discussion Process level evaluation data is not typically required for grant reporting. Are you gathering this type of data? What ideas do you have making process level evaluation valuable?

26. Three Evaluation Systems LevelDescriptionTools OrganizationalEvaluating impact of the RtI project Stakeholders Addressing critical issues (e.g., Discipline/Ethnicity) Addressing program directives (e.g., State Performance Plan) Consumers Valued results (e.g., Improved quality program, fidelity of implementation, successful student outcomes) Annual reports Measures of program quality Measures of fidelity of implementation Measures of student outcomes Survey of consumer satisfaction ProcessEvaluating operational productivity and efficiency of each unit within organization Project management tools FastTrack Basecamp, Manymoon Performer (worker) Evaluating worker productivity and efficiency Job models October 23, 2012 Evaluators’ PLC

27. POLL: I gathered information from today’s call that I can use to inform grant management and evaluation for my state’s SPDG Strongly Agree Agree Disagree Strongly Disagree

28. Contact Information Anna Harms Evaluation and Research Coordinator Michigan’s Integrated Behavior and Learning Support Initiative aharms@oaisd.org (616) 796-1425