1. Comprehensive Evaluation: Institutional Effectiveness Committee Recommendations Presentation to College Council
Office of Institutional Effectiveness & Institutional Effectiveness Committee
November 11, 2017

2. Background Evaluate Continuous Improvement Processes (3 year cycle)
Program Review
Student Learning Outcomes (SLO)/Administrative Unit Outcomes Assessment
Planning
College Council reviewed/informed of:
Evaluation plan (fall 2016)
Data collection efforts (spring 2017)
Preliminary themes presented (spring 2017)

3. Discussion Plan Institutional Effectiveness Committee (fall 2017)
Review detailed results
Make recommendations for improvement
Review Institutional Effectiveness Committee recommendations with campus
College Council (November 2017)
Planning councils (December February 2018)
Academic Senate – 10+1 (December February 2018)
Classified Senate (December February 2018)
Final Recommendations
Academic Senate 10+1 (March 2018)
College Council (March 2018)

4. Evaluation Process Two component evaluation: Content:
Focus Groups with each of the four wing planning councils, Academic Senate, & Classified Senate
Campus-wide perception and opinion survey
Content:
Program Review process & structure
SLO/AUO Assessment process & structure
Planning process & structure
Support
Timeframe
Impact
Communication
TracDat

5. Main Themes & Considerations
Desire to keep processes the same for a period of time
Lessen the culture of fear
Dialogue & utility of processes needs improvement
AUO data collection, peer review, staff/management hiring, staff development
Some indication process cycles are too short
Integration across processes not easily observable
College goals not always driving plans, linked afterwards
Broaden participation, communication and training

6. Program Review
Develop a stronger mechanism for departments to identify and request department specific data for instructional programs. Consider redesigning current form, timeline for requests and communications about the process.
Improve peer review process for both instructional and support departments. Consider process, participants, training and timeline.
Redesign SLO/AUO synthesis inside program review to be more meaningful and less redundant. For instructional wing, consider SLO area to be more focused on program SLO evaluation.
Improve prompts and support for CTE faculty completing both the biennial and core indicator areas in Program Relevancy.

7. SLO/AUO Assessment No changes to course SLO process.
Evaluate each department’s AUOs/KPIs to ensure they are measurable and meaningful.
Clarify AUO process, three-year cycle and training materials to emphasize that AUOs/KPIs can change annually as needed.

8. Planning/Annual Resource Requests (ARRs)
Better articulate the relationship between college goals/objectives/priorities, departmental planning strategies and SLOs/AUOs.
Strengthen planning strategies to be less focused on resource needs and more focused on improvement.
Improve the integration of categorical, ancillary and ASOCC monies into the planning/ARR process.
Improve classified and management staffing ARR process and prioritization of positions. Consider linkage to staffing plans or as a separate process.
Clarify the prioritization and funding mechanism for Professional Development ARRs.
Increase communication about the results of Annual Resource Requests to departments and the campus.

9. TracDat Improve navigation and streamlining of program review module.
Develop training videos to help on demand support.
Improve tracking reports of processes in TracDat.
Support further integration of technology into TracDat (e.g., Canvas, Microsoft BI).

10. Overall – All Processes
Increase classified staff participation and collaboration in all phases of program review, AUOs and the planning process.
Dialogue about results of program review, SLOs and planning strategies outside of the process. Consider department, division or wing meetings as possible discussion arenas.
Consider length and alignment of process cycles.
Ongoing training of processes to keep purpose and outcomes in the forefront.

11. Comprehensive Evaluation Report Data Analyzed by Office of Institutional Effectiveness Click Here for Details

12. Comprehensive Evaluation: Preliminary Themes from Focus Groups Presented to College Council May 2, 2017

13. Preliminary Themes: Program Review
Improved since prior cycle with data analysis and distinction between Instruction/Support wings
More program-specific data support needed
CTE areas duplicating external accreditation
Tying CSLO/AUO results into program review not clear/seems redundant
Peer review process needs refining (training, timing, incorporating feedback)
Classified staff involved to varying degrees (mostly not involved or only on front end; no closing the loop)

14. Preliminary Themes: SLO/AUO Assessment
SLOs compliance-focused and not being used in meaningful way
Culture of fear surrounding assessment
Feels punitive, so results don’t show need for improvement
AUOs useful, but cumbersome
Data collection problematic (sustainability, difficulty getting data, unsure of data needed)
Need help developing assessments/KPIs

15. Preliminary Themes: Planning
Integration across processes not easily observable
College goals not always driving plans; typically addressed after strategies developed
ARR and BSB processes working to get resources
Perception some use BSBs in lieu of ARRs
Staff/Management ARR prioritization/funding not meeting department demand and not reflecting “critical” or “mass” campus needs
ARR decisions not being communicated widely

16. Preliminary Themes: Overall
Overall perception that only minor tweaks are needed in processes
Desire to keep processes the same for a period of time (changes burdensome to re-learn)
Provide ongoing training for all processes for all constituents (keep fresh)
Lessen culture of fear
Dialogue outside of processes needs improvement
Concerns about broad enough participation within departments (“experts” within departments do it all)
Results from program review/assessment primarily being used for ARR justification

17. Preliminary Themes: TracDat
PROS:
Data collected in central location
Reporting/summarizing capabilities
Better than prior MS office based collection
CONS:
Navigation within TracDat not intuitive
Reports hard to read & access
Involves continuous training to use