1. Program Review Data Collection Project Brought to you by: The Data Delivery Team (DDT)

2. What is the DOI’s Program Review? Over the course of the summer our UHCC DOI’s worked together to provide the system with a set of program review data requirements that would be known as the “Irreducible Core Indicators”. These program review data elements were intended to be the “minimum” data requirements for instructional program reviews. By creating a commonly shared language we will be better prepared to compare programs across campuses.

3. Where did the 13 data elements come from? Since committing to do our program reviews it became painfully clear that we would have to be moving quickly to determine what data we would need at the local level so we could begin to develop a tool to retrieve the data. With this in mind we selected 13 key data elements from the DOI’s list so that we could begin the process and move forward with development. Since we currently do not have a reporting platform that could be used to retrieve this data it was necessary to design and develop a dynamic reporting tool, then test and retest the data output until it was being reported as spec’d.

4. What data is being provided to my program? We have taken the 13 chosen data elements from the DOI’s Program Review document and have gone through a number of iterations to ensure that the data we are providing will prove to be useful. The primary purpose of this presentation is to provide clarity as to how the data is being collected, filtered, and displayed to you. The intention here is to reduce the number of questions you may have so you will feel confident about using the data in your program reviews. Click here for the list of data elements in part III. Program Review

5. A word about the timing of the data snapshot... It’s imperative that the same data snapshot be used when doing analysis on data from different sources (MAPS reports, Program Health Indicators, etc.) The data in Banner is constantly being updated. It differs from the “first day of class” to “census date” to “closing date”. Unless otherwise noted, the data snapshot for the 13 following data elements is: “Semester specific data as of the last update in Banner.” For the most part data is last updated during closing each semester, with a few exceptions: change of grade, incomplete grade make up, and administrative exceptions.

6. 2 Distinct Program Course Categories Program courses are broken into 2 categories: "Student Required Core" (SRC) courses are primary program courses required of the student. This does not include general ed, remedial, or elective courses. For example: SRC courses for Accounting could include: ACC 130, OAT 29, or ICS 101. (note the course alphas are different). These apply to all except #8 - #10. "Program Specific Courses" (PSC) courses are a subset of the SRC, used to help identify staffing sufficiency. Note that the program and course alpha are the same. For example: ACC 130, ACC 20, or ACC 201. These apply to #8 - #10.

7. 1. Number of Majors ( Unduplicated headcount of declared majors for both semesters.) What it means to the DDT: Headcount of ALL registered students with a declared HawCC major for each semester. For the academic year this will be reported as unduplicated headcount by program. Unduplicated headcount is defined as a student in Fall OR Spring semesters, OR both. By program they will only be counted once over the academic year. These are students that were registered as of the start of the semester, regardless of whether they may have dropped later.

8. 2. Student Semester Hours for program majors in all program classes. What it means to the DDT: These are all students where the primary campus is HawCC (PriCamp=HAW) and they had an active registration at some point in the semester. Student semester hours (SSH) are the sum of all attempted credits in all “Student Required Core" (SRC) classes.

9. 3. Student Semester Hours for all program classes. This will give a measure of crossover interest in the program. What it means to the DDT: Student semester hours (SSH) are the sum of all attempted credits in all “Student Required Core" (SRC) classes, regardless of campus or major.

10. 4. FTE Course Enrollment. SSH for all program classes/15 What it means to the DDT: Using the output SSH from item #3 : SSH attempted divided by 15 for any semester. These are students that are declaring ANY primary campus--not just HawCC. FTE for a single semester is SSH/15.This number can be reported for the academic year ((SSH Fall + SSH Spring)/30) as well and reported as an average.

11. 5. Number of classes taught (n). (n) corresponds to (n) in #6 and #7. What it means to the DDT: Number of all "Student Required Core" (SRC) courses. This is each and every section taught for this program. This does not include cancelled sections.

12. 6. Average Class Size. What it means to the DDT: (The sum of all class enrollments)/number of class sections taught (n) from #5.

13. 7. Average Class Fit What it means to the DDT: Average Class fit is the ((sum of enrollments/max enrollment)/number of classes taught) x 100 For example: when enrollments = 15, 18, 20 and max enrollment for each is 30, 20, and 15 respectively and 3 classes are taught: [15/30 + 18/20 + 20/15] = [.50 +.90 + 1.33] = 2.73/3 classes taught =.91 x 100 = 91% Avg class fit.

14. 7. Staffing Sufficiency Up to this point (#1 - #7) we have been using "Student Required Core" (SRC) courses for all calculations. For the following items #8 through #10 we will be using "Program Specific Courses” in order to calculate Staffing Sufficiency. For items #11 through #13 we will return to using "Student Required Core" (SRC) courses for the remainder of the calculations.

15. 8. FTE of BOR appointed program faculty Headcount of all on tenure, tenure track, and multi-year contract faculty What it means to the DDT: This is further defined as Full time and Part time faculty only--not lecturers. FTE is for PSC courses taught by BOR Faculty. This is the FTE that were hired into the program. We will collect this information from the Program Initiators since the information is not kept in Banner. This data will be hard coded into the Program Review.

16. 9. FTE for classes taught in program Credits taught/full teaching load Full teaching load is generally defined as 27 or 21 credits depending on program What it means to the DDT: “Credits taught” here is defined as credits for program specific courses (PSC) taught by both faculty and lecturers. Full teaching load is 27 credits for Liberal Arts programs and 21 credits for CTE programs.

17. 10. Student/Faculty ratio (Total SSH/15)/FTE faculty -- Note: FTE faculty is FTE for classes taught in program (from #9) What it means to the DDT: Item #4 FTE Course Enrollment (calculated using the PSC)/item #9, which is FTE faculty. This will be reported for the academic year using a full teaching load of 27 credits for Liberal Arts programs or 21 credits for CTE programs. The Student/Faculty ratio will be reported when results are >or = to.5, they round up to the next whole number.

18. 11. Unduplicated number of Degree/certificates Note: All degrees and certificates awarded in a program in a year to a single student are counted as one item. What it means to the DDT: The following items, #11-#13 will be reported based on the Student Required Core (SRC) courses. This is the highest degree awarded for the program pulled directly from the degree table in Banner. Reported for the academic year.

19. 12. Cost of program per student major. Regular salaries, overload, lecturer costs, supply and maintenance, and amortized equipment/ (SSH of majors /15) What it means to the DDT: Costs/(item #2/15) for a single semester. Add both semester costs up for the academic year. Regular salaries, overload, lecturer costs, supply and maintenance, and amortized equipment will be included in the data request from the program leads. We will need the cost to run the program before we can run this item in our report.

20. 13. Persistence of majors year to year What it means to the DDT: A student with the declared major that moves from spring from one academic year to the fall of the next academic year. The declared major need not have a passing grade (fail or drop) to be included in the persistence pool. We will be reporting on our 3 year academic period. This means that we will report out Spring 03 to Fall 03, and Spring 04 to Fall 04 persistence.

21. Opportunities The core (program) courses or "Student Required Core" for each program are not flagged as such in the Banner system or elsewhere. This requires us to use the General Catalog and using a "best guess" approach, then a review by the Program Initiators for accuracy. This in itself builds in an additional margin for error. We should consider the possibility of adding a field in Banner so that when the courses are added to the catalog they are accompanied with a code denoting "program course“ (SRC or PSC). This will aid in both speed and accuracy of report development going forward.

22. Opportunities continued… There were also many courses that were reported by program initiators as being courses that needed to be included in their program profile but were NOT listed in the catalog. We need to find out why this is occurring so that we can begin to rely on the courses as they are listed in the catalog. What happens when a course is added after the catalog is printed? How come these courses are not being added the next year? Who is responsible for review of what goes into the catalog to ensure that it is accurate for their program year after year?

23. Opportunities continued… The process of determining what program courses belonged to what programs took way too long. First, a course profile was built based on what was in the catalog. All Math, English, and elective courses were stripped out. The program profiles were then sent out to the Program Initiators for review. Then, the exceptions came trickling in…courses that were not listed in the catalog or Gen Ed courses that actually were part of the core program. Then, we had to go through the exception process for each of these which added to the workload. Then, the program profiles had to be updated with all of the exceptions. All of this because we had no reliable source for what program courses belonged to what programs. Much of this unnecessary work seems to be centered around the fact that the catalog is not being properly validated for accuracy prior to being published.

24. Opportunities continued… For #12 we sent out a template to aid in the collection of program costs…none were returned. We need to clarify when and who should be using contact hour to credit hour conversions. We need to build an example of an amortized equipment schedule and give it to the program initiators. We need to determine which account codes the secretaries need to use in order to determine program expenses—my understanding is that they all have access to FMIS. Instructions need to be built for collecting expense data with examples, and then shared with the Program Initiators and their secretaries.

25. Opportunities continued… There are unique courses in Banner that throw off program review data. For example, independent study/directed study classes, experimental classes, and variable credit classes…these classes often need waiver/substitution statements posted to their transcript. How we handle waiv/subs needs to be further developed. We currently have about 2500 lines of comments. This process needs to be mechanized so we can easily locate and rename courses; then sort, copy and paste them into our program profiles. This procedure will require a time investment for process development and data clean up.

26. Opportunities continued… Are we collecting all of the data of interest? How about… admitted students into a classified program that were not enrolled on the first day of the semester? regarding SSH? Are we only interested in students that stay enrolled? Or are we concerned about any student that drops at any time? If yes, dropped during the erase period, or after the erase period? For example, if 50% of the class withdrew by the end of the semester, do we need to know that data? interest in General Ed or remedial courses?

27. Opportunities continued… Determining Full/Part time Faculty and who is a lecturer will be included in the data request from the Program Initiators until it’s determined how to load the information into Banner and then who will provide the necessary training. Determination of full teaching load (other than the reported 27 or 21 credits already reported for the program) will be included in the data request from the Program Initiators until it’s determined how to load the information into Banner and then who will provide the necessary training. Determination of Regular salaries, overload, lecturer costs, supply and maintenance, and amortized equipment will be included in the data request from the Program Initiators. We need to create an example expense template to help our program initiators better understand what account codes to use and how to handle equipment amortization.

28. Questions? If you would like to discuss how we have arrived at the data for your program please do not hesitate to contact us for clarification. We will work to get an answer to you. Thanks for your support in this important work as we move forward. This is the first time in a long time that program reviews have been required and we are all learning together. Let’s try to enjoy the process! Thanks from the Data Delivery Team. Shawn Flood974-7512sgflood@hawaii.edusgflood@hawaii.edu Marv Kitchen974-7710kitchen@hawaii.edukitchen@hawaii.edu David Loeding974-7753loeding@hawaii.eduloeding@hawaii.edu