1. Summary of CPHS Course Evaluations: AY 2007-08 and 2008-09 Chuck Phillips August 2009

2. IDEA Evaluations: Overview Summary StatisticsAY 07-08AY 08-09 Number of Classes Evaluated9981 Average Class size7580 Overall Response Rate75%76% On-line Response Rate Mean (s.d.)64% (s.d. 21%)65% (s.d. 19%) Range0-100%33-100% Old 192 and small classes In-class Response Rate Mean (s.d.)91% (s.d. 8%)93% (s.d. 6%) Range69-100%76-100% Average # of Objectives: E or I4.3

3. Response Rate Issues Low response rates on-line were for either the old therapeutics (192 with several instructors) or classes with very small n (a few electives). Expect average response rate to increase in 09-10 for on-line evals

4. Class format Increase in lecture probably due to fewer individual lab sections being evaluated.

5. AY 2007-2008 AY 2008-2009 Average% of classes below 3.0 % of classes 4.0 or above Amount of Reading CPHS2.8 3.0 ✓ 52% 41% 8% 16% IDEA3.233%15% Amount of work in other (non- reading) assignments CPHS3.4 3.3 34% 32% 28% 20% IDEA3.421%18% Difficulty of subject matter CPHS3.4 36% 33% 31% IDEA3.420%18% Values are similar if within 0.3 1=Much less than most courses, 2=less than most, 3=about average, 4=more than most, 5=much more Amount and Difficulty of Course work: Student Ratings

6. Instructor Related Course Requirements (Some or Much required) Reading and memorization were new categories in 08- 09

7. Instructor Related Course Requirements 2008-09 Breakdown

8. Percent of CPHS classes selecting objective as either Essential or Important (FIF)

9. Student ratings of progress on objectives chosen as Essential or Important 1=no progress 2=slight progress 3=moderate progress 4=substantial progress 5=exceptional progress

10. Percent of Reliable CPHS Classes in each Category vs. IDEA (Adjusted scores*) Expected Distribution A. Progress on relevant objectives B. Excellence of teacher C. Excellence of course Summary (Average of A, B, C) 07-0808-0907-0808-0907-0808-0907-0808-09 Much Higher10%1%0% 4%3%2%1% Higher20%15%13%25%27%19%20% 18% Similar40%53% 53% ✓ 46%39%41%35%46%51% Lower20%13%21%17% 27%15%23%14% Much Lower10%18%13%12% 17% ✓ 9% 27% ✓ 9%16% >30% in top 2 categories: teaching effectiveness appears to be superior to IDEA Database * Results adjusted for 5 factors: Student motivation to take the class regardless of who taught it; Student work habits; Class size; Student effort not attributable to the instructor; Course difficulty not attributable to the instructor

11. CPHS Adjusted Mean Scores A. Progress on relevant objectives B. Excellence of teacher C. Excellence of course Summary (Average of A, B, C) 07-0808-0907-0808-0907-0808-0907-0808-09 All CPHS Courses3.8 4.14.03.83.73.93.8 IDEA System Courses3.8 4.2 3.9

12. Overview On-line evaluations still have lower response rate  Improving with use of fewer ‘co-teachers’ but can improve more CPHS similar to national database on:  Amount of reading  Amount of work in non-reading assignments  Difficulty of subject matter  Do our students have higher expectations?

13. Overview Ave. number of objectives still 4.3  3-5 recommended as max.  High on CT, Memorization, reading For computer application, math/quant, & creative/artistic: Almost none require “Much”

14. Overview Primary class format remains lecture  Should more active learning be stressed? Faculty believe more of our courses address “Learning to apply course material” & “Developing specific skills, competencies, and points of view” compared to IDEA database

15. Overview Students rate us lower on:  Learning to analyze & critically evaluate  Developing clearer understanding and commitment to personal values  Developing skills in expressing myself orally or in writing  Gaining broader understanding & appreciation of intellectual/cultural activity

16. Overview We do better in  Excellence of Teacher  Excellence of Course  Vs. Progress on relevant objectives But all three are lower than expected Everyone needs development: almost no course or instructor in 90 th %ile

17. Questions to Consider Are we balancing teaching methods and course requirements as intended Are we emphasizing correct objectives Are courses rigorous? What has been the affect of the new curriculum? Do we need to require computer application, math/quantitative?

18. Questions to Consider Too much lecturing? Disconnect on what faculty say vs. students Where should we improve?? Others?

19. Continued Assessment ‘When you dance with a bear, you can’t quit just because you’re tired” (Russian proverb) Need to continue assessment work Need to improve toward goals Better measure student outcomes