1. HOMEWORK REVIEW Establishing Baseline Assessments
Student learning outcomes
Course completion rates
Traditional course costs (CPT)

2. C2R ASSESSMENT GOAL
To establish the degree to which improved learning has been achieved as a result of the course redesign.

3. ASSESSMENT PLANNING Step 1. Establish the method of obtaining data.
Step 2. Choose the measurement method.

4. ESTABLISH THE METHOD OF OBTAINING DATA
Baseline “Before” (traditional) and “After” (redesign)
Parallel Sections – Compare traditional sections and redesigned sections (Fall 2008)

5. CHOOSE THE MEASUREMENT METHOD: FIVE MODELS
A. Comparisons of Final Exams
B. Comparisons of Common Content Items Selected from Exams
C. Comparisons of Pre- and Post- Tests
D. Comparisons of Student Work using Common Rubrics
E. Comparisons of Course Grades using Common Criteria

6. FIVE MEASUREMENT METHODS
Common Finals: Coppin State U, Santa Fe College
Pre- and Post-Tests: Edison State College
Multiple Methods: El Paso CC (A & C), Morehead State U (A, C, E), UMES (A, B, E), UNC Charlotte (C & D)
Unknown: Regis U, U of Minnesota, UW Bothell
Common Finals: Saint Leo U, SELU – good
Common Content Items: Oklahoma State U, Western Michigan U (Why not common exams? WMU – 10 questions – what %)
Pre- and Post-Tests: Austin CC
Multiple Methods: ASU (A & D), Auburn U--Physics) (A & B – don’t need B unless doing sub-analysis), NYIT (A, B & E), U of West Alabama (A & C; Really D - rubrics for essays), UCF (A, C, E)
Grades: UMass Lowell - Grades based on 4 hourlies & a final; anything else?
Unknown: Auburn U (Math) – need to decide

7. GRADES ARE NOT A SUFFICIENT MEASURE OF STUDENT LEARNING
Lack of consistency
Different coverage
Different tests and exams
Curving
Inflation
Grades: UMass Lowell - Grades based on 4 hourlies & a final; anything else?
UCF, NYIT – have other measures, omit grades
Use only for course completion!

8. More Is Not Better! Performance in follow-on courses
Student attitude toward subject matter
Student interest in pursuing further coursework in the discipline
Differences in performance among student subpopulations
Deep vs. superficial learning

9. If you have data If you do not have data
Indicate the method you used to obtain data with specifics (# of students, # of sections)
Indicate how you will measure learning with specifics (A – E) and report the results
If you do not have data
Indicate the method you intend to use with specifics (# of students, # of sections)
Indicate how you will measure learning with specifics (A – E) and describe
This is what we asked you to do for your homework.
6 of you have the data, 6 of you do not but only 2 of you reported the data (Austin CC and W Michigan U.)
WMU – inscrutable – “Average of 10 embedded questions = 59.5% + 3.8%”
Remember: 100 minimum in each group

10. Here is what we will ask you to submit with your final proposal in June.
Go over each item.
At least 100 students
(not 2 sections of 25 each – not meaningful)
Fall and spring problem

12. COURSE PLANNING TOOL
A decision-making tool that enables institutions to compare the “before” activities and costs (the traditional course) and the “after” activities and costs (the redesigned course)

13. THE CPT - IS IT WORTH IT?
Provides a structure for you to think about activities and costs
Allows you to consider changes in specific instructional tasks
Permits you to visualize duplication and waste
Enables you to do a cost/benefit analysis re: type of personnel per task

14. ACTIVITIES AND COSTS
Determine all personnel costs expressed as an hourly rate.
Determine the specific tasks associated with offering a course.
Determine how much time each person spends on each of the tasks.
Calculate the total instructional costs.
Redesign the course by task and re-calculate the costs.
Compare the cost-per-student.

15. Instructional Costs per Hour

16. PERSONNEL COSTS Spreadsheet 1
Be sure to include benefits in personnel costs.
Include all personnel types on sheet 1.
Use an average salary by faculty type/role for one section rather than showing each individual salary.
Show one average salary for each type rather than an aggregate.
Use the same personnel costs on all 3 sheets.
Calculation for faculty cost per hour should be for one person teaching one section.

17. Traditional Course Preparation

18. Traditional Course Delivery

19. COST-PER-STUDENT
There may be good reasons for differences, but frequently big differences mean that you have made errors in your CPT draft.
We suggest that you look at like institutions or disciplines to see if you can figure out what is the cause of the difference.

20. IN-CLASS TO OUT-OF-CLASS
Page #1
Typically 45 in, 90 out, 135 total
Most institutions have a rule-of-thumb expectation of something like two hours prep time (out-of-class) for each hour spent in class.
Seems to be a trend in Round III of underestimating outside-of-class time – 2 for 1 was not a common ratio for this group.

21. DO #1 AND #2 CORRESPOND?
The number of hours (in-class plus out-of-class) on the Instructional Costs per Hour sheet (#1) and the Instructional Costs of Traditional Course sheet (#2) should correspond.
This correspondence helps guard against unrealistic estimates of how personnel spend their time on the individual course tasks.
By using this expectation, we avoid the "self-reporting" problem.
Establishing this rule-of-thumb expectation as a framework also encourages faculty members and others to be more realistic when estimating time associated with specific instructional tasks.
Thus, the total on sheet #2 should roughly correspond to the total on sheet #1, which expresses the institutional “rule of thumb.”

22. Redesigned Course Preparation

23. Redesigned Course Delivery

24. MISCELLANEOUS CPT assumptions Section vs. whole course
Read the instructions
Be sure to consult the definitions
Explain worksheets in CPT narrative
Do not add worksheets to CPT except . . .
When more than 3 worksheets are needed
Section vs. whole course:
If a simple extrapolation like UW - show section and multiply
(Be careful re: multiple sections and prep issue)
Otherwise show whole course
e.g., diverse instructional arrangements – personnel, sections per semester 
IX. When You Need More Than 3 Spreadsheets
Speculation vs. fact (what to include in spreadsheet vs. narrative)
So Maine conduct distance courses with savings; count revenue (no - speculative)
If you plan to reduce attrition or increase academic success, please be as specific as you can as to how you plan to do it and why you think you can. (UCF 7% yes; MDCC no)
Below the line items
More than 3 spreadsheets
UB - 2 redesigns
U of Idaho - 3 courses into 1
Colorado - Tri model – C vs. Q– old and expensive, cheap but low quality, new, cheap and good
Redesigns of redesign (Umass, Cal Poly & CMU)

25. COURSE STRUCTURE FORM
A formatted spreadsheet that enables institutions to compare the structure of the traditional course with the that of the redesigned course (types of sections, number of students enrolled and the kinds of personnel)

29. FOR MORE INFORMATION www.theNCAT.org