1. Innovative Educators Webinar December 1, 2010 Jan Norton, Presenter

2.  Terminology  The Black Box  Timetables  Data/Measurement  Data/Measurement Cautions  Program Outcomes & Examples  Student Outcomes & Examples  Closing & Questions

3.  Visions  Missions  Values  Goals  Objectives  Outcomes  Program vs. Student Outcomes  Learning Outcomes

4. . Something Happens

5.  Actions leading to specific, observable results  What students can expect to learn  How students or programs will be different as a result of what happens in the box  What knowledge, skills, and abilities will be improved  What measurable changes occur in tutoring programs

6.  Scope of assessment  ≥ 6 years  2 years  1 year  1 semester  1 course  1 assignment  1 tutoring session

7.  Planning for Assessment  Continual processes  Multiple times per semester  Once or twice per semester  Two or three times per year  Once per year  Every two years  Every 4 years

8.  Why Measure  What to measure  Before (Benchmarks)  During  After (Outcomes)

9.  How to measure  Program-generated data  Course-specific data  Campus data  Campus surveys  Standardized tools  Locally developed tools

10.  More about locally developed tools  Surveys  Likert-type scales  Explicated scales  Behavioral rubrics  Self assessment by student  Tutor assessment  Observer assessment

11.  Direct vs. Indirect Measures  Data access and consistency  Realistic limitations  Cooperation  The Baseline Challenge  Time & money (and staff and comfort and ……)

12.  Possible Areas of Assessment Focus  Establishing baselines  Usage  Satisfaction or quality  Level/variety of services  Staff development  Reaching certification or meeting standards

13.  Outcome Statement  As a result of the new marketing strategy, the number of students attending group tutoring sessions will increase by 10% in Spring 2011.  Timetable  One semester  Starting Measure (Benchmark)  Average of past attendance  Ending Measure  Workshop attendance records

14.  Outcome Statement  During 2010-11, the number of tutors who earn CRLA Level I certification will increase by 5%.  Timetable  One academic year  Starting Measure (Benchmark)  Number of tutors achieving Level I in previous year(s)  Ending Measure  Number of tutors achieving Level I in 2010-11

15.  Outcome Statement  Outside reviewers using the NADE self-study process will score the tutoring program at 85% or above on all sections.  Timetable  One year  Starting Measure (Benchmark)  Internal review & self-study  Ending Measure  External review

16.  Outcome Statement  As a result of tutor training sessions, tutors will demonstrate an improved understanding of Bloom’s Taxonomy.  Timetable  One semester or year, depending on training schedule  Starting Measure (Benchmark)  Pre-test  Ending Measure  Post-test

17. TYPES  Affective  Build confidence  Improve attitude  Reduce anxiety  Cognitive  Mastery of material  Conscious strategies use MEASURES  Qualitative  Interviews & focus groups  Opinion surveys  Observations or self reports  Quantitative  Assignment & course grades  Instrument scores  Observation rubrics

18.  Outcome Statement  After being tutored for at least one hour, clients will be more confident in their ability to accomplish the academic task they focused on in tutoring.  Timetable  One tutoring session  Starting Measure (Benchmark)  Client self-report  Ending Measure  Client self report

19.  Outcome Statement  Students who work with the study skills tutors will demonstrate an increased ability to use memorization strategies.  Timetable  One tutoring session  Starting Measure (Benchmark)  Tutor or observer checklist  Ending Measure  Tutor or observer checklist

20.  Outcome Statement  75% of the students who work with a reading tutor for at least two sessions will score at least 75% on the main ideas reading test.  Timetable  Within three weeks  Starting Measure (Benchmark)  Reading quiz at start of tutoring  Ending Measure  Chapter reading test in class

21.  Outcome Statement  Students who attend at least 10 group tutoring sessions for Algebra Review 090 will earn an average grade of B or above in the course.  Timetable  One semester, length of course  Starting Measure (Benchmark)  Placement score into course  Ending Measure  Course grade

22.  Outcome Statement  After attending mandatory tutoring for all 5 of their developmental education courses, students will be retained until the start of their junior year at a rate at least 25% higher than developmental students for whom tutoring was not required.  Timetable  Two years  Starting Measure (Benchmark)  Developmental course placement  Ending Measure  Enrollment level

23.  Outcome Statement  Students attending SI for World Religions will “[apply] previously understood information, concepts, and experiences to a new situation or setting.” (CAS)  Timetable  Within one semester  Starting Measure (Benchmark)  Pre-survey  Ending Measure  Post-survey

24.  Check with your institutional research or assessment office  Review early drafts of outcomes statements with your supervisor  Pre-test any locally created measurement tool  Be prepared for outcomes that are not met or wildly exceeded  Remember the black box: we do not own or control the box, and our ability to see into it is very limited