Successful Online Implementation Depends on a Culture of Trust Bethel University College of Arts and Sciences Dr. Richard Sherry Dean of Faculty Growth and Assessment

IDEA Use at Bethel, TypeSectionsPercentage Diagnostic, Paper394.5% Short form, Paper % Diagnostic, Online % Short form, Online % Total865

Shift to Online Evaluation

Normal Use of IDEA Context of faculty evaluation –History of use at Bethel General policies for use –Known policies that allow flexibility –General confidentiality –Use in a formative context –Evaluation and multiple voices –Acknowledgment of value and consequences

Who speaks on teaching? Annual Reappointment Faculty self- assessment Department chair observation, review Course evaluations Additional observations if desired Promotion/Tenure Extended self- assessment Department chair Course evaluations 3 peers, including teaching 6 alumni Additional if desired

Faculty Development and IDEA New Faculty Orientation* Guidance to faculty on interpretation and application Approaches by personnel committees to interpretation* CRICRI Follow up with Chair, Faculty Development Coordinator, or Dean Direction for specific sessions on pedagogy, curricular discussions

Committees Reading IDEA Understand the scale Take the long view (trends, types of classes) Understand the context (kind of course) Consider response rates and distribution Look for key indicators Consider the objectives Read the comments FDIFDI

Faculty Concerns about Ratings Earlier skeptical research repeatedly cited ( google: misconceptions "student course evaluations") IDEA’s approach and “comparison” Student ratings of instruction and the “amenity” culture 99 “One bad course” Time invested Ceiling effect Exploring a new course and “grace”

NO OTHER PROFESSOR uses the so called "university wide grading scale". A’s are generally accepted to be a good grade, and C’s are not good. No matter what you might think. This is a completely ridiculous grading system, as it makes the GPA go down in relation to other classes that don’t use this same scale.... Everyone else I know in other college writing classes is getting an A. I’m not saying that everyone deserves an easy A, but I know that I, and many other students work really hard to improve paper after paper, and it just doesn’t happen. FCRFCR

Pilot of IDEA Online and Growing Acceptance The context for IDEA Online: reasonably robust technology Recruiting pilot cases Reasonable lead time Massive follow-up Successful reporting

Engagement with Faculty on Concerns Response rates Ratings differences “Ganging up” and “unsupervised” evaluations

Continuing to respond to technology and faculty Improvements in our processes Customizing group reports Customizing central questions –How has this course helped you connect this area of study to a Christian perspective? –How has this course contributed to your development as a thinking Christian? –How do you feel this course is influencing your life as a believer?

Anatomy of a Failure Implementation at College of Adult and Professional Studies/ Graduate School Neglect of the context Inattention to politics Failure to prepare students and faculty and communicate well

Reminders Communicate policies clearly Be appropriately flexible about evaluation Clarify “voice” and “weight” of ratings Create formative context Provide guidance on implementation and interpretation Provide guidance on committee use

Reminders (2) Communicate to students on value of ratings Manage response rates through monitoring technology Prepare students for the forms Reassure supervisory staff