Presentation on theme: "The Whistleblower Effect: A Quantitative Analysis Demonstrating that Reporting Students for Academic Dishonesty Negatively Impacts Faculty Evaluations."— Presentation transcript:
The Whistleblower Effect: A Quantitative Analysis Demonstrating that Reporting Students for Academic Dishonesty Negatively Impacts Faculty Evaluations Mihran Aroian
Presentation Outline -Student Judicial Services Overview -Faculty-in-Residence program -Faculty Perceptions -Why Conduct this Research -Results -Recommendations -Questions
Student Judicial Services As a part of The University of Texas at Austin’s Office of the Dean of Students, Student Judicial Services (SJS) fosters moral development on campus through the resolution of academic integrity and conduct-related matters as well as educational outreach to students, faculty, and staff. In addition to our educational component, SJS investigates alleged violations of the Institutional Rules (both academic and non-academic) and implement the disciplinary process with a focus on student learning and development.
Primary SJS Roles Outreach/Education Faculty Consultations Case Management 1,600-1,800 annual referrals (AI and conduct) Institutional Rule Revision
Some Statistics 3,300 faculty 2,500 TA’s 50,000 student body 140 academic departments 2011-2013 academic years 320 faculty/TA’s reported cases to SJS 1,155 AI cases over 2 years 50% of all submissions came from (can you guess how many faculty)? 15 out of 320
SJS Data Administrative disposition versus Faculty Disposition – 55/45 Top 3 AI violations Cheating, Plagiarism, Collusion Classification Freshman (10); Sophomore (20%); Junior (20%); Senior (35%); Grad/Professional (15%) GPA – even distribution Gender (M/F) – 65/35
Establishing a Faculty-in- Residence Program at UT Austin
Academic Integrity As Steve Covey says “seek first to understand, then to be understood” Improving AI is simple “just as plain as the nose on your face” For example: What is the shape and color of a Stop Sign? What is the shape and color of a Yield Sign?
Benefits to SJS Dedicated outreach time not available to typical conduct staff Cultivation of relationships outside of “crisis” events typical in conduct offices Increased exposure through meetings with stakeholders previously unidentified by SJS Ally outside of the office to speak with unaffiliated individuals Addition of a “faculty perspective” to conversations within the conduct office Discussions at the Macro-level (student conduct as a field) rather than at the Micro-level (a student’s conduct)
Benefits for Me! Greater appreciation for SJS Understanding the complexity of student conduct Part of the team Ability to cross boundaries Support SJS with other stakeholders Educate campus stakeholders regarding SJS Involvement with student groups Being part of the solution Satisfying my own needs as an educator
Faculty Perceptions Lack of understanding by faculty as to what constitutes cheating. Not all faculty engaged in high AI standards. Faculty unaware of common forms of cheating. Faculty unwilling to expend effort to meet with students and file paperwork. Faculty avoid student conflict/tension AI is an afterthought in the class
Faculty Perceptions Lack of consistent message to students. High level of indifference by many campus stakeholders. Some faculty do not care because they are not rewarded for teaching. Faculty need assistance on identifying cheating and how to deal with students who do cheat.
The Whistleblower Effect: A Quantitative Analysis Demonstrating that Reporting Students for Academic Dishonesty Negatively Impacts Faculty Evaluations Authors: Mihran Aroian Raymond Brown
Scenario A Three students submit essentially the same writing assignment. Student A provided paper to student B & C Student B & C take responsibility Student A believes he is innocent Students upset at instructor Students are found in violation Students are members of the same student organization as are other students in the class
Scenario B A group presentation of six students includes significant plagiarized material Initially, nobody takes responsibility One-on-one conversations Students start throwing each other under the bus Case sent to SJS for investigation Investigation completed the following semester Two of the six found in violation
Question Will the end-of-semester evaluations for these instructors increase, decrease, or not be affected by this situation? Scenario A? Scenario B?
Why This Study? Interviewing stakeholders on campus as the faculty-in-residence Faculty were hesitant to report students Faculty want to avoid conflict/confrontations Belief that reporting students would impede professional advancement As someone who always reports, I was shocked! Original hypothesis Perception by faculty was a false perception and analysis would demonstrate no correlation
Introduction Examine the impact of sanctioning students for acts of academic misconduct on student evaluations of teaching (SETs). use of student evaluations for faculty advancement and promotions faculty may be conflicted between enforcing academic integrity and maximizing their professional advancement with high SET scores.
Method A multi-level modeling design with 8,940 end-of-semester student evaluations 32 faculty members from 17 academic departments was employed to determine if there was a whistleblower effect.
Literature Search Extensive qualitative articles Lack of quantitative analysis Only quantitative studies focused on grade inflation and higher SETs GPAs have been increasing while SAT scores have been decreasing
Genesis of Research Design Initial evaluation - faculty against peer instructors Compare faculty against all faculty within department Compare faculty against all faculty within college Final analysis – compare faculty against themselves
Creation of Sample Compare faculty against themselves Analyze evaluations for course in a semester when they reported cheating Analyze evaluations for same course in a semester when they did not report cheating Evaluation must be for the same class Looked for the closest semester in temporal proximity - summers excluded Must have reported 3 or more cases
Challenges to Methodology Highest reporting instructors reported cheating every semester therefore were not included Cases had to be submitted to SJS prior to 10 days before the end of semester Course Instructor Survey’s are given up to 10 days before the end of the semester
The Course Instructor Survey Five point scale ranging from “strongly disagree” to “strongly agree” Questions asked on survey The course was well organized The instructor communicated information effectively The instructor showed interest in the progress of the student The tests and assignments were usually graded and returned promptly The instructor made me feel free to ask questions, disagree, and express my ideas
Survey Overall rating for the instructor and course ranging from “very unsatisfactory” to “excellent” on a 5- point scale Overall, this instructor was Overall, this course was Key criteria for performance evaluation is the overall instructor rating
Limitations of Analysis Results based on data collected at one university Unable to link evaluations to students Unable to test highest reporting faculty
Results Statistically significant effect for reporting students (p<.001) Estimated mean rating for classes in which students were not reported was 3.95 Estimated mean rating for classes in which students were reported was 3.75 Hypothesis - instructors who report students tend to be more demanding of their students in general than those who do not report
Discussion Is it time for a policy review? Instructors expected to enforce AI Colleges want to instill ethics – beyond college By not enforcing, instructors are implicitly condoning cheating and undermining accurate student assessment Evaluations play a critical role for advancement for both tenure-track and non tenure-track faculty Do you harm yourself when you enforce AI?
Student Responsibility UT participates in the National Assessment of Student Conduct Adjudication Process (NASCAP) Project 21 higher education institutions participate Upon completion of adjudication, students complete survey When asked the question “What was the outcome of your case” 6.9% of UT students answered “my case was dismissed” In reality, only 0.15% of the cases were dismissed
Recommendations Encourage all faculty to communicate expectations of AI Provide regular communication to faculty regarding: How students cheat How to deal with cheaters How to confront students Best practices How to reduce cheating Require students to complete an online AI tutorial each year
Recommendations Encourage faculty to increase frequency of message Each academic department should have an AI point person Faculty should regularly update assignments and exams
Recommendations Include a question about faculty AI expectations and practices on evals AI should be part of new faculty orientation Administrators should include consideration for faculty who report cheating regarding merit review and promotions