Where Have We Been? Where Are We Going? Using Student Surveys to Assess and Improve Literature Courses Kelly Douglass, PhD Asst. Professor, English Riverside Community College

Riverside Community College District  Three campus district  First day enrollment for Fall 2007 district wide: 29,371 (  49% of students taking an English class report educational goal of BA (with/without AA)

English Discipline  Common curriculum across district  39 full time faculty members; 136 part- time faculty members  Discipline offers 44 courses; 19 are literature courses  13 literature courses offered in Spring 2006 (across district)

Why a student survey?  Intimidation about collaborative assessment of different courses  Most other course offerings had been or were being formally assessed by Spring 2006  Body of research indicating reliability

Factors for validity and reliability of student self-reporting 1.The information requested is known to the participants 2.The questions are phrased clearly and unambiguously 3.The questions refer to recent activities 4.Participants think the questions merit a serious and thoughtful response 5.The questions do not threaten, embarrass, or violate the privacy of the participate or encourage the participant to respond in socially desirable ways Qtd. in William E. Knight et. al. “Student Self-Reported Gains Attributed to College Attendance: Comparing Two- Year and Four-Year Institutions” Original Literature Review in G. D. Kuh’s “Using Student and Alumni Surveys for Accountability in Higher Education” in J. C. Burke (Ed.), The Many Faces of Accountability (2005) and The National Survey of Student Engagement: Conceptual Framework and Overview of Psychometric Properties (2001)

Student Learning Outcomes for Literature Courses 1.Recognize and appreciate some of the distinctive features of the major writers, literary works, movements, trends, and genres in ________ literature from _____ to _____. 2.Explain how this literature both reflects and shapes the literary history of ______ with an awareness of how differences of culture, gender, and other social markers may shape a writer’s interactions with this history. 3.Relate literary works from different ______ authors and time periods to one another and synthesize ideas that connect them into a tradition. 4.Employ college-level methods of literary analysis to reading and interpreting _____ literature and literature in general. 5.Demonstrate critical thinking and writing skills through the process of constructing responses to, interpretations of, and arguments about literature.

Pre/Post Test General Questions 1.How many English classes have you taken in college? (include composition courses like 60AB, 50, 1AB) 1.What interests/interested you about this class? 1.What do you most want to get out of the class? / What is the most important thing you gained from this class?

Survey Questions and Corresponding SLOS 4. How confident are you about your writing skills? (SLO 5) 5. How confident are you about your ability to analyze literature (SLO 4) 6. How would you rate your knowledge of the following: a.) Major writers and literary works of period and culture, b.) Literary movements and trends of period and culture c.) Literary genres (SLO 1)

Survey Questions and Corresponding SLOs 7. When you read, are you able to relate what you read to outside circumstances of the period about which you’re reading such as history, culture, and/or politics? (SLO 2) 8. When you read, are you able to relate what you read to your own personal circumstances or philosophies? (SLO 5)

What interests/ed you about this class? Req. for transfer / major / program / career / grad units; Eng. major Subject area interest Interest in Prof. Gen. reading pleasure % scale Desire to see lit. in new ways; analyti- cal skills Wtg. skills ScheduleNo reply Other

What do you most want to get out of this class? (PRE) What is the most important thing you have gained from this class? (POST) % scale A good grade / units for transfer / applies to career More knowledge of / exposure to subject area Increased ability to analyze / understand lit. More writing practice / skills Pleasure / Enjoyment No reply Other

Data for Questions 4 - 8

Question 8 Findings: A Poorly Written Question Measures a skill that is not directly taught Does not effectively represent SLO; narrow definition of skill through personalization

Question 4: How confident are you about your writing skills? “Significant insignificance” in survey results Writing not primary focus of literature courses Students more aware of benefit/growth in content knowledge component than writing component The class made them aware of writing skills as an area for improvement or interest

Responses to Study: Teaching Practices More tools to help students “develop interpretive independence” Reconsideration of writing assignments, particularly more that are better aligned with SLOs In response to student desire for more traditional academic content: More occasional “straight lectures,” particularly on theoretical structures A few more canonical authors Depends very much on what is already being done in the classroom; reactions were very subjective

Responses to Study: SLOs SLO 5: “Demonstrate critical thinking and writing skills through the process of constructing responses to, interpretations of, and arguments about literature.” Subpoint 3: “Support premises with effectively integrated, relevant, thoughtful, and sufficient evidence drawn (as appropriate) from literary texts and the writer’s prior experience and knowledge. Subpoint 5: “Think creatively within and beyond literary studies, making some connections between the literary work, and the intellectual and cultural forces that shape individuals’ lives.”

Responses to Study: Assessment and Future Projects Instructor responses to assessment: Interesting information, but some difficulty translating into classroom change Helpful for refining exactly the kind of assessment to do in individual classrooms Encouragement to create a writing based writing-SLO related assessment plan for the literature classes

Responses to Study: Assessment and Future Projects Assessment Plan: Writing based General prompt for all literature courses Will measure portions of two SLOs: Employ college-level methods of literary analysis to reading and interpreting _____ literature and literature in general. Demonstrate critical thinking and writing skills through the process of constructing responses to, interpretations of, and arguments about literature.