Beth Perrell Arri Stone SPC 1608 Assessment for 2017/2018

College-wide Assessment Work Fall 2015: SPC 1608 Established a Lib Guide with Resources about Source Credibility and Bias Evaluated the Persuasive Speech Spring 2016 Fall 2017 Spring 2018

Last Year’s Plan Related Accomplishments We conducted a flashlight survey to define components of audience analysis indicators We asked faculty to show the citation video We asked faculty to assess students’ verbal citations We asked faculty to assess audience analysis We are creating the script and plan for assessing audience analysis intervention We are reporting the data

Q1 - What primary factors do you listen for when assessing audience analysis?

Q1 - What primary factors do you listen for when assessing audience analysis? # Answer % Count 1 Connect examples to the specific audience 31.58% 12 2 Connect definitions to the specific audience 7.89% 3 Connect testimony to the specific audience 15.79% 6 4 Connect statistics to the specific audience 21.05% 8 5 Have an inclusive common ground statement in their introduction like, “As college students…” 18.42% 7 Q1 - What primary factors do you listen for when assessing audience analysis? # Answer % Count 6 Other _______________ 5.26% 2 Total 100% 38

Conclusions from 2015/2016: Credible Source Representation and Oral Citations Create videos with student examples of correct and incorrect oral citations Add to the LibGuide Use in class

College-wide Assessment Work Fall 2016: SPC 1608 Surveyed 1608 instructors about their requirements for oral footnotes. Created script to reflect the preferred style and solicited feedback. Held auditions. Had OIT record and edit video.

College-wide Assessment Work Spring 2017: SPC 1608 Video was posted to YouTube for ease of access. Faculty had students watch the video. Faculty evaluated verbal citations for EITHER informative or persuasive speeches. Used same question from 2015/2016

College-wide Assessment Work Spring 2018: SPC 1608 Faculty had students watch the video. Faculty evaluated verbal citations for EITHER informative or persuasive speeches. Used same question from 2015/2016 Faculty evaluated audience analysis for EITHER informative or persuasive speeches, to create a baseline for comparison in 2018/2019.

Results

Results from 2015/2016

Results for Spring 2017 – Overall Data Speaker provides oral references to the credible sources (e.g., author/authority, title, date, website, data base, journal, book, etc.) exposing bias of the source as necessary. All Students Response % Accomplished 1505 50.33% Attempted 697 23.31% Not Attempted 316 10.57% Not Assessed 124 4.15% No Longer in Course 348 11.64% Total 2990 100.00%

Results for Spring 2018 – Overall Data Speaker provides oral references to the credible sources (e.g., author/authority, title, date, website, data base, journal, book, etc.) exposing bias of the source as necessary. All Students Response % Accomplished 1501 49.1% Attempted 722 23.6% Not Attempted 275 9.0% Not Assessed 241 7.9% No Longer in Course 317 10.4 Total 3056 100.00%

Comparison: Three Years 17/18 16/17 15/16 17/18 16/17 15/16 17/18 16/17 15/16 17/18 16/17 15/16 17/18 16/17 15/16

Anecdotal Outcomes: SPC 1608 Student Comments What did you hear?

Discussion from LOL/Dean’s Meeting Re: At-Risk Population Benefits across disciplines Overall 1608 success rates

SPC1608

Observations from Nichole African American males and online students were higher on not assessed (on the informative speech especially). The percentage of African American males not submitting the speech makes up for about half of the gap in their scores (so some are not submitting and others are not succeeding), this same pattern is true of the online students (not submitting).

Comparison: 2016, 2017 (FOCUS ON NOT ATTEMPTED CHART…PERSISTENCE Comparison: 2016, 2017 (FOCUS ON NOT ATTEMPTED CHART…PERSISTENCE?) – FINAL Thoughts about Data)

Not Assessed % African American Male 13 Caucasian Male 7 Hispanic Male African American Female 12 Caucasian Female 8 Hispanic Female Online 17 Onsite 5

Results –African-American Males 2017 (verbal Citation) Persuasive Speeches Response % Accomplished 68 34.87% Attempted 52 26.67% Not Attempted 29 14.87% Not Assessed 9 4.62% No Longer in Course 37 18.97% Total 195 100.00%

Results –African-American Males 2018 (verbal Citation) Persuasive Speeches Response % Accomplished 68 33.0% Attempted 52 28.1% Not Attempted 29 15.6% Not Assessed 9 12.9% No Longer in Course 37 10.3% Total 195 100.00%

Not assessed… is there a way to impact whether or not students complete the speech? Other thoughts? Should we add an exploration of this as a baseline for next year?

Next Steps Produce audience analysis video during Summer 2018 Pilot during Fall (Ask for Volunteers – 1 faculty member from each campus) Show video Use “optional” activity

Next Steps Choose new Learning Outcome Leaders (If wanted, Beth and Arri will remain as Outcome Leaders)

Library Resources: Audience Analysis Karli Mair from East Campus

Spring 2018-- Information Literacy (Source citation): The speaker provides oral references to the credible sources (e.g., author/authority, title, date, website, data base, journal, book, etc.) exposing bias of the source as necessary.

NEW Spring 2018-- Oral Communication (audience analysis): The speaker "framed" their content relative to their audience analysis (i.e. using common experiential examples, relevant statistics, polling data and/or language choices) to demonstrate their adaptation of their message to the intended audience.

Total students assessed: 3,064 Spring 2018 2,990 Spring 2017   Response rate: 91% Spring 2018 (3,379 expected) 91% Spring 2017 (3,300 expected)

Retention in the Course by Race and Gender 2016-2017

Race of Respondents (Change Data from Spring 17 and 18) # Answer % Count 1 African American 15.61% 467 2 Asian 4.15% 124 3 Caucasian 25.21% 754 4 Hawaiian 0.23% 7 5 Hispanic 37.98% 1136 6 Indian 0.17% Multi-Race 2.77% 83 8 Unknown 13.87% 415 Total 100% 2991

Student Race (Add Spring 18 line)

Results – Informative Speech Data Informative Speeches Response % Accomplished 605 54.80% Attempted 311 28.17% Not Attempted 129 11.68% Not Assessed 54 4.89% No Longer in Course 5 0.45% Total 1104 100.00% Min Value Max Value Average Value Variance Standard Deviation Total Responses Total Respondents 1 5 1.68 0.80 0.89 1104

Results – Persuasive Speech Data Persuasive Speeches Response % Accomplished 900 59.60% Attempted 386 25.56% Not Attempted 186 12.32% Not Assessed 38 2.52% No Longer in Course 0.00% Total 1510 100.00% Min Value Max Value Average Value Variance Standard Deviation Total Responses Total Respondents 1 4 1.58 0.64 0.80 1510

Comparison: 2016, 2017 Separated

Results –First College Semester Persuasive Speeches Response % Accomplished 139 47.28% Attempted 61 20.75% Not Attempted 28 9.52% Not Assessed 21 7.14% No Longer in Course 45 15.31% Total 294 100.00% Min Value Max Value Average Value Variance Standard Deviation Total Responses Total Respondents 1 5 2.22 2.19 148 294

Comparison: Overall, At-Risk

Comparison: All Data