1. The Process The Results The Repository of Assessment Documents (ROAD) Project Sample Characteristics (“All” refers to all students enrolled in ENGL 1551) Spring 2011 Fall 2011 ROAD sampleAll VariableNMeanMean NMeanMean Age16722.7122.8 19322.6321.6 High school GPA1533.113.10 1783.272.88 YSU GPA1673.142.79 1932.782.61 Hours Completed16753.2341.1 19344.8941.3 ACT English Score13721.7221.1 14617.5618.2 Spring 2011 Fall 2011 ROAD sampleAll EthnicityFreq.PercentPercent Freq.PercentPercent Other74.43.7 73.95.45 Black148.812.4 2916.219.65 White13886.878.9 14379.967.44 GenderFreq.PercentPercent Freq.PercentPercent Female975854 985152 Male704246 954948 Logistic Regression Results P Values for Statistically Significant Variables by Item, Fall and Spring 2011 Rubric Item 1234567 Age(-)0.006(-)0.007 Age Squared(+)0.01(+)0.007 White(+)0.07(-)0.06 Male ACT English(+)0.001(+)<.0001(+)0.01(+)<.0001(+)<.0001(+)<.0001(+)<.0001 The signs of the coefficients are shown in parentheses Frequency Distributions and Means of Rubric Scores, Fall and Spring 2011 Score Rubric Item01234Mean Context of and Purpose for Writing 121131537011 1.87 Content Development 61551355410 1.74 Writing Conventions 12157139439 1.67 Sources and Evidence 151441365113 1.73 Control of Syntax and Mechanics 7821847710 2.00 Student's position (perspective, thesis/hypothesis) 11174121447 1.63 Conclusions 6136152587 1.79 All students taking ENGL 1551 (Writing 2) are asked to upload their final assignment to the repository. Starting Spring 2012 students in upper-division courses will also upload writing samples. Approximately 400 writing samples were randomly selected from the documents uploaded in the spring and fall of 2011. Using a seven-item rubric developed by YSU faculty, instructors hired and trained through the YSU Writing Center score the writing samples. Five of the items are used to evaluate the quality of the writing, two of the items evaluate whether the writing sample demonstrates critical thinking. Since the scores are stored in Banner, it is possible to analyze the relationship between the scores and different student characteristics. With the exception of ethnicity, the characteristics of the ROAD samples are very similar to the characteristics of all students enrolled in ENGL 1551. Students who take ENGL 1551 in the spring tend to have slightly higher grade point averages and higher scores on the English section of the ACT exam than the students who took the course in the fall. The graph on the left is a histogram for the rubric items students scored lowest and the one in which they did the best. The graph on the right shows the relationship between the rubric scores for the writing samples and the students’ scores on the English section of the ACT exam. For all seven rubric items, logistic regression analysis found a statistically significant relationship between the rubric scores and student scores on the English section of the ACT exam. A score of 0 meant there was no evidence of the characteristic associated with that rubric item. For example, a score of 0 would be given for the “Sources and Evidence” item if no source or evidence was used to support the statements made in the writing sample. A score of 4 meant that the writing sample represented exceptional senior-level work.