Multi-Section Assessment of One Course: English 407A Ed Nagelhout Julie Staggers Department of English

Presentation Goals Describe steps for assessing multiple sections of a single course Use English 407A: Business Writing as an extended example Offer recommendations for other multi- section courses Answer Questions

Eight Steps 1.Think Programmatically 2.Select Outcomes 3.Develop Tools 4.Simplify Evaluation Measurements 5.Construct Report Template 6.Examine Findings 7.Improve Teacher Support Structure 8.Review/Evaluate Assessment

1. Think Programmatically Every student should have basically the same experience in every (multi-section) course regardless of teacher or platform. Coherence Consistency Project-Centered Pedagogical Emphasis

English 407A: Business Writing Sections per year Taught primarily by PTIs, GAs, and Adjuncts Business Writing Course Delivery Platforms: Traditional Face-to-Face sections Hybrid sections Distance Ed sections

English 407A: Business Writing To Meet Programmatic Goals Common Syllabus Common Outcomes Common Projects Mentoring Program Teacher Support Teacher Choice

2. Select Outcomes How much can we legitimately do in 16 weeks? Determine what’s important  What do you want them to know?  What do you want them to do? Promote primary and secondary objectives Be explicit Maintain focus

English 407A: Business Writing Common Outcomes Write to multiple audiences, for various purposes Plan and manage short- and long-term writing projects Draft, design, revise, and edit documents Design and implement appropriate research strategies Write collaboratively (e.g., co-authoring, peer revising) Follow and adjust to business writing conventions Design documents for both content and visual appeal Use writing to help prepare and deliver oral presentations Write ethically and responsibly

3. Develop Tools Tools should be designed to determine the achievement of outcomes as effectively and as efficiently as possible Think programmatically Align explicitly with outcomes Use a range of tools

English 407A: Business Writing Quizzes Exercises Introductory Project (2) Case Project (4) Employment Project Major Collaborative Project (3) NOTE: Each project includes multiple deliverables and a project assessment memo

English 407A: Business Writing 1.Primary Criteria 2.Project-Specific Criteria Format Audience Access Information Rhetorical Awareness Organization Argument Business Style Depth of Thought Detail Visual Grammar and Correctness

4. Simplify Evaluation Measurements Evaluation measurements provide the data to help you determine the success of the course Take cut/paste mentality Allow for multiple lenses Provide different ways into the data

English 407A: Business Writing Common Outcomes Write to multiple audiences, for various purposes Plan and manage short- and long-term writing projects Draft, design, revise, and edit documents Design and implement appropriate research strategies Write collaboratively (e.g., co-authoring, peer revising) Follow and adjust to business writing conventions Design documents for both content and visual appeal Use writing to help prepare and deliver oral presentations Write ethically and responsibly

English 407A: Business Writing 1.Primary Criteria 2.Project-Specific Criteria Format Audience Access Information Rhetorical Awareness Organization Argument Business Style Depth of Thought Detail Visual Grammar and Correctness

English 407A: Business Writing

5. Construct Report Template Reporting the data in a way that will allow you to improve your course is paramount, so you need to be proactive How do you report the data? Where do you report the data? Why do you report the data? Create template for application over time

English 407A: Business Writing English 407A Annual Report Department Queries College Queries University Queries

6. Examine Findings Examine findings through the lenses you find most applicable Student Learning Teacher Effectiveness Teacher Support Program Effectiveness

English 407A: Business Writing Preliminary Findings Analyze by Outcome Analyze by Criteria for a Single Project and across Projects Analyze by Platform Analyze by Section and across Sections

English 407A: Business Writing Sample Findings Pilot Study – Spring 2008 – 7 Sections Outcome: “Design documents for both content and visual appeal” Criteria: Format: All projects; All sections Overall Average = 4.23 Low Average = 3.98 (Case Project) High Average = 4.39 (Employment Project)

7. Improve Teacher Support Structure Strong teacher support structures are imperative to successful course delivery and assessment Teacher Effectiveness Teacher Efficiency Teacher Workshops Teacher Materials

English 407A: Business Writing 10-Hour per Week Commitment If four courses is considered full-time, then no teacher should spend more than 10 hours per week per course. This includes time in class, time in office hours, time preparing for class, time responding / evaluating student writing, and time in training. As a program, our primary objective is to help teachers meet this expectation.

English 407A: Business Writing Teacher Workshops Classroom Activities Responding Effectively AND Efficiently to Student Writing (Egg Timer Workshop) Norming Sessions Focus Groups Staff Meetings

English 407A: Business Writing Teacher Materials Course Materials  Syllabus, Schedule, Evaluation Sheets Handouts  Project, Pedagogical, Process/Product Classroom Materials  PPT Slides, Samples, Activities, Mini-Lectures

8. Review/Evaluate Assessment Why are we doing assessment? Are we achieving our goals? Are the outcomes appropriate? Are we improving our course? Are we improving our teaching? Are we looking at the right things? What can we do differently? What can we do better?

English 407A: Business Writing Program Recommendations Revise Project Assessment Memos Incorporate Focus Groups Update/Revise WebCampus Review/Revise Outcomes Review/Revise Criteria Improve Methods  Assessment and Pedagogical

Conclusions/Recommendations Design to fit your needs Go slowly Make incremental progress Think long-term Imagine reduced teacher workload and improved student learning Questions?