1. Course Transformation:
Using an in-class activity to improve student writing
Klaus Libertus
January 2018

2. Goal of the Course Transformation
Improve student writing in a w-course
Decrease instruction on content
Increase hands-on practice opportunities
Introduce online lectures
Allow students to review materials
Study at own pace
Offer more writing opportunities
Writing has to be practiced
More instructor feedback

3. Targeted Course Research Methods Lab Lab component to Research Methods
RM is required for Psychology Major
RM Lab is 40% of overall course grade
Lab meets for 3h and 20 min/week
Enrollment 150 – 240 students
9-10 sections with 24 students each

4. Goals of RM Lab Provide hands-on activities related to: Learn:
Experimental design
Data collection
Data analysis
Learn:
Scientific writing
Use APA formatting
Research literature

5. Significant amount of time should focus on teaching writing
Writing in RM Lab
RM Lab is a designated w course
“W-Courses are designed to teach writing within a discipline through writing assignments that are distributed across the entire term. Students should produce at least pages of written work; a significant portion of this work should be substantially revised in response to instructor feedback and class discussion.”
Significant amount of time should focus on teaching writing

6. But there is one problem…
THERE’S NO WRITING IN
RM LAB
At least not:
During class
With instructor feedback

7. What happens in RM Lab? Prior to transformation:
Experimentation content
Demos
Lecture on basics
Design of own Experiment
Writing lectures
APA lecture
Content lectures
Review lectures
Two statistic reviews
Too much “lab” time is lecture.
Lectures, especially review of stats, are not effective:
Majority of writing mistakes are related to APA format and Statistics
How much time do these activities take up?

8. Use of class time prior transformation
According to student perceptions (Spring 2016)
30% on writing? But what kind of writing activities do students mean?

9. Writing inside/outside RM Lab
Writing lectures
Basic “what goes where”
Some in-class activities that invite brief drafts of paper sections
Not graded, not collected
Usually not completed by all
Homework assignments
Two papers
6 “busy work” assignments
Simplified
Idealized
Unconnected
Writing in class and at home:
Did not form a clear sequence
Was not related to actual scientific writing
Little feedback on homework, no feedback on in-class writing

10. Course Transformation Approach
Reduce lecture time
Eliminate in-class review
Create online review materials
Provide immediate feedback with online mini-quizzes
Allows students to review once or multiple times, as needed
Replace assignments with in-class writing
Writing sequence based on published research
Early writing as low stakes assignments
Put meaningful writing INTO class time

11. Transformation
Initiated during Fall 2016

12. Course Transformation
Fall 2016 class was divided into two syllabi
Old
4 sections (about 24 students each)
Revised
5 sections (about 24 students each)
Most instructors were randomly assigned
Two instructors came from Anthropology and randomly split (one in old and one in revised)
Klaus Libertus was assigned Revised syllabus
To facilitate content creation

13. Old vs. Revised Syllabus
Week
Topic 1
Introduction, data collection 2
APA Style Review 3
Stats Review 4
Library tutorial 5
Paper 1 analysis and outline 6
Paper 1 peer review 7
Stats Review 2 8
Paper 1 due 9
Study 2 Design 10
Data Collection 11
Data analysis 12
Paper 2 draft 13
Writing clinic 14
Paper 2 final
Week
Topic 1
Introduction, data collection 2
APA homework, In-class writing 1 3
Stats homework, In-class writing 2 4
Library tutorial 5
Paper 1 data analysis and outline 6
Paper 1 peer review 7
Stats homework 2, ICW 3 8
Paper 1 due, 9
Study 2 Design 10
Data collection 11
Data analysis 12
Paper 2 draft 13
Writing clinic 14
Paper 2 final
50% of sections used traditional syllabus
3 review lectures
50% of sections used revised syllabus
3 in-class writing activities
IDENTICAL SECOND HALF OF CLASS

14. Review lectures  Online videos
Old syllabus retained lectures from previous semesters
Revised syllabus replaced APA formatting and Statistics lectures
Online lectures with same content
Provided to students via MyPitt Video (Panopto)
Lecture content was identical in both cases
For initial transformation only
Spring 2018 class now uses additional videos from external sources (YouTube, Kahn Academy,…)

15. In-class writing sequence
Revised syllabus included 3 in-class writing activities
Time used for review lectures in old syllabus
Sequence is based on published research article
Open data article
Students will “reverse-engineer” parts of the article
Compare their own writing to published article in the end
Goal: Engaging with real, concrete research increases student interest in the material

17. In-class writing sequence
The In-class writing activity was:
Open materials
Open instructor
ASK ME ANYTHING
With outline/notes provided that students could COPY FROM
No need to worry about Plagiarism
Rationale:
Fully scaffolded writing
Experience writing process without need to first do background research
Low stakes
12.5% of grade in initial transformation for all 3 assignments
Two 5% assignments, one 2.5% assignment
Stakes have been lowered for Spring 2018

18. Note on Assignment article
Selected article:
Rogers, T., & Feller, A. (2016). Discouraged by Peer Excellence: Exposure to Exemplary Peer Performance Causes Quitting. Psychological Science, 27(3), doi: /
Article was NOT selected for its content
Study 2 uses an online survey and conducts t-test and Chi-square tests
Matches what is taught in Statistics review
Data is open and available for download
Class does include a peer review segment
Implications of article for peer review were discussed
Main message: “Don’t get discouraged”
Not a main objective of this assignment

19. Evaluation of Transformation

20. Two evaluations of transformation
Within semester
Old vs. revised syllabus
Fall 2016 class
Different Instructors
Different TFs
Several grades
By instructor themselves
Included surveys assessing student perceptions
Total of 9 sections
5 revised, 4 old
Between semesters
Old vs. revised syllabus
Spring 2016 vs Spring 2017
Same instructor
Klaus Libertus
Rating of Paper 2 only
By “blind” grader
NOT instructor
One section only

21. A) Within semester Measurable course goals Performance Attitudes
Overall Lab grades
Lab grades on Paper 1
Lab grades on Paper 2
Attitudes
3 online surveys (Pre, Mid, Post)
Towards Psychology in general
Experience of Lab
Feeling prepared for writing
Class time distribution

22. 1) Performance Overall lab grades Grades on Paper assignments
Paper 1 after revision
Same topic for all sections
Paper 2
Different topic in each section
Note: Paper 2 is written in pairs

23. 1) Performance No differences in overall grades
No differences in Paper grades

24. 1) Performance No differences between original and revised syllabus
Not too surprising
Different instructors
Better approach
Grade papers of same instructor from different years
Have naïve grader
Will be used in evaluation approach B

25. 2) Attitudes Towards Psychology 10 questions
Rated on 7 point Likert scale
Strongly disagree – Strongly agree
5 positive, 5 negative items
Derived one composite score

26. Results: Attitude towards Psychology
Ratings were slightly higher in revised class
Not statistically significant
Most likely ceiling effect – everybody loves Psychology

27. 2) Attitudes Lab Experience 5 questions Same 7 point Likert scale
3 positive, 2 negative items
Derived one composite score

28. Results: Lab Experience* *
Ratings significantly higher for Revised syllabus at Mid and Post
Overall lab experience has improved with revised syllabus

29. 2) Attitudes Preparation for writing
Given before submission of Paper 1 and Paper 2
4 questions, all positive, composite score

30. Results: Preparation for writing† *
Students felt more prepared for writing using revised syllabus
Effect reduced for POST (p = .078)

31. 2) Attitudes Distribution of class time
Student perception on how much time was devoted on different aspects of class

32. Results: Distribution of class time
Students felt that more time was spent on writing
Strong increase, 10% more time
Same pattern when comparted to Spring 2016

33. B) Between semesters Measurable course goals Performance
Lab grades on Paper 2
Rated by blind coder
Grading dimensions
Scientific writing style (appropriate for Psychology)
Logic of argumentation (esp. Intro and Discussion)
Clarity of writing
Overall quality of the paper

34. Continuous ratings

35. Results: Paper 2 performance
Statistically, no significant differences between groups (note: small N, only 32 papers). Patterns suggest improvement.

36. Summary Comparison of old and revised syllabus reveals:
Increase in instruction on writing
Improvement to lab experience for students
Stronger feeling of preparedness
Improvements in writing clarity and quality

37. Continue to use revised syllabus Reduce stakes for in-class writing
Improve quality of online lectures/videos
Increased use of outside sources

38. Thank you for your attention and feedback
Special thanks to Beth Matway who helped develop the writing sequence
Special thanks to Carol Washburn for helping to design the revised syllabus
Special thanks to Barb Kucinski and Ben Rottman for their comments and suggestions on
this transformation
Thank you for your attention and feedback