1. The Writing Transfer Project Promoting Transfer through Reflection: A Cross-Institutional Study of Metacognition, Identity, and Rhetoric
Put names on here.
Dr. Dana Lynn Driscoll, Oakland University
Dr. Gwen Gorzelsky, Wayne State University
Dr. Carol Hayes, George Washington University
Dr. Ed Jones, Seton Hall University

2. Special Thanks
The ERS Seminar on Critical Transitions: Writing and the Question of Transfer
George Washington University for Year 1 funding for raters and coders
Oakland University, Seton Hall University, and Wayne State University for additional funds for research and travel
Conference on College Composition and Communication and the Spencer Foundation for Grant Funding for Year 2 data analysis
Our students and faculty involved in the study

3. Reflection as a Means to Understand Transfer and Metacognition: Pedagogy, Assessment, and Cross-Institutional Results
Basically, what I’m doing in this talk is presenting our framing research, methodology, and some of our initial cross-institutional results so that you can have an overall sense of the big picture of our study. The remaining talks will examine particular pieces of this dataset in more detail.
Dr. Dana Driscoll
Department of Writing and Rhetoric
Oakland University

4. Background: Transfer
Students substantially struggle in transferring writing skills from high school to college, throughout college courses, and from college to the workplace (Carson, Chase, & Gibson, 1993; Beaufort, 2007; Driscoll, 2011; Wardle, 2009; Beaufort, 2007).
New views of transfer (Driscoll & Wells, 2012; Lobato, 2003; Schwartz, Bransford, & Sears; Slomp; Wardle) stress the interaction between individual and context as key to investigating how learning transfers from original to subsequent contexts. Like traditional theories, these new views highlight the role of metacognition in promoting transfer.
Most of the transfer research we have is from single-institution studies and describes what is happening; the goal of our project is to approach transfer research from a cross-institutional, longitudinal perspective.
We all know that transfer of learning is difficult and challenging for students. New views of transfer stress the interaction between individual and context as key to investigating transfer. These views emphasize both individual factors (like dispositions, motivation, metacognition, and self-efficacy) and contextual factors (like pedagogy, curriculum and social systems). We are really interested in studying the spaces where these two kinds of factors meet in writing classrooms in diverse contexts in higher education.
In this multi-institutional study, which we are presenting today, we demonstrate how the use of student self-reflections in writing courses (including reflections that emphasize rhetorical awareness, writing process knowledge, transfer-focused thinking and habits of mind) can generate empirical evidence about both individual and contextual factors, including curricula and pedagogies. This kind of data can be useful for individual programs, and aid us in developing a more general understanding of learning to write and transfer.
Most of the transfer research we have is from single-institution studies and describes what is happening; the goal of our project is to approach transfer research from a cross-institutional, longitudinal perspective and see the influence of a curricular intervention.

5. Background: WAW, Metacognition, and Reflection
Metacognition (described as one of the eight “Habits of Mind”) is a student’s ability to recognize learning processes and to regulate their learning processes (Schraw and Dennison, 1994)
Scholars contend that students’ struggle concerning transfer results, in part, from the entrenched approach to teaching FYW which has traditionally lacked content knowledge about writing (Beaufort, 2007).
Some argue that FYW should emphasize metacognition and acquaint students with research and theory on writing by using the writing about writing (WAW) approach (Downs & Wardle, 2007).
Scholars like Yancey(1998) and Taczak (2011) advocate using reflection in composition pedagogy because it both fosters transfer and reveals writers’ metacognition.
Metacognition (described as one of the eight “Habits of Mind” in the framework for post-secondary writing instruction) is a student’s ability to recognize learning processes and to regulate those processes (Schraw and Dennison, 1994). Its critical for transfer of learning, and we are finding that it can be facilitated by reflective writing.
Reflective writing has been recognized as a powerful pedagogical tool; reflective writing has also been at the forefront of attempts to encourage transfer in writing-about-writing FYC classrooms (Taczak; Schon; Wardle). This presentation links these two frameworks by demonstrating how in our project, reflection is critical as a pedagogical strategy and as a means to research/assess/measure key areas (transfer, metacognition, declarative/procedural knowledge, identity, and habits of mind).
We see reflection as being particularly useful for both pedagogy and transfer research and we’ve used it heavily in our study. Reflection allows us to ask students to be metacognitively aware of their own learning processes, to connect that learning to other places, and to critically reflect on their successes and struggles as writers. These reflections, then, give us as researchers a window into the mind of the student and his or her processes.

6. Research Questions
Overarching research question: What is the effect of a WAW/highly rhetorical curriculum on transfer of learning, as explored in diverse writing course contexts?
This presentation addresses the following questions:
Are the courses in our study effective in producing better writers?
Are the courses in our study effective in encouraging metacognition and transfer in terms of:
transfer-focused thinking to the future?
transfer while students are enrolled in their initial course?
adaptation of prior knowledge to tasks while in the course?
How does transfer-focused thinking show up beyond the courses?
The four participating universities developed a shared methodology that collectively examined the role of metacognition and reflective writing in transfer of learning. The use of common reflective prompts (which you have on your handout) allowed for comparison among the four institutions; additional measures were also used. While the institutions differ in course levels and curricula as you’ll see in the next slide, we’ll examine what our shared methodology and data collection reveal about the role of reflective writing and metacognition in transfer of learning.
The following are some of the research questions we asked; I’ll be focusing on a subset of questions from our first year of data, which are listed on this slide.
*******************************

7. Methodology – Study Sites & Participants
Year 1 participants are from….
2 FYW sections at a mid-sized Northeastern Catholic University (Seton Hall University, New Jersey)
25 sections of FYW courses and 1 section of an upper-level course at a suburban mid-sized public Midwestern university (Oakland University, Michigan)
9 sections of a sophomore/junior-level writing course at a large urban public Midwestern urban university (Wayne State, Michigan)
14 sections of FYW courses at a large private Northeastern urban university (George Washington University, Washington D.C.)
Year 2 participants are a sampling of students from Year 1, and include….
7 students from the two FYW sections at SHU
15 students from the sophomore/junior-level sections at WSU
7 students from the upper-level writing course at OU
22 students from the FYW sections at GWU
This project emerged in the context of the Elon University Writing Knowledge Transfer Seminar in June Our four universities—public and private schools with varying student demographics and geographical locations—are collaborating to test the WAW and a similar rhetorical approach, as well as test specific reflective elements embedded to investigate whether it encourages transfer of learning. You can see what courses are included in our study on this slide. We just finished the initial analysis of our 2nd year of data collection at GWU only a week ago, so we are still very much in the process of understanding what our data means.
<material on handout; go through quickly>

8. Methodology – Data Collection
During Year 1, , we collected the following data:
pre- and post-semester writing samples to examine change over time
self-reflections from across the semester to understand the role of reflective writing in transfer and related areas
pre- and post-semester Writing Knowledge Transfer survey responses about student writing strategies, beliefs, perceptions, and dispositions to understand self-reported beliefs/attitudes related to transfer
During Year 2, , we collected the following data:
60 minute in-depth interviews on writing, transfer, and related areas
WID writing samples from each interviewee
Follow-up Writing Knowledge Transfer Survey responses
We collected three kinds of data in year 1. Today we’ll be specifically talking about the reflective writing and the writing sample results.

9. Methodology – Rating and Coding
Development of shared research methods, analysis methods, and rating/coding approaches
Met at ELON seminar for preparation and analysis work (2011, 2012, 2013); met at GWU for a coding week in August 2012 & June 2013
Raters/coders were trained graduate student teachers working in the WID program at GWU (24 in year 1; 31 in year 2)
They rated:
Student writing:
208 Year 1 writing samples (68 matched pairs)
66 Year 2 writing samples
Examined: Audience, Role of Sources, Use of Genre: Purpose and Development, Contextualizing; Genre-driven organization, style
Student reflective writing:
398 reflective pieces of writing with 98 possible codes; codes were applied 14,127 times
Coded 38 interviews in Year 2
Examined: Transfer-focused thinking, writing knowledge, metacognitive strategies, sense of identity as writers, writing-related dispositions
In summer 2012 and 2013, we trained a group of graduate students working in the WID program at GWU (24 from year 1; 31 from year two). Raters read and rated a sample of 208 Year 1 writing samples and 66 year 2 samples using a scoring rubric. We normed the raters to ensure consistency and calculated inter-rater reliability. Because we rated beginning- and end-of-term writing samples, the rubric measured growth in students’ ability to use writing studies’ conceptual and procedural knowledge in producing their texts. By defining rubric categories in terms of genre appropriateness, we achieved the flexibility needed to evaluate writing samples produced in varying institutional and classroom contexts.
In addition to rating the writing samples, we used a multi-level coding schema to code a random sample of 436 reflective pieces of writing. The coding schema includes categories for students’ descriptions of transfer-focused thinking, writing knowledge, metacognitive strategies, identity as writers, and dispositions. We normed during rating and coding to ensure consistency, checking on inter-coder reliability during the norming. This schema includes a total of 98 possible codes (including sub-codes), which were applied 14,127 different times. We normed the coders to ensure consistency, checking on inter-coder reliability during the norming.

10. Methodology – Data Analysis
Inferential and descriptive statistical analysis of the pre- and post- writing samples from year 1 (paired samples t-test, effect size, means)
Inter-rater reliability calculations were done to see how closely our raters and coders coded (on the fly during year 2, and post-rating during year 1)
Codes were analyzed both qualitatively and quantitatively
Quantitatively, we examined patterns in the number of codes, the kinds of assignments, and the kinds of students over time
Qualitatively, we examined the nature of the comments, how students were framing their experiences
We are in the process of this analysis work now, and expect to find much more as we continue to dig into the data (especially from year 2)
After rating and coding, we performed a number of analysis techniques using mixed methods approaches. These included inferential and descriptive statistics, inter-rater reliability calculations, and qualitative and quantitative analysis of our codes, as described on this slide. Our analysis work continues; what results we present in the next few slides are just the beginning of what we hope to learn from this dataset.

11. Year 1 Results: Writing Analysis
We were able to compare 68 sets of student papers from writing collected prior to the beginning of the semester to the end of the term (across four universities). Students showed significant improvement in writing while enrolled in our courses (p<0.001; effect size 0.385).
The first question we wanted to know from our Year 1 data was whether or not students improved overall in their writing. And we are pleased to say that yes, they did significantly improve based on comparisons between beginning and end of semester writing samples.

12. Year 1 Results: Writing Analysis
We can see from this second chart that students’ writing improved in all areas; the greatest gains were in the areas of genre awareness (which is a very common focus on transfer of learning and WAW based instruction); contextualizing, and their use of sources.
All of the results are significant at the p<0.02 level or above.

13. Key areas for a Metacognitive Understanding of Transfer
Emphasis in this talk is on transfer-focused metacognitive processes, which we call “transfer-focused thinking”
These include three areas (generated from our dataset):
Anticipating connections to future contexts
What (learning strategies, writing and rhetorical knowledge, content knowledge)
Where (educational, professional, personal, general connections)
Prior Knowledge (from previous course to current writing course)
Anticipating prior knowledge’s usefulness in present setting
Using/adapting prior knowledge and dealing with challenges
Negative transfer
Using Knowledge and Skills (from current course to other context)
Negative transfer / Challenges
For the next results, we are going to look at metacognitive understandings of transfer, specifically, what students know and also how they report applying and adapting knowledge to other contexts. This data comes out of our reflective writing dataset.
<walk through slide>

14. Transfer-focused Thinking: Anticipating Future Connections
Type of Connection (What, where)
Number of times coded in reflective writing in 398 documents (year 1)
Number of times coded in 36 interviews (year 2)
Connection to future educational experiences (where)
119 12
Connection to future personal life (where) 10 1
Connection to future professional life (where) 82 8
General connection to future (where) 94 4
Learning Strategies (what) 67 2
Non-writing content (what) 3
Unclear/unknown connection
Writing & Rhetorical knowledge (what)
212 13
Total Anticipating Connections Made
599 45
Through reflective writing, we were able to examine students’ initial beliefs in Year 1 of the study concerning transfer of learning, their self-reported attempts to transfer, and their transfer-focused thinking at the end of the semester. Students in our study made substantial connections beyond the course. Most these connections were made when we explicitly asked them at the end of the semester where and how they would use that knowledge in future circumstances. Of the the 436 reflections we read, we coded 599 instances of students anticipating connections to future contexts from Year 1 and found 45 instances in interviews.
I should also mention here that nearly all of these categories in year 1 in terms of codes increased as the semester progressed—that is, we can see differences in how many connections students are making as they progress through the course—as they get further and further, they demonstrate a clearer and clearer understanding of those connections. This is especially true of their reflective writing they were doing with each paper.

15. Transfer: Prior Knowledge
Type of Prior Knowledge
Number of times coded in 398 documents (Y1)
Number of times coded in 38 interviews (Y2)
Anticipating Prior Knowledge
108 6
Challenge with Prior Knowledge 68 37
Negative/Unsure About Prior knowledge 46 14
Using prior knowledge unsuccessfully 5
Using/Adapting Prior knowledge successfully 29 66
Prior knowledge was another area we looked at—we didn’t explicitly ask them about prior knowledge in the prompts in year 1, but we still saw that students were interacting with prior knowledge in our courses—successfully, unsuccessfully, and also anticipating where and how it might be used. In year 2, we see more successful adaptation of prior knowledge.

16. Transfer: Using knowledge/skills
Type of knowledge/skill use
Number of times coded in 398 documents (Y1)
Number of times coded in 38 interviews (Y2)
General use of knowledge/skills 31
122
Adapting knowledge 2 6
Connection to current educational life (where) 23 76
Connection to current personal life (where) 8
Connection to current professional life (where) 4 12
Learning Strategies (what) 5
Negative Transfer 1 7
Non-writing content (what)
Writing & Rhetorical knowledge (what) 10 98
During year 1, we didn’t expect to see much transfer itself while students were enrolled in the course (this is why we are doing longitudinal follow-ups with our students at various universities). We did see small amounts in year 1, but even with a much smaller set of participants in Year 2, we saw quite a bit of using of knowledge and skills in our interviews. While this is a very broad understanding of our data, one of the things we will do is dig into what these instances were and how students are transferring knowledge, especially writing and rhetorical knowledge, into diverse contexts. This shows us that opportunities exist, and students do report transfer.

17. Reflective Example
“At the beginning of the semester after being introduced to the framework of this course I was somewhat skeptical. The idea of being a pilot course for what sounds to be a study on our class to see how it can be beneficial to students in the future did not really seem too inviting….As time has gone on my perspective towards our English 3010 course has changed for the better. After being taught more in depth about the meaning of the ideas for this course such as “discourse community” and how it relates to our professional outlook I was able to grasp a better understanding for what the purpose of this course really is…For example, I feel as though being a novice in a particular field such as accounting, the idea of genre analysis and its function helps me to identify the different areas of writing within that field and its purpose and convention that I would not have been able to before.”
Here is just one example of the kinds of reflective writing we were seeing at the end of our courses—this one comes from Wayne State’s Intermediate Composition Course.
(Or take this out and put it on handout?)

18. Discussion - Results
To summarize, our initial analysis of data suggests the following:
Reflective writing prompts that scaffold in a WAW course can be useful in encouraging transfer-focused thinking, especially towards future activity
Students report a *lot* of transfer-focused thinking and anticipating use of prior knowledge in year 1, but we see less evidence in their reflections of actual use in year 1 (by that we mean descriptions of what they did and how they used it)
In year 2, however, students report transferring knowledge in their courses.
We also saw very positive results in students’ growth as writers in all of these courses
So overall, we’ve seen improvements in student writing in our courses and also a lot of transfer-focused thinking is happening thanks to the reflective prompts; its happening cross-institutionally and in really interesting ways.

19. Discussion – Reflective Prompts
Reflective writing prompts can be shared across very diverse courses to better compare and study student writing and thinking at different levels and institutions
Reflection can be used as a pedagogical, research, and assessment tool

20. Multi-Institutional Research
Need to go beyond single-site or single-class studies of transfer.
Multi-institutional research allows for generalizabilty/aggregability and understanding nuances in context.
A multi-level study has the potential to reveal which factors related to knowledge transfer are more important at different points in a student’s development.
In addition, we are developing a methodology for conducting multi-institutional writing transfer research that can be replicated and expanded by other scholars (as described in “Multi-Institutional Research and Generalizability: The Case for the Writing Transfer Project” currently in preparation)
Previous investigations of transfer and WAW are single-institution studies that focus on FYW—our study is the first multi-institutional, multi-level investigation of transfer in college-level writing instruction.
Our study allows us to produce more generalizable results that may suggest ways that students in different educational contexts may exhibit different degrees of knowledge transfer.
A multi-level study has the potential to reveal which factors related to knowledge transfer are more important at different points in a student’s development. What is contextually-based and what is generalizable across contexts.
In addition, we are developing a methodology for conducting multi-institutional writing transfer research that can be replicated and expanded by other scholars (as described in “Multi-Institutional Research and Generalizability: The Case for the Writing Transfer Project” currently under review)

21. References
Beaufort, A. (2007). College writing and beyond: A new framework for university writing instruction. . Utah: Utah State UP.
Carson, J. G., Chase, N. D., and Gibson, S. U. (1993). Academic demands of the undergraduate curriculum: What students need. Washington, DC: Fund for the Improvement of Postsecondary Education.
Downs, D., & Wardle, E. (2007). Teaching about writing, righting misconceptions: (Re)Envisioning “first-year composition" as “introduction to writing studies.” College Composition and Communication, 58(4),
Driscoll, D. L. (2011). Connected, disconnected, or uncertain: Student attitudes about future writing contexts and perceptions of transfer from first-year writing to the disciplines. Across the Disciplines, 8(2).
Driscoll, D. L., & Wells, J. (2012). Beyond knowledge and skills: Writing transfer and the role of student dispositions. Composition Forum, 26.
Lobato, J. (2003). How design experiments can inform a rethinking of transfer and vice versa. Educational Researcher, 32(1),
National Research Council (1999). How people learn: brain, mind, experience, and school. Washington D.C.: National Academy Press.
National Science Foundation. “Transfer of Learning: Issues and Research Agenda.” NSF Workshop Report, Web.
Schraw, G., & Dennison, R. S. (1994). Assessing metacognitive awareness. Contemporary Educational Psychology, 19,
Schwartz, D. L., Bransford, J. D., and Sears, D. (2005). Efficiency and innovation in transfer.” Transfer of Learning from a Modern Multidisiplinary Perspective. Ed. Jose P. Mestre. Information Age Publishing, 1-51.
Taczak, K. (2011). "Connecting the dotsaching about Writing, Righting Misconceptions: (Re)Envisioning "First-Year Composition" as "Introduction to : Does reflection foster transfer?” Ph.D., Florida State University, Tallahassee.
Wardle, E. (2009). "Mutt genres" and the goal of FYC: Can we help students write the genres of the university? College Composition and Communication, 60(4),