Maestro: A Computer Tutor for Writers Kurt Rowley, Ph.D. Maestro Principal Investigator 'They [students] seem to be less distressed about pre-writing, drafting & editing...'

Introduction A Computer Tutor for Writers (CTW) titled 'Maestro' was developed by Air Force Research Laboratory (AFRL) as a technology transfer project  Advanced cognitive training technologies  Best available information about writing composition  Best available research methods Goal of improving confidence in education "..we can never have meaningful progress in educational programs until we can have the same level of confidence in them that physicians can have in procedures or medications passed by the FDA." (Robert Slavin)

Background Effectiveness of computer-based tools in improving student writing  Zellermayer and colleagues tested a computerized 'writing partner' (1991): They determined that students can internalize guidance and produce improved writing products with use of facilitative writing software.  Vygotsky (1978) and Salomon (1988) suggested the notion of 'procedural facilitation' in which visualizations of procedural skills are used as support structures for students.

Background Effectiveness of computer-based tools in improving student writing (continued)  AFRL background studying 'intelligent tutoring systems' using supportive learning environments  Patricia Carlson, English Professor from Rose Hulman Institute of Technology (Indiana) with colleagues at AFRL designed a supportive writing environment 'R-WISE' following the 'procedural facilitation' strategy (1994, 1996, 1998)  R-WISE provided a rich background for the development of the Maestro CTW

Background

Teacher Inputs Many lessons learned from R-WISE, inputs collected from 25 R-WISE English teachers  Address student learning styles  Motivational strategies (R-WISE included multiple individual writing exercises, teachers wanted students to write entire papers in the tutor)  Incorporate teacher-generated assignments and State-required writing standards  Address entire writing process as a single skill  Use a single, consistent user interface

Cognitive Model Design of a CTW required a writing model  Overall instructional goal selected for CTW was to address teacher concerns and build an instructional system that could teach the writing process as a singular skill  Flower & Hayes cognitive model of writing process selected as primary guide  Added Scardemalia & Bereiter notion of 'knowledge transforming' practiced by expert writers vs. 'knowledge telling' by novices

Cognitive Model WRITING TASK ENVIRONMENT Writing assignments (prompts, topics, audience) The writer uses workspaces to help with text production WRITER’S KNOWLEDGE Goals Ideas Writing Plan (organize ideas) Drafting (translate ideas into paragraphs) Revision Publish (review and edit) Monitor Writing Goal Achievement Adapted from: Hayes & Flower, 1980 Prewrite ComposeEdit WRITING PROCESS The design of the Maestro CTW was based on cognitive studies of the Expert Writing Process TIMs & Advice (memory, topic research, audience, writing plans)

CTW Design

Designed to Address Individual Student Needs Individualized guidance for progressive development of cognitive writing process skills  Tailored Instructional Modules accommodate student learning styles and interests, based on automated survey  Advice guides student through the writing process making suggestions and summarizing writing process skills (solicited and unsolicited)  Workspaces provide procedural facilitation, helping students to systematically utilize an expert-level writing process Students motivated to learn writing process  Teachers select realistic, multimedia prompts  Publish to real audience (print-outs, used for assignments)

75 Instructional Modules

400+ Advice Statements

22 Workspaces (note-taking example)

22 Workspaces (Revision Example)

CTW Study  Full school-year study with students and teachers using CTW to learn writing process and to complete writing assignments.  Pre-test / post-test, contrasted groups design  Combined N=471, control group N=174  Treatment groups:  1 (2+ sessions, 2-6 hours using CTW), N=99  2 (4+ sessions, 6-11 hours using CTW), N=163  3 (6+ sessions, 11+ hours using CTW), N=36

CTW Study  ANOVA showed differences between the groups were significant F [(3, 471) = 3.595, p<.05)]  Posthoc Tukey's Honestly Significant Difference Test showed that the mean score differences were significant between group 3 (6+ sessions) and the control group, and groups 3 and 1, but not between groups 2 and 3  The Pre/Posttest result of primary interest was a 11% score gain for group 3

CTW Study

Feedback from Teachers  Students more motivated to write, little lab management trouble, even at low SES schools  Students produce more coherent essays  Teachers like the individualized pacing  Teachers like the fact that the software is consistent with the writing processes they are teaching

Conclusions  The Maestro CTW appeared to produce positive effects, although statistical power is weak due to low N for the full treatment (group 3)  The study suggests that writing process skills can be taught / reinforced through procedural facilitation using computer software  Further research with specific populations is merited, particularly with groups that need help managing the writing process.

Publications Rowley, K. & Meyer, N. (2003). The effect of a computer tutor for writers on student writing achievement. Journal of Educational Computing Research, 29(2), Steuck, K., Rowley, K. & Kretschmer, M. (2000). Partnering to implement computer-based tutoring systems in secondary schools. Journal of Interactive Instruction Development, 12(1), Rowley, K., Carlson, P., & Miller, T. (1998). A cognitive technology to teach composition skills: Four studies with the R-WISE writing tutor. Journal of Educational Computing Research, 18(3),