Developing a Reading Strategy ITS: Competing Constraints from Theory, Technology, Pedagogy, and Experiments Danielle S. McNamara, Irwin Levinstein, Chutima Boonthum, and Srinivasa Pillarisetti University of Memphis Psychology / Institute for Intelligent Systems Funded by the IES Reading Program and the NSF IERI Program

iSTART Investigators Co-PIs and Senior Researchers: Irwin Levinstein (ODU), Keith Millis (NIU), Joe Magliano (NIU), Grant Sinclair, Katja Wiemer-Hastings (NIU), Max Louwerse, Art Graesser Postdocs & Staff: Cedrick Bellissens, Rachel Best, Chutima Boonthum, Zhiqiang Cai, David Dufty, Joyce Kim, Chris Kurby, Phil McCarthy, Tenaha O’Reilly, Yasuhiro Ozuru, Margie Petrowski, Srinivasa Pillarisetti, Roger Taylor Many students at Memphis, ODU, and NIU

Interactive Strategy Training for Active Reading and Thinking Currently provides self-explanation reading strategy training that combines training to self-explain text and training to use active reading strategies is adaptive engages the trainee in an interactive learning environment using animated agents Goal is to provide via the internet a variety of empirically supported interventions to improve strategies for reading and thinking McNamara, Levinstein, & Boonthum, 2004

Introduction Module –Teacher-Agent and 2 Student Agents discuss reading strategies Demonstration Module –Genie self-explains a text –Merlin provides feedback –Trainee is asked to identify strategies Practice Module –Trainee types self-explanations to science text –Merlin guides the trainee and provides feedback

Based on SERT Self-Explanation Reading Training Training to self-explain text using reading strategies (e.g., paraphrasing, bridging, elaborating) Self Explanation: say aloud or type an explanation McNamara, 2004

History : Funded by McDonnell and ODU –Develop and test SERT : Funded by NSF IERI –Test SERT in high-school classrooms –Develop iSTART : Funded by IES Reading –Increase adaptivity - add texts, modules, student model –Develop teacher interface : Developed iSTART v : Developed iSTART v : Developed iSTART v3.0 Currently developing Teacher Interface

Overarching Goals and Considerations Follow Original SERT Script as closely as possible But, take advantage of computer environment –Facilitates individualized interaction –Enables more fine-tuned feedback –Increases time for practice –Escapes (some) social dynamics of classroom Anticipated older computers in high schools Anticipated recursive development and frequent revisions

Architecture (e.g., software on server) Programs – nonproprietary: Java, MySQL Agents vs. Text Pedagogical Agents vs. Real People Synthesized Voices vs. Human Voices Full bodied vs. Talking Heads Cartoon-like vs. Human-like Develop Agents vs. Use Microsoft Agents Initial Decision Making ( )

Initial Version

Version 2.0 Changes Presentation order of the five strategies Mini-demonstration Dialogue scripts (e.g., more examples, short dialogues) Multiple-choice quizzes revised Synthesized voices improved Revised interface

Changed background colors to reduce distraction (probably not one of our best moves)

Added box labels

Changed boxes from yellow on blue, to black on white

Boxes are uniform size – they don’t grow

Developed pause and repeat buttons. Scroll Bars replaced buttons.

Boxes, characters, text, and speech bubbles no longer overlap

Motivation for Changes Human Factors Observations during testing Data and theory

Theory vs. Pedagogy vs. Data Come up with more ideas than we can test But, have to avoid the kitchen sink –Can’t make every modification you think of Progress is made by relating ideas to theory But, testing them remains complicated And, not all ideas turn out to be good ones Testing the revisions –We ain’t in Experimental Psychology Land anymore –So, hard to know if each revision ‘works’ Time constraints

Two Examples Data indicated that there was a problem Theory pointed to solutions Testing told the tale

Demonstration Section Students asked to do a wide range of tasks –Identify and locate strategies –Locate text that is the source of strategies –Point, click, highlight College students did fine –High School students – not so much Revised to increase scaffolding and reduce WM load Data indicates that changes were effective –Students increase in levels

Increase Paraphrasing Hypothesized that less skilled students needed more practice at basic skills –Developed Paraphrasing Practice Module Conducted Experiment –Students received dedicated practice in paraphrasing (without self explanation) Predicted that less skilled students would benefit from more paraphrasing practice Au Contraire – the more skilled students benefited from the extra practice and the less skilled students benefited more from the version without it

Can you test it? The New Demonstration included a host of changes –scaffolding, reduce number of choices, visually chunk self-explanation, etc. The Paraphrase Module consisted of a single modification (per se)

Considerations The fun factor –must be inherently interesting or challenging, but not too much The boredom factor –Can’t be too repetitious or too long The embarrassment factor –e.g., 'Genie to the rescue' failed because other students saw rescued students singled out Avoiding distractions –developed a means for students to adjust volume pitch and speed of voice, but never used it because they would play Theory vs. Intuition (stay the path)

Thanks! For more information and papers