Presentation on theme: "A beginning understanding of the interplay between offline and online reading comprehension ability when adolescents read on the Internet Julie Coiro,"— Presentation transcript:
A beginning understanding of the interplay between offline and online reading comprehension ability when adolescents read on the Internet Julie Coiro, University of Rhode Island International Reading Association Research Conference (May 3, 2008)
Where are we headed? Introduction to the current situation Theoretical Frameworks Summarizing the increasing complexities that appear to characterize online reading comprehension Quantitative measures, procedures, and findings Qualitative measures, procedures, and findings What do these findings mean for literacy theory, research, and classroom practice? Time for discussion (15 minutes)
The current situation We do little to help struggling adolescent readers and their teachers (Biancarosa & Snow, 2004; Partnership for Reading, 2003; RAND Reading Study Group [RRSG], 2002). As a result, there is a large and increasing gap in adolescents reading achievement (NCES, 2001; 2002; 2003). At the same time, Internet technologies have increasingly become a part of our daily lives (e.g., Rainee & Hilton, 2005; Friedman, 2005), introducing additional challenges to reading (Coiro, 2003; Coiro & Dobler, 2007; NICHD, 2000; RRSG, 2002).
The current situation Unfortunately, we know little about… which skills/strategies/practices contribute the most to successful online reading comprehension; which students demonstrate these skills and which do not; how to characterize, measure, teach, and chart progress of these skills/strategies/practices in the classroom. Unbelievably, we are doing little to learn more…. No states include the reading comprehension demands of the Internet in their standards or in state reading assessments (Leu, Ataya, & Coiro, 2002).
Whats the big deal? Researchers urged to move beyond group comparison studies to focus more closely on the attributes of skilled and less- skilled online readers (Abrami & Bernard, 2006; Dillon & Greene, 2003). It is crucial that we learn more about the nature of online reading comprehension or risk even larger gaps in reading achievement and increasing numbers of readers who struggle in the networked information contexts that will dominate their future (e.g., ETS, 2003; IRA, 2001; Leu, 2007).
Purpose of the study This sequential mixed-methods study sought to: Investigate the extent to which new online reading proficiencies may be required to comprehend information on the Internet; Explore the nature of online reading comprehension ability among three adolescent readers who read online at different levels of proficiency.
Theoretical Frameworks Reading comprehension is an active, constructive process of making meaning (Pearson et al., 1992; Pressley et al., 1989; RRSG, 2002). New literacies perspective (Coiro et al., in press; Leu, Kinzer, Coiro, and Cammack, 2004) Adapting diachronic perspectives of literacy practices (Gutierrez & Stone, 2000) Developmental theory of reading (Alexander, 1997; 2003; 2005)
Developmental Perspectives Alexander (2006) argued, profiles of successful and struggling readers are reflective of developmental forces (p. 413). Stages of acclimation, competency, and proficiency may characterize readers over a lifetime Might these levels be useful for distinguishing strategy use among lower, average, and higher performing online readers over the course of a reading task as they read for information on the Internet?
Increasing Complexities of Text Structure Impacts Reading Comprehension Offline Information Text Comprehension Hypertext Comprehension Internet Text Comprehension (Online Reading Comprehension) Offline Narrative Text Comprehension
Offline Text Structures And Reading Comprehension A range of active, strategic processes are required to comprehend narrative text (Duke & Pearson, 2002; Paris et al., 1991; Pressley & Afflerbach, 1995) Information texts pose additional challenges with complex concepts, specialized vocabulary, and unfamiliar text structures (Guthrie & Mosenthal, 1987; Kintsch, 1990; Meyer, Brant, & Bluth, 1980) Reader characteristics play a central role in reading comprehension (RRSG, 2002; Guthrie; Alexander).
Key Reader Characteristics Reading comprehension ability Skilled readers apply a range of cognitive strategies such as determining key ideas, critically analyzing information, making inferences, etc. ( Pearson, Roehler, Dole, & Duffy, 1992; RRSG, 2002). Strategy use depends on the readers purposes and the types of texts they read (Duke & Pearson, 2002; Pressley, 2000)
Key Reader Characteristics Prior knowledge The knowledge a reader brings to any text or learning situation (e.g., Anderson & Pearson, 1984) Skilled readers draw on general, topic-, and text-specific knowledge to make predictions, reason strategically, and remember main ideas (e.g., Alexander, Kulikowich, & Schulze, 1991; Alexander, Shallert, & Hare, 1991).
Key Reader Characteristics Dispositions Habits of mind; tendencies to approach and/or respond to situations in certain ways (see Katz, 1988; Carr & Claxton, 2002) Distinct aspects of reading dispositions including beliefs, goals, values, purposes, and needs can influence comprehension –(e.g., Baker & Wigfield, 1999; Guthrie & Wigfield, 1997; Horner & Shewry, 2002; RRSG, 2002)
Added Complexities of Informational Hypertext Structures Hypertexts (closed systems) present multiple, non-linear pathways (Landow, 1994) Hypertexts extend the range of possible interconnections and pathways between texts (often, some paths are better than others) (Bolter, 1991; Burbules & Callister, 2000; Reinking, 1997) Content hidden beneath multiple layers and connected by links with fewer surrounding context cues (requiring high levels of inferencing) (Foltz, 1996; Spyrakis, 2000) Authors intertextual connections may be different than the readers connections, creating more complexity (Caney, 1999)
Reader Characteristics and Hypertext Comprehension Hypertexts…require skills and abilities beyond those required for comprehension of conventional, linear print (RRSG, 2002) Prior knowledge influences navigational patterns and comprehension recall e.g., Lawless & Kulikowich (1998); Lawless, Mills, & Brown, (2003); Yang (1997) Affective dispositions differ between more and less successful hypertext readers e.g., Balcytiene (1999); Yang (1997)
Added Complexities of Internet (Online) Text Structures Online texts are not bound within a closed system with only one organizational structure (Lawless & Schrader, 2007) Online texts change daily in structure, form, and content (Zakon, 2005) Online texts often contain hidden social, economic, and political agendas not typically found in closed hypertext learning systems (Cope & Kalantzis, 2000; Fabos, 2008) Online texts introduce infinite intertextual connections and intercultural negotiations (Snyder & Bulfin, 2008; Warshauer & Ware, 2008)
Online Text Comprehension Hill & Hannafin (1997) found metacognitive strategies, prior knowledge, and perceived self-efficacy influenced how adults learned from Internet text Studies suggest many adolescents are ill-equipped to deal with new online reading comprehension demands: Querying search engines e.g., Eagleton & Guinee (2002); Sutherland-Smith (2002) Understanding search results e.g., Coiro & Dobler (2004); Henry (2006); Leu et al. (2004) Critically evaluating online information e.g., Burbules & Callister (2000); Fabos (2004)
Online Text Comprehension Three studies have directly explored Internet use as a complex, multifaceted process of reading comprehension Coiro & Dobler (2007) found skilled readers employed both similar and more complex applications of (1) prior knowledge; (2) inferential reasoning strategies; and (3) self- regulated reading processes. Coiro & Dobler (2004) found traditionally skilled readers with Internet reading experience were aware of and demonstrated strategic reading processes to a higher degree than their less-skilled peers.
Internet Text Comprehension The New Literacies Research Team (2005) used this knowledge to design strategy interventions with seventh grade students and measure growth in online reading and science learning. Found higher achievement levels in online reading comprehension and conceptual science knowledge Developed a preliminary, psychometrically sound measure of online reading comprehension (ORCA-Blog) Performance in online reading skills (ORCA-Blog) did not correlate with traditional reading skills (DRP), suggesting that the two instruments measured complementary, but orthogonal skill sets.
Initial Evidence of something new (r=0.19, n = 89, N.S.) Leu, D. Castek, J., Hartman, D., Coiro, J., Henry, L., Kulikowich, J., Lyver, S. (2005). Offline Reading = CT State Reading Test Online Reading Comprehension= ORCA Blog
Internet Text Comprehension Some research suggests the skills sets are similar, but more complex (e.g., Coiro & Dobler, in press). Other research suggests the skill sets are complementary but orthogonal (e.g., NLRT, 2005). The current study builds on these findings to investigate the extent to which new (and old) comprehension proficiencies may be required on the Internet.
Key Research Questions RQ1: In a regression analysis, does performance on one measure of online reading comprehension ability significantly predict performance on a second, parallel measure of online reading comprehension ability over and above (a) offline reading comprehension ability and (b) prior knowledge? RQ2: What specific patterns of skill and strategy use appear to distinguish three students of varying levels of online reading comprehension ability as they engaged in a series of online information requests about science content?
Quantitative Sample 510 7th graders from a convenience sample sorted by strata into two groups (economically advantaged and economically challenged) 60 students randomly selected from each strata Final sample included 118 ethnically, economically, and academically diverse seventh graders
Measuring Prior Knowledge What do you know about… the lungs the breathing process oxygen carbon monoxide poisoning animation reliable information Topic-Specific Task-Specific Conceptual measure with six items scored with a three point rubric (r =.849) Range Mean = (SD = 7.12)
Measuring Online Reading Comprehension Ability Online Reading Comprehension Assessment (ORCA-Scenarios I and II) Three integrated tasks in a Treasure Hunt that asked students to locate, evaluate, synthesize, and communicate information using the Internet Twenty open-ended items; each scored using a 4-point rubric (0-3 points) for a maximum score of 60 points ORCA Scenario I (r =.918) and Scenario II (r =.909)
Quantitative Procedures 118 students completed measure of prior knowledge and ORCA-Scenario I (recorded with Camtasia) Standardized reading scores were collected from each district 16 weeks later, completed same measure of prior knowledge and ORCA-Scenario II (recorded with Camtasia) This 2nd PK measure was used in regression model
Findings: Predicting Online Reading Comprehension R 2 Offline Reading Comprehension Additional R 2 Prior Knowledge Additional R 2 Online Reading Comprehension Total R 2.351* 35.1% *.579* N = 118 Offline Reading Comprehension = CT State Reading Test Online Reading Comprehension = ORCA Scenario I and II (in Quia interface) Effect Size.33 - large effect Cohen (1988) Regression Analysis 7.4%15.4%57.9%
Something different, new, unique? Something predicts online reading comprehension over and above offline reading comprehension & prior knowledge Multiple possibilities (combinations) of offline and online reading ability Alternative explanation: Process-based strategy use vs. product-based skills (but its still reading comprehension)
Tested for Interaction Effects There was a significant negative interaction effect between prior knowledge and online reading comprehension (ORCA-Scenario I), t(105) = -2.28, p <.05. When entered into the whole regression model as a fourth predictor variable, this interaction effect explained a small but additional 1.9 percent of variance in the model, which was significant FD(1,104) = 4.782, p <.05 Findings: Predicting Online Reading Comprehension
Findings - Main Effects of Prior Knowledge Among readers with… Effect of prior knowledge Result HIGH online reading comprehension PK had no significant effect on ORCA2 performance t (106) =.079 p >.05 AVERAGE online reading comprehension PK had no significant effect on ORCA2 performance t (106) =1.965 p >.05 LOW online reading comprehension PK had a positive & significant effect on ORCA2 performance t (106) =2.898 p <.01**
A changing role for prior knowledge? PK had a positive and significant effect only among students with low online reading ability; for those with average or high online reading ability, PK had no significant effect on online reading performance This suggests…higher levels of online reading ability may help compensate for lower levels of topic-specific prior knowledge when adolescents complete online reading-for-information tasks.
Part II So what more can we learn from a richer qualitative look at the patterns of strategy use among three diverse online readers? How might these patterns begin to help us better understand the stories behind the numbers?
Qualitative Sample Three focal students selected based on a combination of their online and offline reading performance (after Time 1) A higher performing online reader and higher performing offline reader [Nicole] A lower performing online reader and lower performing offline reader [Christian] A higher performing online reader but lower performing offline reader [Mikaela]
Qualitative Procedures Directly after ORCA 2, conducted retrospective think-aloud interviews with three focal students Internet use at home & school Their personal response to the task General reading habits offline and online Elaborate on strategy use while video recording played back Semi-structured follow-up interviews with each students reading/language arts teacher and computer teacher Teachers perceptions and curriculum components
Qualitative Data Analysis Iterative stages of a diachronic, developmental, contrastive, case study design Case study of three different readers Diachronic analysis to identify phases of online reading behaviors along a chronological timeline Contrastive case analysis of strategy use Pinpoint developmental aspects of online reading comprehension along a continuum within six phases of online reading
Six phases of online reading (Reading for Information in ORCA-Scenarios I and II) Phase 1. Understanding the task directions Phase 2. Selecting and revising search terms Phase 3. Reading to evaluate the relevancy of hyperlinks in search engines and websites Phase 4. Reading to critically evaluate accuracy, reliability, and commercial bias within and across websites Phase 5. Synthesizing information across three websites to make a best choice Phase 6. Communicating answers to online information requests
ACTION THINK-ALOUD EMERGING PATTERNS Analyzing patterns within subphases of online reading
Finding #1 Qualitative Finding #1 A developmental progression of reading comprehension skills and strategies distinguished the three readers performance within each phase Lower-performing online reader Average-performing online reader Higher-performing online reader He was not sure what to pay attention to - he was confused so he just kept reading. He acknowledged this difficulty in the interview, but had a limited repertoire of strategies to address his confusion about where to start. She explained she was reading it slowly to make sure it made sense. She was reading all the questions and writing them down to help remember. She was aware of her difficulty, and used a compensatory strategy (write down clues). She was questioning the task and monitoring her understanding of the topic and the vocabulary. She mentioned the 2nd task was more challenging and wasnt quite sure which detail to focus on first. Phase 1: Understanding the task directions [monitoring understanding]
LowerAverageHigher Locating.com strategyWhole phraseUses keywords Evaluating Reliability Struggled to locate About Author page Judged reliability based on length of coverage Examined authors level of integrity/expertise Evaluating Commercial Bias Ran out of time and did not have opportunity to demonstrate strategy use Detected related advertising but over- generalized caution (believed all information was fake) Detected related advertising and expressed balanced caution (may be exxagerated/slanted) CommunicatingCopied the long address by hand Copy/pasted address with mouse shortcut At each phase, developmental differences appeared to characterize three students with different overall levels of online reading proficiency
Go to COIRO IRA2008 Part 3
Understanding Task Directions (Developmental Differences) High(50 seconds) I was reading the clues and I was like, Oh, United Kingdom! I like the United Kingdom! But then I got to the asthma part and I was like, what does asthmas have to do with carbon monoxide poisoning, and then I saw it. And then the animation part - I love having animation! Avg.(3 min. 30 sec) I read the clues and I read the questions, and then up here [pointing to the link to the MSN homepage], it goes to the link. I read the question and I wrote down (on the paper next to her) what the questions are, so I dont have to click back and forth later on to see if I got the right thing. Low(3 min 26 sec) Well, my plan is to like, to check it out, what to do, which one to look at, and where to look at it…I was thinking that I wouldnt be able to find it - that it was going to be hard.
LOCATING [ Nicole: Task 2] Higher Performing Online Reader 0:00-0:20 1:11. The address is 0:20-0:30 0:30-0:47
LOCATING [ Christian: Task 2] Lower Performing Online Reader 0:00-4:15 4:15-5:30 6:15-8:20 10:00-10:45 In questions 1-4, I just cant find it.
MOST RELIABLE: [Nicole - higher performing] This site is most reliable because it gives a lot of information. It is also done by a college institute. MOST RELIABLE: [Mikaela - average performing] This site is most reliable because the site is all about the facts on carbon monoxide poisoning and because you learn more about carbon monoxide poisoning. MOST RELIABLE: [Christian - lower performing] [ran out of time looking for authors name] I was all shocked cuz I was like kinda lost, cause its kinda hard Critically Evaluating Reliability (Developmental Differences)
LEAST RELIABLE: [Nicole - higher performing] This site looks like theyre trying to sell me stuff - I didnt think it was the least reliable, but I didnt think it was the most reliable - it was in the middle. Cause if they try to sell you something, that doesnt always mean its not reliable. It means that it gives you a slight thought that it might not be reliable, but it might just be trying to help you, like picture carbon monoxide detectors now and think the life lost may be yours next time. LEAST RELIABLE: [Mikaela - average performing] This site is the least reliable because 1) the site has the least facts and 2) because the site is the least accurate for people to learn anything because theres only 2 1/2 paragraphs. LEAST RELIABLE: [Christian - lower performing] This site said For Immediate Release - I think they just want to put that out really quick - it means they really dont want to help, they just put information out fast. Critically Evaluating Reliability (Developmental Differences)
Qualitative Finding #2 Developmental differences appeared to be affected by five key dimensions of offline and online reading comprehension ability including : (1) fluency; (2) self-regulated reading; (3) inferential reasoning; (4) critical reasoning; and (5) metacognitive knowledge about what, how, and when to employ particular online reading processes Remember: Strategic knowledge = declarative, procedural, and conditional See Figure 1 in Handout for Details OfflinePKOnlineTotal R %7.4%15.4%57.9%
The Interplay Between Offline and Online Reading Ability OfflinePKOnlineTotal R %7.4%15.4%57.9% Primarily offline Primarily online Primarily offline with new online procedures and contexts Combine offline and new online Primarily offline with new online procedures and contexts 1. Understanding task directions 2. Selecting & revising search terms 3. Evaluating relevancy of hyperlinks 6. Communicating 4. Evaluating accuracy, reliability and commercial bias 5. Synthesizing 35.1% 15.4%plus
Online reading comprehension might be conceived as overlapping and highly integrated dimensions of strategic offline and online reading processes rather than a linear sequence of isolated reading skills. Thus, online reading comprehension instruction should weave offline and online reading strategy practice within authentic tasks that integrate online locating, evaluating, synthesis, and communication. Qualitative Finding #3
Implications for Literacy Theory Preliminary evidence of the psychological reality of new literacies required to comprehend information on the Internet (Leu et al, 2004). Further informs long-debated theoretical questions about reading comprehension by providing preliminary evidence that online reading comprehension may best be characterized as both: a highly integrated set of strategic reading processes (see Thorndike, 1974) that can be organized into sets of different sub-skills (see Davis, 1972) related to six phases of online reading
Implications for Literacy Research Findings contribute to emerging work that seeks to identify the range of skills and strategies that effectively characterize online reading comprehension (TICA Project, ). Findings help us begin to more precisely understand the possible similarities and complexities / differences between offline and online reading comprehension processes revealed in previous work (e.g., Coiro & Dobler, 2007; Leu et al, 2005). Findings extend work identifying qualitative shifts in strategic offline processing that distinguish novices from experts (e.g., Alexander et al.) to offer preliminary patterns of qualitative shifts in online comprehension strategy use that appeared to distinguish higher-performing online readers from lower-performing online readers.
Implications for Classroom Practice Inform diagnosis and remediation within each phase of online reading comprehension ability We can no longer assume that one reader will do equally well or be equally challenged across all dimensions of an online reading task (or that how you read offline predicts entirely how you read online) Preliminary set of criteria for determining individual strengths and weaknesses can inform realistic next steps for remediation within and across the phases of online reading comprehension
Implications for Classroom Practice Inform instruction of online reading comprehension Design leveled (Mosenthal, 1996) and explicit instruction in: Locating & communicating skills to enable access to online texts Critical evaluation skills for all levels of readers to facilitate deeper questioning of texts and sources Synthesis strategies that consider the overlaps between evaluation, synthesis, and communication skills
We need to be cautious in our interpretations… Tasks did not represent the full extent of online literacy experiences or new literacy components Focused on externally assigned questions as opposed to self-selected topics of inquiry Narrow view of evaluation, synthesis, and communication Scoring system reflected the task demands Cant really generalize from three case studies that in some ways were more similar than different (but findings can inform a replication study with larger N)
Significance of the Study Having the skills to comprehend information on the Internet will play a central role in academic success in an information age. A better understanding of the nature of the new skills & strategies of online reading comprehension can: Inform an emerging theory of new literacies Provide rich insights into online reading profiles Inform the development of diagnostic instruments and better instructional strategies, particularly for those students who struggle the most with online reading Pursuing the ideas that emerged may prompt new focus for addressing several of the literacy challenges we face today.
Thank you. Questions? Concerns? Ideas? Julie Coiro University of Rhode Island