Measuring Knowledge Acquired From Information Text John T. Guthrie Marcia H. Davis

Purposes Measure knowledge gained from information text. Measure reading strategies for information text comprehension. Construct a computer-based predictor of knowledge gained from information text. Measure motivation for reading comprehension.

Tasks for Measuring Information Text Comprehension Background knowledge elicitation – writing Questioning – browsing and writing Searching in multiple texts with notetaking Summarizing in writing

Writing Sample

Notes: 1. Stating a concept refers to a clear reference to one of the ecological concepts describing an organism’s interaction with its environment. Concepts consist of: feeding, locomotion, competition, predation, reproduction, respiration, communication, defense, and adaptation to environment.

Rubric for Coding Student Text-based Writing Level 1: Minimal statement of very few characteristics of a biome or an organism. There are no ecological concepts, or definitions. This may consist of no information beyond the student’s name as identifying information. 1-4 organisms correctly classified to a biome; OR  1-2 factual characteristics about either one of the biomes, but no definition of a biome; OR  1-9 organisms correctly identified, but not classified into any biome; OR d) Student’s name but no information; less than a-c.

Level 2: Students identify characteristics of one or more biomes, or they present several organisms correctly classified to a biome. There are no full definitions of biomes accompanied classifications of organisms or their adaptations to the biome. The information is minimal, factual, and may appear as a list. Information is largely accurate. 5-up correctly classified organisms; OR 3-6 characteristics, that are not definitional presented for the two biomes combine; OR

Level 3: Students present one or more ecological concept with minimal supporting information and correct classifications of organisms to biomes. A higher level principle, which may entail multiple concepts, may be presented with no rationale or supporting information about biomes or organisms. Also included here may be a well-formed, fully elaborated definition of both biomes accompanied by a substantial number of organisms accurately classified into the biomes.  1 weakly stated ecological concept AND 2-10 organisms correctly classified to biomes but no relation to concept; OR 2-4 concepts briefly stated in a disorganized, incoherent structure or list with no support and biomes are not identified or described; OR Strong biome definition; AND 3-or more correctly classified organisms; OR Clearly stated principle linking 2 or more concepts but no supporting information.

Level 4: Students display conceptual understanding of organisms and their survival mechanisms in one or more biomes. These are represented by specific organisms and their physical characteristics and behavioral patterns that enable them to exhibit the concept as a part of their survival. They may include higher level principles, such as food chains or interactions among ecological concepts, with very limited supporting information. 1-4 coherently stated ecological concepts with minimal supporting information linking the organism information to the biome. [Coherent statement of concepts contains references to specific organisms and an aspect of the environment or other organisms it is interacting with] OR 1 coherent concept with supporting information; AND 4-10 correct classifications of organisms to their biome; OR 1-2 higher level principle or food chain (linking multiple ecological concepts) with vague and limited supporting information about the organisms; OR

Level 5: Students show command of ecological concepts, and relationships among different organism and various biomes. They describe organism, their structural characteristics and their behaviors. The interaction of the organism and the environment is central to the statement. 2-4 ecological concepts with specific, supporting information linking the organism mentioned to their biomes; may also have 2-3 relevant facts about one or both biomes; OR 1-3 ecological concepts with specific, extensive coherent supporting information about these concepts and the adaptations of a few (1-3) organisms to the biome; OR c) Weak or partially incorrectly stated food chain (or higher order principle) with additional concepts and 6-10 classifications.

Level 6: Students describe complex relationships among multiple organisms and their habitats. These may appear as food chains in one or two biomes or as energy exchange in the living environment. Students support the principles with examples from diverse organisms. High level principles that depict interdependencies among organisms in specific habitats are emphasized. Food chain or food web, which refer to one biome, or both biomes separately or both biomes simultaneously or energy chain; AND correct classifications of 6-20 organisms to biomes; OR  Food chain or food web AND detailed, accurate account of physical characteristics or adaptive behavioral patterns of a few organisms; OR High level principle that shows relationship of two or more ecological concepts (e.g., competition and reproduction). Supporting evidence about the organism and their relationship to each other and/or the biomes descriptions are substantial and explicit.

Strategy Predictors of Reading Comprehension Gates MTC PK Search Question MacGinitie - Multiple Text Comprehension .35** Prior Knowledge .47** .40** .14 .22** .26** Questioning .28** .20** .15* .06

Computer-based Task of Passage Comprehension Survival Move Protect Fat Den Fur Paddle Swim Webbed

Reading Task 500 words students read Computer-based rating 9 key words paired Rating 1 (not related); 5 (some what related); 9 (highly related)

Output of PathFinder Path correlation = -.3 , +1.0 Path coherence = -.3, 1.0 Correlations of PathFinder to multiple text comprehension = (John will give correlations to place here)

Strategy Predictors of Reading Comprehension Gates MTC PK Search Question Coherence MacGinitie - Multiple Text Comprehension .35** Prior Knowledge .47** .40** .14 .22** .26** Questioning .28** .20** .15* .06 Path .17* .12 -.02 -.05 .03 Correlation .42** .01 .10 .33**

Motivation for Reading Comprehension: Motivation for Reading Questionnaire (MRQ) Student self-report 18 items (Internal motivation) Reliability .85 - .9 Internal motivation (curiosity, involvement, preference for challenge, self-efficacy) External motivation (competition, recognition, grades)

Motivation for Reading Questionnaire

Reading Engagement Index (Teacher’s Rating) Behavioral, Cognitive, Motivational, Social Engagement in Reading Seven items: 1-4 scale Reliability: Exceeding .90

Reading Engagement Index

Correlation of Student Self-report Questionnaire to Reading Comprehension Gates MacGinitie Multiple Text Comprehension MRQ Self Report REI Teacher Report - .35** .23** .12 .39** .37** .31**

Conclusions Multiple text comprehension shows content validity for gaining knowledge from information text Multiple text comprehension correlated with Gates MacGinitie Cognitive strategies predict multiple text comprehension

Conclusions Computer based text comprehension (pathfinder) correlates with multiple text comprehension It is less well predicted by cognitive strategies than multiple text comprehension

Conclusions Motivation measured by teacher ratings correlates higher with comprehension than self-report. Motivation predicts complex comprehension tasks higher than simpler comprehension tasks. Motivation predicts reading comprehension strategies.