Presentation on theme: "1 Contextual Vocabulary Acquisition: From Algorithm to Curriculum Michael W. Kibby Department of Learning & Instruction and The Reading Center William."— Presentation transcript:
1 Contextual Vocabulary Acquisition: From Algorithm to Curriculum Michael W. Kibby Department of Learning & Instruction and The Reading Center William J. Rapaport Department of Computer Science & Engineering Department of Philosophy, and Center for Cognitive Science Karen M. Wieland Department of Learning & Instruction, The Reading Center, and The Nichols School NSF ROLE Grant REC
4 Why Learning Word Meanings Is Important
5 Why Learning Word Meanings Is Important Reason 1 National Assessment of Educational Progress- Reading (NAEP-Reading)
6 Meaning Vocabulary Assessment on NAEP-R Meaning vocabulary is the application of one’s understanding of word meanings to passage comprehension.
7 Vocabulary knowledge is considered to be one of the five essential components of reading as defined by the No Child Left Behind legislation.
8 NAEP will not test definitions in isolation from surrounding text; i.e., –students will not be assessed on their prior knowledge of definitions of words on a list.
10 Examples: Altruistic Magnanimously Dispersed Impetus Forage Soothing Lost in thought Huddled Abide Piqued Beholden Marathon journey Legacy Abated Social contract Grudge
11 Three Reasons NAEP-R Does Not Test a Specific Word List 1.Knowledge of the explicit definition of a word is not what is required for reading comprehension.
12 2.The meaning of a word is too often dependent upon the context. e.g. cast The fisherman cast his line. The members of the cast took a bow. They put a cast on my broken arm. The yard is littered with shells cast off by the cicadas.
13 3.Writers often use words in a manner that goes beyond their concrete, familiar definition, but do so in ways that skilled readers can interpret effectively.
14 Why Learning Word Meanings Is Important Reason 2 Learning new things and their words changes or increases our perception and organization of the world
15 The Lego™ Notion of Learning New Things
16 Why Learning Word Meanings Is Important Reason 3 Reading comprehension mandates knowing the meaning (concept, thing) associated with words in the text. When students do not know meanings of words in a written text, comprehension often disrupted.
18 Why Learning Word Meanings Is Important Reason 4 Learning new things and words facilitates students’ abilities to use words judiciously— which is much valued in our society
21 Why Learning Word Meanings Is Important Reason 5 The Profound Effects of Limited Vocabulary
22 Profound effects of limited vocabulary continued Limited vocabulary is associated with lower IQ scores. Limited vocabulary is associated with limited reading comprehension. –In grades 7+, vocabulary and reading comprehension correlate.75 to.85.
23 Social Class and Meaning Vocabulary Hart, Betty, & Risley, Todd R. (1995). Meaningful differences in the everyday experience of young children. Baltimore, MD: Brookes.
24 Studied 42 children’s vocabulary growth from their 9th month to their 36th month. Researchers –Visited each child’s home once a month. –Observed and tape recorded for one hour every word spoken to or by child. –Recorded hours for every child.
25 Actual and Estimated Number of Words Heard from Months
26 “The Invisible Curriculum”
27 Cumulative Number of New Words (Hart & Risley, 1995)
29 From Algorithm to Curriculum
30 A Brief Background on the Counting of Words Carroll, Davies, Richman (1971), The American Heritage Word Frequency Book–called the WFI. A count of 5,088,721 running words (called tokens) in printed English for grades 3-9.
31 Of These 5,088,721 Words There were 86,741 different words. But the following 13 were counted as different words: addadditiveadditives addsadditionadditions addedaddendaddends addingadditionalADDITION as well as Add (capitalized).
32 Nagy & Anderson (1984) Sampled WFI words. Put each word in 1 of 6 classes varying in semantic relation to other words. –Classes 0, 1, 2 closely related semantically. –Classes 3, 4, 5 progressively more distant. Estimated there are 139,020 different words in semantic categories 0, 1, & 2.
33 But 45,453 of these are base words— knowing these 45,453 means a reader knows all 139,020. Adding 43,080 in classes 3, 4 & 5 brings total to 88,583 different word families in printed school texts for grades 3-9.
34 Learning New Words is Natural
36 Edna Heidbretter, The Attainment of Concepts taught persons to associate nine pairs of visual shapes and pronounceable pseudo word told persons this was a memory task
41 pran mulf relk I SETS Trials to Learn 27...
42 pran mulf relk III SETS Trials to Learn
43 pran mulf relk IIIIII SETS Trials to Learn
44 pran mulf relk IIIIIIIV SETS Trials to Learn
45 pran mulf relk IIIIIIIVV SETS Trials to Learn
46 pran mulf relk IIIIIIIVV SETS Trials to Learn
47 Definition of “CVA” “ C ontextual V ocabulary A cquisition” = def the acquisition of word meanings from text –“incidental” –“deliberate” by reasoning about –contextual clues –background knowledge (linguistic, factual, commonsense) Including hypotheses from prior encounters (if any) with the word without external sources of help –No dictionaries –No people
48 CVA: From Algorithm to Curriculum 1.Computational theory of CVA –Based on: algorithms developed by Karen Ehrlich (1995) verbal protocols (case studies) –Implemented in a semantic-network-based knowledge-representation & reasoning system SNePS (Stuart C. Shapiro & colleagues) 2.Educational curriculum to teach CVA –Based on our algorithms & protocols –To improve vocabulary & reading comprehension –Joint work with Michael Kibby & Karen Wieland Center for Literacy & Reading Instruction
49 Project Goals Develop & implement computational theory of CVA based on verbal protocols (case studies) Translate algorithms into a curriculum –To improve CVA and reading comprehension in science, technology, engineering, math (“STEM”) Use new case studies, based on the curriculum, to improve both algorithms & curriculum
50 People Do “Incidental” CVA We know more words than explicitly taught –Average high-school grad knows ~45K words learned ~2.5K words/year (over 18 yrs.) –But only taught ~400/school-year ~ 4800 in 12 years of school (~ 10% of total) Most word meanings learned from context − including oral & perceptual contexts –“incidentally” (unconsciously) How?
51 People Also Do “Deliberate” CVA You’re reading; You understand everything you read, until… You come across a new word Not in dictionary No one to ask So, you try to “figure out” its meaning from “context” How? –guess? derive? infer? deduce? educe? construct? predict? … –our answer: Compute it from inferential search of “context”, including background knowledge
52 What does ‘brachet’ mean?
53 (From Malory’s Morte D’Arthur [page # in brackets]) 1. There came a white hart running into the hall with a white brachet next to him, and thirty couples of black hounds came running after them.  2.As the hart went by the sideboard, the white brachet bit him.  3.The knight arose, took up the brachet and rode away with the brachet.  4.A lady came in and cried aloud to King Arthur, “Sire, the brachet is mine”.  10.There was the white brachet which bayed at him fast.  18. The hart lay dead; a brachet was biting on his throat, and other hounds came behind. 
54 Computational cognitive theory of how to learn word meanings From context –I.e., text + grammatical info + reader’s prior knowledge With no external sources (human, on-line) –Unavailable, incomplete, or misleading Domain-independent –But more prior domain-knowledge yields better definitions “definition” = hypothesis about word’s meaning –Revisable each time word is seen
55 Cassie learns what “brachet” means: Background info about:harts, animals, King Arthur, etc. No info about:brachets Input:formal-language (SNePS) version of simplified English A hart runs into King Arthur’s hall. In the story, B12 is a hart. In the story, B13 is a hall. In the story, B13 is King Arthur’s. In the story, B12 runs into B13. A white brachet is next to the hart. In the story, B14 is a brachet. In the story, B14 has the property “white”. Therefore, brachets are physical objects. (deduced while reading; Cassie believes that only physical objects have color)
56 --> (defineNoun "brachet") Definition of brachet: Class Inclusions: phys obj, Possible Properties: white, Possibly Similar Items: animal, mammal, deer, horse, pony, dog, I.e., a brachet is a physical object that can be white and that might be like an animal, mammal, deer, horse, pony, or dog
57 A hart runs into King Arthur’s hall. A white brachet is next to the hart. The brachet bites the hart’s buttock. --> (defineNoun "brachet") Definition of brachet: Class Inclusions: animal, Possible Actions: bite buttock, Possible Properties: white, Possibly Similar Items: mammal, pony,
58 A hart runs into King Arthur’s hall. A white brachet is next to the hart. The brachet bites the hart’s buttock. The knight picks up the brachet. The knight carries the brachet. --> (defineNoun "brachet") Definition of brachet: Class Inclusions: animal, Possible Actions: bite buttock, Possible Properties: small, white, Possibly Similar Items: mammal, pony,
59 A hart runs into King Arthur’s hall. A white brachet is next to the hart. The brachet bites the hart’s buttock. The knight picks up the brachet. The knight carries the brachet. The lady says that she wants the brachet. --> (defineNoun "brachet") Definition of brachet: Class Inclusions: animal, Possible Actions: bite buttock, Possible Properties: valuable, small, white, Possibly Similar Items: mammal, pony,
60 A hart runs into King Arthur’s hall. A white brachet is next to the hart. The brachet bites the hart’s buttock. The knight picks up the brachet. The knight carries the brachet. The lady says that she wants the brachet. The brachet bays at Sir Tor. [background knowledge: only hunting dogs bay] --> (defineNoun "brachet") Definition of brachet: Class Inclusions: hound, dog, Possible Actions: bite buttock, bay, hunt, Possible Properties: valuable, small, white, I.e. A brachet is a hound (a kind of dog) that can bite, bay, and hunt, and that may be valuable, small, and white.
61 General Comments System’s behavior human protocols System’s definition OED’s definition: = A brachet is “a kind of hound which hunts by scent”
62 Computational cognitive theory of how to learn word meanings from context (cont.) 3 kinds of vocabulary acquisition: –Construct new definition of unknown word What does ‘brachet’ mean? –Fully revise definition of misunderstood word Does “smiting” entail killing? –Expand definition of word used in new sense Can you “dress” (i.e., clothe) a spear? Initial hypothesis; Revision(s) upon further encounter(s); Converges to stable, dictionary-like definition; Subject to revision
63 Motivations & Applications Part of cognitive-science projects –Narrative text understanding –Syntactic semantics (contra Searle’s Chinese-Room Argument) Computational applications: –Information extraction –Autonomous intelligent agents: There can be no complete lexicon Agent/info-extraction system shouldn’t have to stop to ask questions Other applications: –L1 & L2 acquisition research –Computational lexicography ** Education: improve reading comprehension **
64 State of the Art Vocabulary Learning: –Some dubious contributions: Useless “algorithms” Contexts that include definition –Useful contribution: (good) reader’s word-model = updateable frame with slots & defaults Psychology: –Cues to look for (= slots for frame): Space, time, value, properties, functions, causes, classes, synonyms, antonyms –Can understand a word w/o having a definition Computational Linguistics: –Systems need scripts, human informants, ontologies Not needed in our system –CVA Word-Sense Disambiguation Essay question vs. multiple-choice test
65 State of the Art: Computational Linguistics Information extraction systems Autonomous intelligent agents There can be no complete lexicon Such systems/agents shouldn’t have to stop to ask questions
66 State of the Art: Computational Linguistics Granger 1977: “Foul-Up” –Based on Schank’s theory of “scripts” (schema theory) –Our system not restricted to scripts Zernik 1987: self-extending phrasal lexicon –Uses human informant –Ours system is really “self-extending” Hastings 1994: “Camille” –Maps unknown word to known concept in ontology –Our system can learn new concepts Word-Sense Disambiguation: –Given ambiguous word & list of all meanings, determine the “correct” meaning Multiple-choice test –Our system: given new word, compute its meaning Essay question
67 State of the Art: Vocabulary Learning (I) Elshout-Mohr/van Daalen-Kapteijns 1981,1987: –Application of Winston’s AI “arch” learning theory –(Good) reader’s model of new word = frame Attribute slots, default values Revision by updating slots & values –Poor readers update by replacing entire frames –But EM & vDK used: Made-up words Carefully constructed contexts –Presented in a specific order
68 Elshout-Mohr & van Daalen-Kapteijns Experiments with neologisms in 5 artificial contexts When you are used to a view it is depressing when you live in a room with kolpers. –Superordinate information At home he had to work by artificial light because of those kolpers. During a heat wave, people want kolpers, so sun-blind sales increase. –Contexts showing 2 differences from the superordinate I was afraid the room might have kolpers, but plenty of sunlight came into it. This house has kolpers all summer until the leaves fall out. –Contexts showing 2 counterexamples due to the 2 differences
69 State of the Art: Psychology Johnson-Laird 1987: –Word understanding definition –Definitions aren’t stored –“During the Renaissance, Bernini cast a bronze of a mastiff eating truffles.”
70 State of the Art: Psychology Sternberg et al. 1983,1987: –Cues to look for (= slots for frame): Spatiotemporal cues Value cues Properties Functions Cause/enablement information Class memberships Synonyms/antonyms –To acquire new words from context: Distinguish relevant/irrelevant information Selectively combine relevant information Compare this information with previous beliefs
71 Sternberg The couple there on the blind date was not enjoying the festivities in the least. An acapnotic, he disliked her smoking; and when he removed his hat, she, who preferred “ageless” men, eyed his increasing phalacrosis and grimaced.
72 State of the Art: Vocabulary Learning (II) Some dubious contributions: Mueser 1984: “Practicing Vocabulary in Context” –BUT: “context” = definition !! Clarke & Nation 1980: a “strategy” (algorithm?) 1.Look at word & context; determine POS 2.Look at grammatical context E.g., “who does what to whom”? 3.Look at wider context [E.g., search for Sternberg-like clues] 4.Guess the word; check your guess
73 CVA: From Algorithm to Curriculum “guess the word” = “then a miracle occurs” Surely, we computer scientists can “be more explicit”!
74 CVA: From algorithm to curriculum … Treat “guess” as a procedure call (“subroutine”) –Fill in the details with our algorithm –Convert the algorithm into a curriculum To enhance students’ abilities to use deliberate CVA strategies –To improve reading comprehension … and back again! Use knowledge gained from CVA case studies to improve the algorithm I.e., use Cassie to learn how to teach humans & use humans to learn how to teach Cassie
75 Why not use a dictionary? Because: People are lazy (!) Dictionaries are not always available Dictionaries are always incomplete Dictionary definitions are not always useful –‘chaste’ = df clean, spotless / “new dishes are chaste” –‘college’ = df a body of clergy living together and supported by a foundation Most words are learned via incidental CVA, not via dictionaries Most importantly: –Dictionary definitions are just more contexts!
76 Why not use a dictionary? Merriam-Webster New Collegiate Dictionary: –chaste. 1.innocent of unlawful sexual intercourse –student: stay away from that one! 2.celibate –student: huh? 3.pure in thought and act: modest –student: I have to find a sentence for that? 4.a: severely simple in design or execution: austere –student: huh? “severely”? “austere”? b: clean, spotless –student: all right!: “The plates were still chaste after much use.” –Deese 1967 / Miller 1985
77 Why not use a dictionary? Merriam-Webster (continued): –college. 1.a body of clergy living together and supported by a foundation 2.a building used for an educational or religious purpose 3.a: a self-governing constituent body of a university offering living quarters and instruction but not granting degrees… b: a preparatory or high school c: an independent institution of higher learning offering a course of general studies leading to a bachelor’s degree… –Problem: ordering is historical!
78 Why not use a dictionary? Merriam-Webster (continued): –infract:infringe –infringe:encroach –encroach: to enter by gradual steps or by stealth into the possessions or rights of another to advance beyond the usual or proper limits; trespass
79 Why not use a dictionary? Collins COBUILD Dictionary –“Helping Learners with Real English” –chaste. 1.Someone who is chaste does not have sex with anyone, or only has sex with their husband or wife; an old-fashioned use, used showing approval. EG She was a holy woman, innocent and chaste. 2.Something that is chaste is very simple in style, without much decoration. EG …chaste houses built in 1732
80 Why not use a dictionary? Collins COBUILD Dictionary –college. 1.A college is 1.1 an institution where students study for qualifications or do training courses after they have left school. … infract [not in dictionary] infringe. 1.If you infringe a law or an agreement, you break it. encroach. 1.To encroach on or upon something means to slowly take possession or control of it, so that someone else loses it bit by bit.
81 Question (objection): Teaching computers teaching humans! But : Our goal: –Not: teach people to “think like computers” –But: to explicate computable & teachable methods to hypothesize word meanings from context AI as computational psychology: –Devise computer programs that are essentially faithful simulations of human cognitive behavior –Can tell us something about human mind. We are teaching a machine, to see if what we learn in teaching it can help us teach students better.
82 How Does Our System Work? Uses a semantic network computer system –semantic networks = “concept maps” –serves as a model of the reader –represents: reader’s prior knowledge the text being read –can reason about the text and the reader’s knowledge
83 Fragment of reader’s prior knowledge: m3 = In “real life”, white is a color m6 = In “real life”, harts are deer m8 = In “real life”, deer are mammals m11 = In “real life”, halls are buildings m12 = In “real life”, b1 is named “King Arthur” m14 = In “real life”, b1 is a king(etc.)
84 m16 = if v3 has property v2 & if v2 is a color & if v3 v1 thenv1 is a kind of physical object
85 Reading the story: m17 = In the story, b2 is a hart m24 = In the story, the hart runs into b3 (b3 is King Arthur’s hall) – not shown (harts are deer) – not shown
86 The entire network showing the reader’s mental context consisting of prior knowledge, the story, & inferences. The definition algorithm searches this network & abstracts parts of it to produce a (preliminary) definition of ‘brachet’.
87 Implementation SNePS (Stuart C. Shapiro & SNeRG) : –Intensional, propositional semantic-network knowledge-representation & reasoning system –Node-based & path-based reasoning I.e., logical inference & generalized inheritance –SNeBR belief revision system Used for revision of definitions –SNaLPS natural-language input/output –“Cassie”: computational cognitive agent
88 How It Works SNePS represents: – background knowledge + text information in a single, consolidated semantic network Algorithms deductively search network for slot-fillers for definition frame Search is guided by desired slots –E.g., prefers general info over particular info, but takes what it can get
89 Noun Algorithm Find or infer: Basic-level class memberships (e.g., “dog”, rather than “animal”) –else most-specific-level class memberships –else names of individuals Properties of Ns (else, of individual Ns) Structure of Ns (else …) Functions of Ns (else …) Acts that Ns perform (else …) Agents that perform acts w.r.t. Ns & the acts they perform (else…) Ownership Synonyms Else do: “syntactic/algebraic manipulation” “Al broke a vase” a vase is something Al broke –Or: a vase is a breakable physical object
90 Verb Algorithm Find or infer: –Predicate structure: Categorize arguments/cases –Results of V’ing: Effects, state changes –Enabling conditions for V Future work: –Classification of verb-type –Synonyms [Also: preliminary work on adjective algorithm]
91 Belief Revision Used to revise definitions of words with different sense from current meaning hypothesis SNeBR (ATMS; Martins & Shapiro 88) : –If inference leads to a contradiction, then: 1.SNeBR asks user to remove culprit(s) 2.& automatically removes consequences inferred from culprit SNePSwD (SNePS w/ Defaults; Martins & Cravo 91) –Currently used to automate step 1, above AutoBR (Johnson & Shapiro, in progress) & new default reasoner (Bhushan & Shapiro, in progress) –Will replace SNePSwD
92 Revision & Expansion Removal & revision being automated via SNePSwD by ranking all propositions with kn_cat: most intrinsic info re: language; fundamental background info certain(“before” is transitive) story info in text (“King Lot rode to town”) life background info w/o variables or inference (“dogs are animals”) story-comp info inferred from text (King Lot is a king, rode on a horse) life-rule.1 everyday commonsense background info (BearsLiveYoung(x) Mammal(x)) life-rule.2 specialized background info (x smites y x kills y by hitting y) least certain questionable already-revised life-rule.2; not part of input
93 Belief Revision: “smite” Misunderstood word; 2-stage “subtractive” revision Background knowledge includes: (*) smite(x,y,t) hit(x,y,t) & dead(y,t) & cause(hit(x,y,t),dead(y,t)) P1: King Lot smote down King Arthur D1: If person x smites person y at time t, then x hits y at t, and y is dead at t Q1: What properties does King Arthur have? R1: King Arthur is dead. P2: King Arthur drew Excalibur. Q2: When did King Arthur do this? SNeBR is invoked: –KA’s drawing E is inconsistent with being dead –(*) replaced: smite(x,y,t) hit(x,y,t) & dead(y,t) & [dead(y,t) cause(hit, dead)] D2: If person x smites person y at time t, then x hits y at t & (y is dead at t) P3: [another passage in which ~(smiting death)] D3: If person x smites person y at time t, then x hits y at t
94 Belief Revision: “dress” “additive” revision Bkgd info includes: (1)dresses(x,y) z[clothing(z) & wears(y,z) (2)Spears don’t wear clothing (both kn_cat=life.rule.1 ) P1: King Arthur dressed himself. D1: A person can dress itself; result: it wears clothing. P2: King Claudius dressed his spear. [Cassie infers: King Claudius’s spear wears clothing.] Q2: What wears clothing? SNeBR is invoked: –KC’s spear wears clothing inconsistent with (2). –(1) replaced: dresses(x,y) z[clothing(z) & wears(y,z)] v NEWDEF –Replace (1), not (2), because of verb in antecedent of (1) (Gentner) P3: [other passages in which dressing spears precedes fighting] D2: A person can dress a spear or a person; result: person wears clothing or person is enabled to fight
95 Figure out meaning of word from what? context (i.e., the text)? –Werner & Kaplan 52, McKeown 85, Schatz & Baldwin 86 context and reader’s background knowledge? –Granger 77, Sternberg 83, Hastings 94 context including background knowledge? –Nation & Coady 88, Graesser & Bower 90 Note: –“context” = text context is external to reader’s mind Could also be spoken/visual/situative (still external) –“background knowledge”: internal to reader’s mind What is (or should be) the “context” for CVA?
96 Some Proposed Preliminary Definitions (to extract order out of confusion) Unknown word for a reader = def –Word or phrase that reader has never seen before –Or only has vague idea of its meaning Different levels of knowing meaning of word –Notation: “X”
97 Proposed preliminary definitions Text = def –(written) passage –containing X –single phrase or sentence … several paragraphs
98 Proposed preliminary definitions Co-text of X in some text = def –The entire text “minus” X; i.e., entire text surrounding X –E.g., if X = ‘brachet’, and text = “There came a white hart running into the hall with a white brachet next to him, and thirty couples of black hounds came running after them.” Then X’s co-text in this text = “There came a white hart running into the hall with a white ______ next to him, and thirty couples of black hounds came running after them.” –Cf. “cloze” tests in psychology But, in CVA, reader seeks meaning or definition –NOT a missing word or synonym: There’s no “correct” answer! –“Co-text” is what many mean by “context” BUT: they shouldn’t!
99 Proposed preliminary definitions The reader’s prior knowledge = def –the knowledge that the reader has when s/he begins to read the text –and is able to recall as needed while reading “knight picks up & carries brachet” ? small Warnings: –“knowledge” truth so, “prior beliefs” is better –“prior” vs. “background” vs. “world”, etc. See next slide!
100 Proposed preliminary definitions Possible synonyms for “prior knowledge”, each with different connotation: –Background knowledge: Can use for information that author assumes reader to have –World knowledge: General factual knowledge about things other than the text’s topic –Domain knowledge: Specialized, subject-specific knowledge about the text’s topic –Commonsense knowledge: Knowledge “everyone” has –E.g., CYC, “cultural literacy” (Hirsch) These overlap: –PK should include some CSK, might include some DK –BK might include much DK
101 Steps towards a Proper Definition of “Context” Step 1: –The context of X for a reader = def 1.The co-text of X 2. “+” the reader’s prior knowledge Both are needed! –After reading: “the white brachet bit the hart in the buttock” most subjects infer that brachets are (probably) animals, from: That text, plus: Available PK premise: “If x bites y, then x is (probably) an animal. –Inference is not an enthymeme! (argument with missing premise)
102 Proper definition of “context”: (inference not an enthymeme because): –When you read, you “internalize” the text You “bring it into” your mind Gärdenfors 1997, 1999; Jackendoff 2002 “Missing” premise might be in reader’s mind! –This “internalized text” is more important than the actual words on paper: Text:“I’m going to put the cat out” Misread as:“I’m going to put the car out” –leads to different understanding of “the text” –What matters is what the reader thinks the text is, Not what the text actually is Therefore …
103 Proper definition of “context”: Step 2: –The context of X for a reader = def A single KB, consisting of: 1. The reader’s internalized co-text of X 2. “ + ” the reader’s prior knowledge
104 Proper definition of “context”: But: What is “+” ? –Not: mere conjunction or union! –Active readers make inferences while reading. From text = “a white brachet” & prior commonsense knowledge = “only physical objects have color”, reader might infer that brachets are physical objects From “The knight took up the brachet and rode away with the brachet.” & prior commonsense knowledge about size, reader might infer that brachet is small enough to be carried –Whole > sum of parts: inference from [internalized text + PK] new info not in text or in PK I.e., you can learn from reading!
105 Proper definition of “context”: But: Whole < sum of parts! –Reader can learn that some prior beliefs were mistaken Or: reader can decide that text is mistaken (less likely) Reading & CVA need belief revision! operation “ + ”: –input:PK & internalized co-text –output:“belief-revised integration” of input, via: Expansion: –addition of new beliefs from ICT into PK, plus new inferences Revision: –retraction of inconsistent prior beliefs together with inferences from them Consolidation: –eliminate further inconsistencies
113 B-R Integrated KBText PK1 PK2 PK3 PK4 T1 I(T1) internalization P5 inference T2 I(T2) P6 T3 I(T3) P7 Note: All “contextual” reasoning is done in this “context”:
114 Proper definition of “context”: The context that reader R should use to hypothesize a meaning for R’s internalization of unknown word X as it occurs in text T = def –The belief-revised integration of R’s prior knowledge with R’s internalization of the co-text T–X.
115 Proper definition of “context”: One more detail: X needs to be internalized Context is a 3-place relation among: –Reader, word, and text Final(?) def.: –Let T be a text –Let R be a reader of T –Let X be a word in T (that is unknown to R) –Let T-X be X’s co-text in T. –Then: The context that R should use to hypothesize a meaning for R’s internalization of X as it occurs in T = def –The belief-revised integration of R’s prior knowledge with R’s internalization of T-X.
116 This definition agrees with… Cognitive-science & reading-theoretic views of text understanding –Schank 1982, Rumelhart 1985, etc. & AI techniques for text understanding: –Reader’s mind modeled by KB of prior knowledge Expressed in AI language (for us: SNePS) –Computational cognitive agent reads the text, “integrating” text info into its KB, and making inferences & performing belief revision along the way –When asked to define a word, Agent deductively searches this single, integrated KB for information to fill slots of a definition frame –Agent’s “context” for CVA = this single, integrated KB
117 Distinguishing Prior Knowledge from Integrated Co-Text So KB can be “disentangled” as needed for belief revision or to control inference: Each proposition in the single, integrated KB is marked with its “source”: –Originally from PK –Originally from text –Inferred Sources of premises
118 Some Open Questions Roles of spoken/visual/situative contexts Relation of CVA “context” to formal theories of context (e.g., McCarthy, Guha…) Relation of I(T) to prior-KB; e.g.: –Is I(T i ) true in prior-KB? It is “accepted pro tem”. –Is I(T) a “subcontext” of pKB or B-R KB? How to “activate” relevant prior knowledge. Etc.
119 Problem in Converting Algorithm into Curriculum “A knight picks up a brachet and carries it away …” Cassie: –Has “perfect memory” –Is “perfect reasoner” –Automatically infers that brachet is small People don’t always realize this: –May need prompting: How big is the brachet? –May need relevant background knowledge –May need help in drawing inferences Teaching CVA =? teaching general reading comprehension –Vocabulary knowledge correlates with reading comprehension
120 Research Methodology AI team: –Develop, implement, & test better computational theories of CVA –Translate into English for use by reading team Reading team: –Convert algorithms to curriculum –Think-aloud protocols To gather new data for use by AI team As curricular technique (case studies)
121 5-Minute Intermission
122 CVA & Science Education Original goal: CVA in & for science education –Use CVA to improve reading of STEM materials A side effect: CVA as science education –There are no ultimate authorities to consult No answers in the back of the book of life! As true for STEM as it is for reading about STEM – Goal of education = To learn how to learn on one’s own Help develop confidence & desire to use that skill –CVA as scientific method in miniature furthers this goal: Find clues/evidence (gathering data) Integrate them with personal background knowledge Use together to develop new theory (e.g., new meaning) Test/revise new theory (on future encounters with word)
124 Conclusion Developing a computational theory of CVA, which can become … a useful educational technique for improving vocabulary and reading comprehension a model of the scientific method a useful tool for learning on one’s own.
125 Participants Santosh Basapur (IE)Adam Lammert (Vasser/ugrad) Taha Suglatwala (CSE) Aishwarya Bhave (CSE) Amanda MacRae (LAI)Matthew Sweeney (ENG/CSE/ugrad) *Marc Broklawski (CSE) Brian Morgan (LAI)Matthew Watkins (CEN) Chien-chih Chi (PHI)*Scott Napieralski (CSE) *Karen Wieland (LAI) Justin Del Vecchio (CSE) Vikranth Rao (CSE/ugrad) Yulai Xie (CSE) Karen Ehrlich (Fredonia) Laurie Schimpf (LAI)Valerie Raybold Yakich (GEO) Jeffrey Galko (PHI)Ashwin Shah (CSE)& SNeRG members Christopher Garver (CSE) Stuart C. Shapiro (UB/CSE) (+ new students, Spring 2003 … Paul Gestwicki (CSE)Anuroopa Shenoy (CSE) … & Fall 2004 Kazuhiro Kawachi (LIN) Rajeev Sood (CSE)(* = supported on NSF grant)
126 Web Page ml
127 An Analysis of Think-Aloud Protocol of Good Readers Using CVA Strategies During Silent Reading
128 Limited Data Describing CVA Processes Ames. Deighton. Harmon.
129 Goals of Our Study Understand CVA processes. –Improve Cassie (computational model). –Build a more effective CVA curriculum.
130 Beginning and Ending Research Questions What text cues are used for CVA? What cognitive processes are used in CVA, including activation of background information?
131 Research questions continued What sense or meaning of an unknown word is gained from CVA? Do prior textual encounters with a hard word affect CVA in a new text?
132 Methodology: Hard Words Identified small set of “hard words.” –Some words from earlier work with Cassie. –Dale & O’Rourke and Carroll, Davis & Richman used as guides. –Some words came from scanning science and current event texts.
133 Methodology: Text Sets For each hard word: –identified a set of 7-17 authentic texts, –1-2 instances of each word in each text. Sometimes, replaced hard word with a neologism (a non-word): e.g., itresia for estuary
134 Methodology: Participants High school students. Excellent or outstanding readers. Readers given pre-test.
137 Methodology: Procedures Worked 1-1 with researcher. Told there was an unknown word. Asked to construct a dictionary-like definition of the highlighted target word. Read each passage, one at a time. Researcher explained meaning of other unknown words in text.
138 Procedures continued When hard word encountered, reader thought aloud while trying to gain sense of word’s meaning. Sessions recorded on audio tape. Sessions transcribed verbatim.
139 Our Procedures Vary from Normal CVA Processes and Conditions CVA usually activated by disruption of comprehension / text processing. Then... –Reader notes there is a hard word in the text. –Reader notes that lack of that word’s meaning disrupts comprehension. –Reader decides to figure out meaning of word.
140 Methodology: Analyses of Verbal Protocol Analyzed transcribed verbal protocol over and over again. Began by coding. Changed coding many times. Right now, still coding some data, but mostly writing multi-page summaries of each protocol.
141 Overview of Four Sets of Findings 1.Approaches to CVA. 2.Text cues and use of text cues for CVA. 3.Cognitive processes in CVA. 4.Sense of word meaning from CVA.
142 Findings–1: Readers’ Approaches to CVA Most read entire text, then returned to target word to work on its meaning. Some looked first at the target sentence, then read entire text. A few stopped reading at the target word or end of sentence to work on word meaning immediately.
143 Findings–2: Use of Text Cues Generally, readers did not initially re-inspect text for cues. Rather, think-alouds usually started with readers hypothesizing a meaning on basis of general passage comprehension and background knowledge.
144 Use of text cues continued As readers read further passages and were more confirmed in their hypothesis, many then did reinspect text to find support for their hypothesis. In this re-inspection, readers generally did not select the sentences we had predicted they would use.
145 These Findings Lead Us in Three Directions: To an unanticipated, but logical, conclusion about use of text cues. To curricular implications. To change our focus from examining the expected use of context cues to identifying the cognitive processes emerging from the data.
146 Why Was Rereading for Text Cues Not a Reader’s 1 st Step? Researchers knew text cues that provided useful information, but we knew the hard word’s meaning. Readers did not know the hard word’s meaning, so the relevant text cues had no particular salience for them.
147 Our Conclusion Using Context for CVA is Easy When You Already Know Meaning of Word!
148 Curricular Implications Teachers should model CVA with words they do not know in texts they have not previously seen. Students should practice CVA with words they do not know in texts they have not previously seen.
149 Findings-3: Cognitive Processes in CVA 1.Generated hypothesis / built model. 2.Drew inferences from general passage meaning and prior knowledge. 3.Used language cues.
150 Cognitive Processes in CVA-continued 4.Used information processing/knowledge acquisition processes (Sternberg, 1987). 5.Used knowledge from prior CVA texts. 6.Closure was rare.
151 CVA Cognitive Processes–1: Hypothesis / Model Building All readers hypothesized a meaning of the hard word.
152 On Further Encounters With Word, Readers... Confirmed hypothesis if congruent with text, usually stating rationale; Revised hypothesis if not congruent, usually stating rationale; or If hypothesis not congruent with text, but not enough information in text to revise it, readers questioned hypothesis.
153 Like Elshout-Mohr & Van Daalen- Kapteijns’ Good Readers Our readers generally modified hypotheses in keeping with text. Our readers appeared to “know” that what they stated was a tentative hypothesis.
154 Unlike Werner & Kaplan (1952) & Deegan (1995) It was rare for a reader to force text meaning to fit a previous hypothesis about word meaning.
155 CVA Cognitive Processes–2: Inferences from General Passage Meaning and Prior Knowledge Prior knowledge was: –Essential. –Idiosyncratic. –Pervasive.
157 CVA Cognitive Processes–3: Use of Language Cues Familiar expressions Figurative language Connected series
158 CVA Cognitive Processes–4: Sternberg’s Information Processing Separating relevant and irrelevant information. Combining relevant cues from text. Comparing new text information with hypothesized meaning and prior knowledge.
159 CVA Cognitive Processes–5: Knowledge from Prior CVA Texts Content of previous texts became part of prior knowledge. Readers used this prior knowledge from previous texts in subsequent texts.
160 CVA Cognitive Processes–6: Closure was Rare Tentativeness of hypothesis was generally maintained. Students rarely felt they had solid knowledge of word’s meaning.
161 Findings–4: Sense of Intended Word Meaning from CVA Right and wrong are not appropriate descriptors when contrasting meanings from CVA to intended meaning. Rational and defensible are better descriptors than right or wrong. Gradual and cumulative over texts.
162 Sense of the word continued 1.Don’t Know (dk) any meaning. 2.Not intended–No logical justification for sense of word. 3.Not intended–Reasonable justification for sense of word proffered. 4.Not intended–Based on language patterns, not general text meaning.
163 Sense of the word continued 5.Vague or partial sense of intended meaning. 6.Approximate sense of intended meaning. 7.Nearly full sense of intended meaning. 8.Full intended meaning.
164 Protocol Study Limitations We required reader to think about the unknown word and its meaning. –Readers ordinarily might choose to skip or ignore word. –Readers ordinarily might not note word.
165 Protocol limitations continued Our readers encountered word in multiple, consecutive texts. –Therefore, readers had immediate memory of the previous encounter. –This is atypical, as readers ordinarily might not encounter new word a second time for a long period.
166 Protocol limitations continued We sometimes ask leading questions: e.g., –To activate background knowledge. –To direct reader back to text. –To elicit reasoning processes. –To ask reader to think again.
167 Protocol limitations continued At times, we used non-words in place of real words: e.g., schmalion for tatterdemalion vedosarn for taciturn itresia for estuary.
168 Protocol Study Strengths Used authentic texts. Words were, generally, difficult and not previously known by readers. Used one hard word in repeated texts, not multiple hard words each in a different text.
169 How to Improve CVA in Classrooms Guess? Magical Mathematical Formula? CVA Strategies!
170 How to improve CVA continued 1.CVA curriculum should help students learn to apply reasoning and prior knowledge to textual cues. a.Read passage for full comprehension. b.Draw inferences from language, meaning, and prior knowledge. c.Summarize into a meaning-hypothesis frame.
171 How to improve CVA continued 2.Teachers: a.Model CVA with words whose meanings are unknown. b.Scaffold groups applying CVA strategies with words whose meanings are unknown. c.Guide small groups in think-alouds of CVA.
172 How to improve CVA continued 3.Students: –Independently apply CVA.
173 Developing Meaning Vocabulary in Classroom Traditional Methods –CVA.–Dictionaries –Morphology. –Direct teaching. Helping children build meaning vocabularies is not a set of specific teaching methods.
174 Developing Meaning Vocabulary in Classroom Most important is building a culture that fosters interest in words. –Leads to word learning naturally. –Facilitates life-long interest and word- learning abilities.
175 Recommendations Such classrooms have a culture that stimulates: –Word consciousness. –Word curiosity. –Word fun. –Wide reading.
176 Rationales for Our Recommendation 1.We know learning words is natural! Fully 90% (2,700±) of words learned per year are learned incidentally, outside of direct school instruction. Directly teaching words results in a gain of few words–e.g.,
177 Rationale for recommendations, continued But facilitating attraction and attentiveness to words students hear or see in their normal daily life could increase natural word learning by 100s, if not 1000s. These cognitive and affective factors promoted by such a culture will serve the student well for a lifetime.
178 Rationale for recommendations, continued 2.We know a child’s environment profoundly affects vocabulary growth.
179 Actual and Estimated Number of Words Heard from Months
180 Cumulative Number of New Words (Hart & Risley, 1995)