turing:CCS/575S10.ppt version: 20100315

Artificial Intelligence, Natural Language, the Chinese Room, & Reading for Understanding William J. Rapaport Department of Computer Science & Engineering, Department of Philosophy, Department of Linguistics, and Center for Cognitive Science rapaport@buffalo.edu http://www.cse.buffalo.edu/~rapaport

What Is AI? 2 Contrasting Views “The science of making machines do things that would require intelligence if done by humans.” Marvin Minsky Using humans to tell us how to program computers e.g., play chess, solve calculus problems, … e.g., see, use language, … “classical AI”

What Is AI? (continued) “The use of computer programs and programming techniques to cast light on the principles of intelligence in general and human thought in particular.” Margaret Boden Using computers to tell us something about humans. Cognitive science

What Is AI? (cont’d) My view: AI is …

My view: AI “computational cognition” asks: 2 further questions: What Is AI? (cont’d) My view: AI “computational cognition” asks: How much of cognition is computable? 2 further questions: What is computation? Turing machine How will we know if (some aspect of) cognition is computable? Turing test

Alan Turing British, 1912-1954 “invented” the idea of computation Turing “machine” cracked the Nazi “Enigma” code during WW II devised test for AI: Turing “test”

What Is a Computer?

Turing Machine Everything that is (humanly) computable can be computed using: a machine with: an “infinite” tape divided into squares (with ‘0’ on each) a movable read-write head a programming language with only: 2 nouns: 0, 1 2 verbs: move(left or right), print(0 or 1) 3 grammar rules: sequence (begin do this; then do that end) where “this”/“that” = print, move, or any grammatical instruction selection (if current square=0 then do this else do that) repetition (while current square=0 [or: current square=1] do this) Not everything can be computed! “halting problem”: can’t algorithmically detect infinite loops can cognition be computed?

(How) Can Computers Think? (Human) cognition is computable (?)  (Human) cognitive states & processes can be expressed as algorithms They can be implemented on non-human computers

How Computers Can Think (cont’d) Are computers executing such cognitive algorithms merely simulating cognitive states & processes? Or are they actually exhibiting them? Do such computers think? Answer: Turing’s Test Objection: Searle’s Chinese-Room Argument My reply: Computers can understand just by manipulating symbols (like a Turing machine)

The Imitation Game MAN WOMAN INTERROGATOR “I’m the woman”

The Turing Test The Imitation Game COMPUTER MAN “I’m the woman” WOMAN “I’m the woman” INTERROGATOR

The Turing Test #2 The Imitation Game COMPUTER MAN MAN “I’m the woman” WOMAN “I’m the woman” man man INTERROGATOR

The Turing Test #3 The Imitation Game COMPUTER HUMAN MAN “I’m the woman” WOMAN “I’m the woman” human human INTERROGATOR

The Turing Test Questions Responses I H? / C? “I believe that at the end of the century the use of words and general educated opinion will have altered so much that one will be able to speak of machines thinking without expecting to be contradicted.” - Turing 1950

Thinking vs. “Thinking” © The New Yorker, 5 July 1993

Thinking vs. “Thinking”, cont’d. Cartoon works because: You don’t know whom you’re communicating with via computer Nevertheless, we assume we are talking to a human I.e., entity with human cognitive capacities = Turing’s point, namely: Argument from Analogy: Solution to Problem of Other Minds = I know I think; how do I know you do? You are otherwise like me  (probably) you are like me w.r.t. thinking

Thinking vs. “Thinking”, cont’d. BUT: A dog that passes as a human isn’t a human John Searle: A computer that simulates thinking (passes a Turing test) isn’t really thinking. The Chinese-Room Argument

Thinking vs. “Thinking”, cont’d. pre-1950: computers were (only) human 2000: computers are (only) machines General educated opinion: Computers are viewed not as implementing-devices but in functional, I/O terms Ditto for ‘think’ (more later) BUT: it’s not really thinking (?)

The Chinese Room © MacroVU Press

The Chinese-Room Argument It’s possible to pass TT, yet not (really) think story + questions H I (in Chinese) (who can’t (native Chinese understand Ch.) speaker) + responses (Eng.) program (in fluent Chinese) for manipulating [Ch.] “squiggles”

Searle’s Chinese-Room Argument (1) Computer programs just manipulate symbols (2) Understanding has to do with meaning (3) Symbol manipulation alone is not sufficient for meaning (4)  No computer program can understand ¬ (3): You can understand just by manipulating symbols!

Contextual Vocabulary Acquisition Could Searle-in-the-room figure out a meaning for an unknown squiggle? Yes! Same way you can figure out a meaning for an unfamiliar word from context

What Does ‘Brachet’ Mean What Does ‘Brachet’ Mean? (From Malory’s Morte D’Arthur [page # in brackets]) 1. There came a white hart running into the hall with a white brachet next to him, and thirty couples of black hounds came running after them. [66] As the hart went by the sideboard, the white brachet bit him. [66] The knight arose, took up the brachet and rode away with the brachet. [66] A lady came in and cried aloud to King Arthur, “Sire, the brachet is mine”. [66] There was the white brachet which bayed at him fast. [72] 18. The hart lay dead; a brachet was biting on his throat, and other hounds came behind. [86] Predefine ‘hart’, ‘sideboard’, ‘bay’. Sentence 3: if no one suggests that brachet is small, ask how big it is.

Reader’s Prior Knowledge Where is the “context”? Reader’s Prior Knowledge Text PK1 PK2 PK3 PK4

Reader’s Prior Knowledge Text T1 PK1 PK2 PK3 PK4

Internalized (Co-)Text integrated with Reader’s Prior Knowledge PK1 PK2 PK3 PK4 internalization I(T1)

Belief-Revised Internalized Text integrated with Reader’s Prior Knowledge internalization T1 PK1 PK2 PK3 PK4 I(T1) inference P5

B-R Integrated KB Text PK1 PK2 PK3 PK4 T1 I(T1) T2 P5 I(T2) P6 internalization T1 I(T1) T2 inference P5 I(T2) P6

B-R Integrated KB Text T1 PK1 PK2 PK3 PK4 I(T1) T2 T3 P5 I(T2) P6 internalization I(T1) T2 inference T3 P5 I(T2) P6 I(T3)

B-R Integrated KB Text T1 PK1 PK2 PK3 PK4 I(T1) T2 T3 P5 I(T2) P6 internalization I(T1) T2 inference T3 P5 I(T2) P6 I(T3)

Reader’s Mind Text T1 PK1 PK2 PK3 PK4 P7 I(T1) T2 T3 P5 I(T2) P6 I(T3) “Context” for CVA is the reader’s mind, not the (co-)text Reader’s Mind Text internalization T1 PK1 PK2 PK3 PK4 P7 I(T1) T2 inference T3 P5 I(T2) P6 I(T3)

“Words should be thought of not as having intrinsic meaning, but as providing cues to meaning.” Jeffrey L. Elman, “On the Meaning of Words and Dinosaur Bones: Lexical Knowledge Without a Lexicon” (2009) “Words might be better understood as operators, entities that operate directly on mental states in what can be formally understood as a dynamical system.” Jeffrey L. Elman, “On Words and Dinosaur Bones: Where Is Meaning?” (2007)

Computational CVA Implemented in SNePS (Shapiro 1979; Shapiro & Rapaport 1992) Intensional, propositional, semantic-network knowledge-representation, reasoning, & acting system “intensional”: e.g., can represent fictional objects “propositional”: can represent sentences in a text “semantic network”: labeled, directed graph with nodes linked by arcs indexed by node from any node, can describe rest of network Serves as model of the reader (“Cassie”)

Computational CVA (cont’d) KB: SNePS representation of reader’s prior knowledge I/P: SNePS representation of word in its co-text Processing (“simulates”/“models”/is?! reading): Uses logical inference, generalized inheritance, belief revision to reason about text integrated with reader’s prior knowledge N & V definition algorithms deductively search this “belief-revised, integrated” KB (the wide context) for slot fillers for definition frame… O/P: Definition frame slots (features): classes, structure, actions, properties, etc. fillers (values): info gleaned from context (= integrated KB) My colleagues and I have developed a computational theory of CVA, which is implemented in a propositional semantic-network knowledge-representation-and-reasoning system (SNePS; Shapiro & Rapaport 1987, 1992, 1995; Shapiro & Group 2004). Our computational system begins with a stored knowledge base containing SNePS representations of relevant PK. It takes as input SNePS representations of a passage containing an unfamiliar word. The processing begins with inferences drawn from these two sources of information. When asked to define the word, it applies definition algorithms (for nouns and for verbs; adjectives are under investigation) that deductively search the resulting network for information of the sort that might be found in a dictionary definition, outputting a definition frame (or schema) whose slots are the kinds of features that a definition might contain (e.g., class membership, properties, actions, spatio-temporal information, etc.) and whose slot-fillers contain information gleaned from the network. (Details of the underlying theory, representations, processing, inferences, belief revision, and definition algorithms are presented in Ehrlich 1995, 2004; Ehrlich & Rapaport 1997, 2004; Rapaport & Ehrlich 2000; Rapaport & Kibby 2002; Rapaport 2003.) We are investigating ways to make our system more robust, to embed it in a natural-language-processing system, and to incorporate what Sternberg and colleagues (1983ab, 1987) call “internal” context, i.e., morphological (and etymological) analysis.

Cassie learns what “brachet” means: Background info about: Cassie learns what “brachet” means: Background info about: harts, animals, King Arthur, etc. No info about: brachets Input: formal-language (SNePS) version of simplified English A hart runs into King Arthur’s hall. • In the story, B12 is a hart. • In the story, B13 is a hall. • In the story, B13 is King Arthur’s. • In the story, B12 runs into B13. A white brachet is next to the hart. • In the story, B14 is a brachet. • In the story, B14 has the property “white”. • Therefore, brachets are physical objects. (deduced while reading; PK: Cassie believes that only physical objects have color)

--> (defineNoun "brachet") Definition of brachet: Class Inclusions: phys obj, Possible Properties: white, Possibly Similar Items: animal, mammal, deer, horse, pony, dog, I.e., a brachet is a physical object that can be white and that might be like an animal, mammal, deer, horse, pony, or dog

A hart runs into King Arthur’s hall A hart runs into King Arthur’s hall. A white brachet is next to the hart. The brachet bites the hart’s buttock. [PK: Only animals bite] --> (defineNoun "brachet") Definition of brachet: Class Inclusions: animal, Possible Actions: bite buttock, Possible Properties: white, Possibly Similar Items: mammal, pony,

A hart runs into King Arthur’s hall. A white brachet is next to the hart. The brachet bites the hart’s buttock. The knight picks up the brachet. The knight carries the brachet. [PK: Only small things can be picked up/carried] --> (defineNoun "brachet") Definition of brachet: Class Inclusions: animal, Possible Actions: bite buttock, Possible Properties: small, white, Possibly Similar Items: mammal, pony,

A hart runs into King Arthur’s hall A hart runs into King Arthur’s hall. A white brachet is next to the hart. The brachet bites the hart’s buttock. The knight picks up the brachet. The knight carries the brachet. The lady says that she wants the brachet. [PK: Only valuable things are wanted] --> (defineNoun "brachet") Definition of brachet: Class Inclusions: animal, Possible Actions: bite buttock, Possible Properties: valuable, small, white, Possibly Similar Items: mammal, pony,

A hart runs into King Arthur’s hall A hart runs into King Arthur’s hall. A white brachet is next to the hart. The brachet bites the hart’s buttock. The knight picks up the brachet. The knight carries the brachet. The lady says that she wants the brachet. The brachet bays at Sir Tor. [PK: Only hunting dogs bay] --> (defineNoun "brachet") Definition of brachet: Class Inclusions: hound, dog, Possible Actions: bite buttock, bay, hunt, Possible Properties: valuable, small, white, I.e. A brachet is a hound (a kind of dog) that can bite, bay, and hunt, and that may be valuable, small, and white.

Algorithms (for Computers) Generate initial hypothesis by “syntactic manipulation” Algebra: Solve an equation for unknown value X Syntax: “Solve” a sentence for unknown word X “A white brachet (X) is next to the hart”  X (a brachet) is something that is next to the hart and that can be white. “Define” node/word X in terms of immediately connected nodes/words Deductively search wide context to update hypothesis Look for: class membership, properties, structure, acts, agents, etc. Define” node/word X in terms of some (but not all) other connected nodes/words Output definition “frame” (schema)

A Computational Theory of CVA A word does not have a unique meaning. A word does not have a “correct” meaning. Author’s intended meaning for word doesn’t need to be known by reader in order for reader to understand word in context Even familiar/well-known words can acquire new meanings in new contexts. Neologisms are usually learned only from context Every co-text can give some clue to a meaning for a word. Generate initial hypothesis via syntactic/algebraic manipulation But co-text must be integrated with reader’s prior knowledge Large co-text + large PK  more clues Lots of occurrences of word allow asymptotic approach to stable meaning hypothesis CVA is computable CVA is “open-ended”, hypothesis generation. CVA ≠ guess missing word (“cloze”);  CVA ≠ word-sense disambiguation Some words are easier to compute meanings for than others (N < V < Adj/Adv) CVA can improve general reading comprehension (through active reasoning) CVA can & should be taught in schools

CVA as Symbol Manipulation We “solved” each sentence for the unknown word in terms of the rest of the text together with our background knowledge Just as we can solve an algebra problem in terms of the rest of the equation By manipulating symbols which is what computers do!

Computational Natural-Language Understanding What else is needed? besides symbol manipulation

Mind as a Symbol-Manipulation System To understand language, a cognitive agent must: Take discourse as input Understand ungrammatical input Make inferences & revise beliefs Make plans For speech acts To ask/answer questions To initiate conversation Understand plans Speech-act plans of interlocutor Construct user model Learn (about world, language) Have background/world/commonsense knowledge Remember What it heard, learned, inferred, revised = have a mind! All of this can be done by computers manipulating symbols!

Reading for Understanding Research Initiative Institute of Education Sciences US Department of Education 5 or 6 “R&D Core Teams” 1 or 2 “Network Assessment Teams” “Manhattan Project” / “Apollo Mission” to improve (the teaching of) reading for understanding/reading comprehension $20 million over 5 years

A Center for Reading for Understanding A Research and Development Core Team to Integrate Vocabulary, Writing, Reasoning, Multimodal Literacies, and Oral Discourse to Improve Reading Comprehension Principal location: UB Satellite locations: Niagara University, Penn State University Affiliated school districts: Niagara Falls CSD, Cleveland Hill USD, State College Area SD

Projects Writing Intensive Reading Comprehension Jim Collins (UB/LAI) Contextual Vocabulary Acquisition Bill Rapaport (UB/CSE & CCS) Multimodal Literacies Kathleen Collins (Penn State) Virtual Interactive Environments to improve vocabulary and reading comprehension Lynn Shanahan & Mary McVee (UB/LAI) Interactively Modeled Metacognitive Thinking Rob Erwin (Niagara U) Bilingualism and Basic Cognitive Processes Janina Brutt-Griffler (UB/LAI) Experimental Design & Statistical Analysis Ariel Aloe (UB/CSEP)

Writing-Intensive Reading Comprehension — Jim Collins (LAI) Based on previous successful research “Thinksheets” that guide students’ writing about a text improve their reading comprehension Thinksheets begin with questions whose answers can be found directly in the text Then gradually ask more abstract questions requiring inference. Goals: extend this research apply to more grade levels semi-automate development of thinksheets

Multimodal Literacies — Kathleen Collins (Penn State) Previous research: if teachers work with professional artists to develop strategies for integrating the arts as forms of inquiry and meaning-making into the school curriculum, then more students are able to engage successfully with core curricula. Multimodal interventions & enhancements will be designed to include the following broad categories: Visual/Graphic (still photography, illustration, digital video) Oral/Interactional (scripted dialogue, interviews, guided conversations, podcasts) Tactile (movement, model building, sculpting)

Virtual Interactive Environments to Develop Vocabulary Knowledge — Lynn Shanahan & Mary McVee (LAI) Develop extensible & effective computational framework for improving vocabulary knowledge through virtual-contextual learning Couple a virtual world, large projection screen, stereo sound, & positional tracking using commodity devices (e.g., Wii controllers). Virtual environment will be created using a programming toolkit developed for use in computer games and engineering simulations. NYSCEDII Rather than interact with a computer monitor, the environment will use projection technology to provide learners with the opportunity to interact with objects of actual size, increasing the immersion into the virtual world.

Interactively Modeled Metacognitive Thinking — Rob Erwin (NU) Instead of teaching specific strategies for reading comprehension… (which have recently been shown to be worse than teaching textual content) … Have students develop their own strategies on an as-needed basis Then compare to strategies- & content-approaches

Bilingualism & Basic Cognitive Processes — Janina Brutt-Griffler (LAI) What are the cognitive advantages of being a bilingual reader? What are the cognitive aspects of text comprehension and possible linguistic advantages observed in bilingual readers?

Contextual Vocabulary Acquisition — Bill Rapaport (CSE & CCS) Previous research: A reader's understanding of a word's meaning in a context is a function of both the context (the surrounding words) and the reader's prior knowledge. Already accomplished: A procedure for successful CVA can be expressed in terms so precise that they can be programmed into a computer. That computational procedure can then be converted into a strategy teachable to human readers We propose to embed this procedure in a curricular intervention that can help readers improve both their vocabulary and their reading comprehension.

Our goal is not to improve vocabulary per se, but: CVA (cont’d) Our goal is not to improve vocabulary per se, but: to improve reading comprehension by active thinking about the text with a specific goal of vocabulary enrichment in mind and to provide readers with a method that can be used independently… e.g., when they are reading on their own …to learn new vocabulary and to improve comprehension.

Our CVA algorithms fall into this category CVA (cont’d) GOFAI: if we know how to explicitly teach some cognitive task to humans e.g., play chess, do calculus, prove theorems then we can explicitly program a computer to do that task pretty much as humans do it. Our CVA algorithms fall into this category Can we do the converse? Can a human reader learn vocabulary & improve reading comprehension by carrying out our algorithm?

Not “think like a computer” CVA (cont’d) Not “think like a computer” I.e., rigidly, mechanically, uncreatively But: what we have learned by teaching a computer to do CVA can now help us teach human readers who need guidance in CVA.

CVA (cont’d) Research questions: Can a computer algorithm be translated into a successful curricular intervention? Does this computer-based curriculum improve (meaning) vocabulary? CVA students vs. “typical context-based” students vs. “no treatment” control Each read same passage with single unfamiliar word & figure out a meaning from context CVA students vs. “direct-method” control at time t Tested on meaning at time t > t

CVA (cont’d) Can a computer algorithm be translated into a successful curricular intervention? Does the algorithm-based curriculum improve reading comprehension? CVA students vs. “typical context-based” students vs. no-treatment students Test for reading comprehension on passages… containing unfamiliar words with no unfamiliar word

Other issues: CVA (cont’d) “teacher training”/professional development oral discussion: explicit instruction on using language to reason is valuable use thinksheet as “detective”/“scientist” notebook role & nature of prior knowledge what is needed, how to identify it, how to elicit it, etc. test at different ages & in STEM vs. ELA develop software for use in classroom student can ask Cassie how she figured it out