Presentation is loading. Please wait.

Presentation is loading. Please wait.

Search Engines & Question Answering Giuseppe Attardi Dipartimento di Informatica Università di Pisa.

Similar presentations


Presentation on theme: "Search Engines & Question Answering Giuseppe Attardi Dipartimento di Informatica Università di Pisa."— Presentation transcript:

1 Search Engines & Question Answering Giuseppe Attardi Dipartimento di Informatica Università di Pisa

2 Question Answering IR: find documents relevant to query IR: find documents relevant to query –query: boolean combination of keywords QA: find answer to question QA: find answer to question –Question: expressed in natural language –Answer: short phrase (< 50 byte)

3 Trec-9 Q&A track 693 fact-based, short answer questions 693 fact-based, short answer questions –either short (50 B) or long (250 B) answer ~3 GB newspaper/newswire text (AP, WSJ, SJMN, FT, LAT, FBIS) ~3 GB newspaper/newswire text (AP, WSJ, SJMN, FT, LAT, FBIS) Score: MRR (penalizes second answer) Score: MRR (penalizes second answer) Resources: top 50 (no answer for 130 q) Resources: top 50 (no answer for 130 q) Questions: 186 (Encarta), 314 (seeds from Excite logs), 193 (syntactic variants of 54 originals) Questions: 186 (Encarta), 314 (seeds from Excite logs), 193 (syntactic variants of 54 originals)

4 Commonalities Approaches: Approaches: –question classification –finding entailed answer type –use of WordNet High-quality document search helpful (e.g. Queen College) High-quality document search helpful (e.g. Queen College)

5 Sample Questions Q: Who shot President Abraham Lincoln? A: John Wilkes Booth Q: How many lives were lost in the Pan Am crash in Lockerbie? A: 270 Q: How long does it take to travel from London to Paris through the Channel? A: three hours 45 minutes Q: Which Atlantic hurricane had the highest recorded wind speed? A: Gilbert (200 mph) Q: Which country has the largest part of the rain forest? A: Brazil (60%)

6 Question Types Class 1 Answer: single datum or list of items C: who, when, where, how (old, much, large) Class 2 A: multi-sentence C: extract from multiple sentences Class 3 A: across several texts C: comparative/contrastive Class 4 A: an analysis of retrieved information C: synthesized coherently from several retrieved fragments Class 5 A: result of reasoning C: word/domain knowledge and common sense reasoning

7 Question subtypes Class 1.A About subjects, objects, manner, time or location Class 1.B About properties or attributes Class 1.C Taxonomic nature

8 Results (long)

9 Falcon: Architecture Question Question Semantic Form Expected Answer Type Answer Paragraphs Answer Semantic Form Answer Answer Logical Form Paragraph Index Question Processing Paragraph Processing Answer Processing Paragraph filtering Collins Parser + NE Extraction Abduction Filter Coreference Resolution Question Taxonomy Question Expansion WordNet Collins Parser + NE Extraction Question Logical Form

10 Question parse Who was the first Russian astronaut to walk in space WPVBDDTJJNNPNPTOVBINNN NP PP VP S S

11 Question semantic form astronaut walkspace Russian first PERSON first(x)  astronaut(x)  Russian(x)  space(z)  walk(y, z, x)  PERSON(x) Question logic form: Answer type

12 Expected Answer Type sizeArgentina dimension QUANTITY WordNet Question: What is the size of Argentina?

13 Questions about definitions Special patterns: Special patterns: –What {is|are} …? –What is the definition of …? –Who {is|was|are|were} …? Answer patterns: Answer patterns: –…{is|are} –…, {a|an|the} –… -

14 Question Taxonomy Reason Number Manner Location Organization Product Language Mammal Currency Nationality Question Game Reptile Country City Province Continent Speed Degree Dimension Rate Duration Percentage Count

15 Question expansion Morphological variants Morphological variants –invented  inventor Lexical variants Lexical variants –killer  assassin –far  distance Semantic variants Semantic variants –like  prefer

16 Indexing for Q/A Alternatives: Alternatives: –IR techniques –Parse texts and derive conceptual indexes Falcon uses paragraph indexing: Falcon uses paragraph indexing: –Vector-Space plus proximity –Returns weights used for abduction

17 Abduction to justify answers Backchaining proofs from questions Backchaining proofs from questions Axioms: Axioms: –Logical form of answer –World knowledge (WordNet) –Coreference resolution in answer text Effectiveness: Effectiveness: –14% improvement –Filters 121 erroneous answers (of 692) –Requires 60% question processing time

18 TREC 13 QA Several subtasks: Several subtasks: –Factoid questions –Definition questions –List questions –Context questions LCC still best performance, but different architecture LCC still best performance, but different architecture

19 LCC Block Architecture Passage Retrieval Passage Retrieval Answer Extraction  Theorem Prover  Answer Justification  Answer Reranking Axiomatic Knowledge Base Answer Extraction  Theorem Prover  Answer Justification  Answer Reranking Axiomatic Knowledge Base WordNetNER WordNetNER Document Retrieval Document Retrieval Keywords Passages Question Semantics Captures the semantics of the question Selects keywords for PR Extracts and ranks passages using surface-text techniques Extracts and ranks answers using NL techniques QA Question Parse  Semantic Transformation  Recognition of Expected Answer Type  Keyword Extraction Question Parse  Semantic Transformation  Recognition of Expected Answer Type  Keyword Extraction Question ProcessingAnswer Processing

20 Question Processing Two main tasks Two main tasks –Determining the type of the answer –Extract keywords from the question and formulate a query

21 Answer Types Factoid questions… Factoid questions… –Who, where, when, how many… –The answers fall into a limited and somewhat predictable set of categories Who questions are going to be answered by… Where questions… –Generally, systems select answer types from a set of Named Entities, augmented with other types that are relatively easy to extract

22 Answer Types Of course, it isn’t that easy… Of course, it isn’t that easy… –Who questions can have organizations as answers Who sells the most hybrid cars? –Which questions can have people as answers Which president went to war with Mexico?

23 Answer Type Taxonomy Contains ~9000 concepts reflecting expected answer types Contains ~9000 concepts reflecting expected answer types Merges named entities with the WordNet hierarchy Merges named entities with the WordNet hierarchy

24 Answer Type Detection Most systems use a combination of hand-crafted rules and supervised machine learning to determine the right answer type for a question. Most systems use a combination of hand-crafted rules and supervised machine learning to determine the right answer type for a question. Not worthwhile to do something complex here if it can’t also be done in candidate answer passages. Not worthwhile to do something complex here if it can’t also be done in candidate answer passages.

25 Keyword Selection Answer Type indicates what the question is looking for: Answer Type indicates what the question is looking for: –It can be mapped to a NE type and used for search in enhanced index Lexical terms (keywords) from the question, possibly expanded with lexical/semantic variations provide the required context. Lexical terms (keywords) from the question, possibly expanded with lexical/semantic variations provide the required context.

26 Keyword Extraction Questions approximated by sets of unrelated keywords Questions approximated by sets of unrelated keywords Question (from TREC QA track) Keywords Q002: What was the monetary value of the Nobel Peace Prize in 1989? monetary, value, Nobel, Peace, Prize Q003: What does the Peugeot company manufacture? Peugeot, company, manufacture Q004: How much did Mercury spend on advertising in 1993? Mercury, spend, advertising, 1993 Q005: What is the name of the managing director of Apricot Computer? name, managing, director, Apricot, Computer

27 Keyword Selection Algorithm 1. Select all non-stopwords in quotations 2. Select all NNP words in recognized named entities 3. Select all complex nominals with their adjectival modifiers 4. Select all other complex nominals 5. Select all nouns with adjectival modifiers 6. Select all other nouns 7. Select all verbs 8. Select the answer type word

28 Passage Retrieval Extracts and ranks passages using surface-text techniques Passage Retrieval Passage Retrieval Answer Extraction  Theorem Prover  Answer Justification  Answer Reranking Axiomatic Knowledge Base Answer Extraction  Theorem Prover  Answer Justification  Answer Reranking Axiomatic Knowledge Base WordNetNER WordNetNER Document Retrieval Document Retrieval Keywords Passages Question Semantics QA Question Parse  Semantic Transformation  Recognition of Expected Answer Type  Keyword Extraction Question Parse  Semantic Transformation  Recognition of Expected Answer Type  Keyword Extraction Question ProcessingAnswer Processing

29 Passage Extraction Loop Passage Extraction Component Passage Extraction Component –Extracts passages that contain all selected keywords –Passage size dynamic –Start position dynamic Passage quality and keyword adjustment Passage quality and keyword adjustment –In the first iteration use the first 6 keyword selection heuristics –If the number of passages is lower than a threshold  query is too strict  drop a keyword –If the number of passages is higher than a threshold  query is too relaxed  add a keyword

30 Passage Scoring Passages are scored based on keyword windows Passages are scored based on keyword windows –For example, if a question has a set of keywords: {k1, k2, k3, k4}, and in a passage k1 and k2 are matched twice, k3 is matched once, and k4 is not matched, the following windows are built: k1 k2 k3 k2 k1 Window 1 k1 k2 k3 k2 k1 Window 2 k1 k2 k3 k2 k1 Window 3 k1 k2 k3 k2 k1 Window 4

31 Passage Scoring Passage ordering is performed using a sort that involves three scores: Passage ordering is performed using a sort that involves three scores: –The number of words from the question that are recognized in the same sequence in the window –The number of words that separate the most distant keywords in the window –The number of unmatched keywords in the window

32 Answer Extraction Extracts and ranks answers using NL techniques Passage Retrieval Passage Retrieval Answer Extraction  Theorem Prover  Answer Justification  Answer Reranking Axiomatic Knowledge Base Answer Extraction  Theorem Prover  Answer Justification  Answer Reranking Axiomatic Knowledge Base WordNetNER WordNetNER Document Retrieval Document Retrieval Keywords Passages Question Semantics QA Question Parse  Semantic Transformation  Recognition of Expected Answer Type  Keyword Extraction Question Parse  Semantic Transformation  Recognition of Expected Answer Type  Keyword Extraction Question ProcessingAnswer Processing

33 Ranking Candidate Answers n Answer type: Person n Text passage: “Among them was Christa McAuliffe, the first private citizen to fly in space. Karen Allen, best known for her starring role in “Raiders of the Lost Ark”, plays McAuliffe. Brian Kerwin is featured as shuttle pilot Mike Smith...” Q066: Name the first private citizen to fly in space.

34 Ranking Candidate Answers n Answer type: Person n Text passage: “Among them was Christa McAuliffe, the first private citizen to fly in space. Karen Allen, best known for her starring role in “Raiders of the Lost Ark”, plays McAuliffe. Brian Kerwin is featured as shuttle pilot Mike Smith...” n Best candidate answer: Christa McAuliffe Q066: Name the first private citizen to fly in space.

35 Features for Answer Ranking Number of question terms matched in the answer passage Number of question terms matched in the answer passage Number of question terms matched in the same phrase as the candidate answer Number of question terms matched in the same phrase as the candidate answer Number of question terms matched in the same sentence as the candidate answer Number of question terms matched in the same sentence as the candidate answer Flag set to 1 if the candidate answer is followed by a punctuation sign Flag set to 1 if the candidate answer is followed by a punctuation sign Number of question terms matched, separated from the candidate answer by at most three words and one comma Number of question terms matched, separated from the candidate answer by at most three words and one comma Number of terms occurring in the same order in the answer passage as in the question Number of terms occurring in the same order in the answer passage as in the question Average distance from candidate answer to question term matches Average distance from candidate answer to question term matches

36 Lexical Chains Question: When was the internal combustion engine invented? Answer: The first internal combustion engine was built in Lexical chains: (1) invent:v#1  HYPERNIM  create_by_mental_act:v#1  HYPERNIM  create:v#1  HYPONIM  build:v#1 Question: How many chromosomes does a human zygote have? Answer: 46 chromosomes lie in the nucleus of every normal human cell. Lexical chains: (1)zygote:n#1  HYPERNIM  cell:n#1  HAS.PART  nucleus:n#1

37 Theorem Prover Q: What is the age of the solar system? QLF: quantity_at(x2) & age_nn(x2) & of_in(x2,x3) & solar_jj(x3) & system_nn(x3) Question Axiom: (exists x1 x2 x3 (quantity_at(x2) & age_NN(x2) & of_in(x2,x3) & solar_jj(x3) & system_nn(x3)) Answer: The solar system is 4.6 billion years old. Wordnet Gloss: old_jj(x6)  live_vb(e2,x6,x2) & for_in(e2,x1) & relatively_jj(x1) & long_jj(x1) & time_nn(x1) & or_cc(e5,e2,e3) & attain_vb(e3,x6,x2) & specific_jj(x2) & age_nn(x2) Linguistic Axiom: all x1 (quantity_at(x1) & solar_jj(x1) & system_nn(x1)  of_in(x1,x1)) Proof: ¬quantity_at(x2) | ¬age_nn(x2) | ¬of_in(x2,x3) | ¬solar_jj(x3) | ¬system_nn(x3) Refutation assigns value to x2

38 Is the Web Different? In TREC (and most commercial applications), retrieval is performed against a smallish closed collection of texts. In TREC (and most commercial applications), retrieval is performed against a smallish closed collection of texts. The diversity/creativity in how people express themselves necessitates all that work to bring the question and the answer texts together. The diversity/creativity in how people express themselves necessitates all that work to bring the question and the answer texts together. But… But…

39 The Web is Different On the Web popular factoids are likely to be expressed in a gazillion different ways. On the Web popular factoids are likely to be expressed in a gazillion different ways. At least a few of which will likely match the way the question was asked. At least a few of which will likely match the way the question was asked. So why not just grep (or agrep) the Web using all or pieces of the original question. So why not just grep (or agrep) the Web using all or pieces of the original question.

40 AskMSR Process the question by… Process the question by… –Forming a search engine query from the original question –Detecting the answer type Get some results Get some results Extract answers of the right type based on Extract answers of the right type based on –How often they occur

41 Step 1: Rewrite the questions Intuition: The user’s question is often syntactically quite close to sentences that contain the answer Intuition: The user’s question is often syntactically quite close to sentences that contain the answer –Where is the Louvre Museum located? The Louvre Museum is located in Paris –Who created the character of Scrooge? Charles Dickens created the character of Scrooge.

42 Query rewriting Classify question into seven categories –Who is/was/are/were…? –When is/did/will/are/were …? –Where is/are/were …? a. Hand-crafted category-specific transformation rules e.g.: For where questions, move ‘is’ to all possible locations Look to the right of the query terms for the answer. Look to the right of the query terms for the answer. “Where is the Louvre Museum located?”  “is the Louvre Museum located”  “is the Louvre Museum located”  “the is Louvre Museum located”  “the is Louvre Museum located”  “the Louvre is Museum located”  “the Louvre is Museum located”  “the Louvre Museum is located”  “the Louvre Museum is located”  “the Louvre Museum located is”  “the Louvre Museum located is”

43 Step 2: Query search engine Send all rewrites to a Web search engine Send all rewrites to a Web search engine Retrieve top N answers ( ) Retrieve top N answers ( ) For speed, rely just on search engine’s “snippets”, not the full text of the actual document For speed, rely just on search engine’s “snippets”, not the full text of the actual document

44 Step 3: Gathering N-Grams Enumerate all N-grams (N=1,2,3) in all retrieved snippets Enumerate all N-grams (N=1,2,3) in all retrieved snippets Weight of an n-gram: occurrence count, each weighted by “reliability” (weight) of rewrite rule that fetched the document Weight of an n-gram: occurrence count, each weighted by “reliability” (weight) of rewrite rule that fetched the document –Example: “Who created the character of Scrooge?” Dickens 117 Christmas Carol 78 Charles Dickens 75 Disney 72 Carl Banks 54 A Christmas 41 Christmas Carol 45 Uncle 31

45 Step 4: Filtering N-Grams Each question type is associated with one or more “data-type filters” = regular expressions for answer types Each question type is associated with one or more “data-type filters” = regular expressions for answer types Boost score of n-grams that match the expected answer type. Boost score of n-grams that match the expected answer type. Lower score of n-grams that don’t match. Lower score of n-grams that don’t match.

46 Step 5: Tiling the Answers Dickens Charles Dickens Mr Charles Scores merged, discard old n-grams Mr Charles Dickens Score 45

47 Results Standard TREC contest test-bed (TREC 2001): 1M documents; 900 questions Standard TREC contest test-bed (TREC 2001): 1M documents; 900 questions –Technique does ok, not great (would have placed in top 9 of ~30 participants) –But with access to the Web… they do much better, would have come in second on TREC 2001

48 Harder Questions Factoid question answering is really pretty silly. Factoid question answering is really pretty silly. A more interesting task is one where the answers are fluid and depend on the fusion of material from disparate texts over time. A more interesting task is one where the answers are fluid and depend on the fusion of material from disparate texts over time. –Who is Condoleezza Rice? –Who is Mahmoud Abbas? –Why was Arafat flown to Paris?

49 IXE Components

50 IXE Framework Object Store Indexer OS Abstraction Text MaxEntropy Sent. Splitter Readers POS Tagger NE Tagger Passage Index Clustering Crawler Search Web ServiceWrappers Unicode RegExp Tokenizer Suffix Trees Files Mem Mapping Threads Synchronization Python Perl Java EventStream ContextStream GIS

51 Language Processing Tools Maximum Entropy classifier Maximum Entropy classifier Sentence Splitter Sentence Splitter Multi-language POS Tagger Multi-language POS Tagger Multi-language NE Tagger Multi-language NE Tagger Conceptual clustering Conceptual clustering

52 Maximum Entropy Machine Learning approach to classification: Machine Learning approach to classification: –System trained on test cases –Learned model used for predictions Classification problem described as a number of features Classification problem described as a number of features Each feature corresponds to a constraint on the model Each feature corresponds to a constraint on the model Maximum entropy model: the model with the maximum entropy of all the models that satisfy the constraints Maximum entropy model: the model with the maximum entropy of all the models that satisfy the constraints Choosing a model with less entropy, would add ‘information’ constraints not justified by the empirical evidence available Choosing a model with less entropy, would add ‘information’ constraints not justified by the empirical evidence available

53 MaxEntropy: example data FeaturesOutcome Sunny, HappyOutdoor Sunny, Happy, DryOutdoor Sunny, Happy, HumidOutdoor Sunny, Sad, DryOutdoor Sunny, Sad, HumidOutdoor Cloudy, Happy, HumidOutdoor Cloudy, Happy, HumidOutdoor Cloudy, Sad, HumidOutdoor Cloudy, Sad, HumidOutdoor Rainy, Happy, HumidIndoor Rainy, Happy, DryIndoor Rainy, Sad, DryIndoor Rainy, Sad, HumidIndoor Cloudy, Sad, HumidIndoor Cloudy, Sad, HumidIndoor

54 MaxEnt: example predictions ContextOutdoorIndoor Cloudy, Happy, Humid Rainy, Sad, Humid

55 MaxEntropy: application Sentence Splitting Sentence Splitting Not all punctuations are sentence boundaries: Not all punctuations are sentence boundaries: –U.S.A. –St. Helen –3.14 Use features like: Use features like: –Capitalization (previous, next word) –Present in abbreviation list –Suffix/prefix digits –Suffix/prefix long Precision: > 95% Precision: > 95%

56 Part of Speech Tagging TreeTagger: statistic package based on HMM and decision trees TreeTagger: statistic package based on HMM and decision trees Trained on manually tagged text Trained on manually tagged text Full language lexicon (with all inflections: words for Italian) Full language lexicon (with all inflections: words for Italian)

57 Training Corpus IlDET:def:*:*:masc:sg_il presidenteNOM:*:*:*:masc:sg_presidente dellaPRE:det:*:*:femi:sg_del RepubblicaNOM:*:*:*:femi:sg_repubblica franceseADJ:*:*:*:femi:sg_francese FrancoisNPR:*:*:*:*:*_Francois MitterrandNPR:*:*:*:*:*_Mitterrand haVER:aux:pres:3:*:sg_avere propostoVER:*:pper:*:masc:sg_proporre …

58 Named Entity Tagger Uses MaxEntropy Uses MaxEntropy NE categories: NE categories: –Top level: NAME, ORGANIZATION, LOCATION, QUANTITY, TIME, EVENT, PRODUCT –Second level: E.g. QUANTITY: MONEY, CARDINAL, PERCENT, MEASURE, VOLUME, AGE, WEIGHT, SPEED, TEMPERATURE, ETC. See resources at CoNLL (cnts.uia.ac.be/connl2004) See resources at CoNLL (cnts.uia.ac.be/connl2004)

59 NE Features Feature types: Feature types: –word-level (es. capitalization, digits, etc.) –punctuation –POS tag –Category designator (Mr, Av.) –Category suffix (center, museum, street, etc.) –Lowercase intermediate terms (of, de, in) –presence in controlled dictionaries (locations, people, organizations) Context: words in position -1, 0, +1 Context: words in position -1, 0, +1

60 Sample training document Today the Dow Jones industrial average gained thirtyeight and three quarter points. When the first American style burger joint opened in London 's fashionable Regent street some twenty years ago, it was mobbed. Now it's Asia 's turn. The temperatures hover in the nineties, the heat index climbs into the hundreds. And that's continued bad news for Florida where wildfires have charred nearly three hundred square miles in the last month and destroyed more than a hundred homes.

61 Clustering Classification: assign an item to one among a given set of classes Classification: assign an item to one among a given set of classes Clustering: find groupings of similar items (i.e. generate the classes) Clustering: find groupings of similar items (i.e. generate the classes)

62 Conceptual Clustering of results Similar to Vivisimo Similar to Vivisimo –Built on the fly rather than from –Predefined categories (Northern Light) Generalized suffix tree of snippets Generalized suffix tree of snippets Stemming Stemming Stop words (articulated, essential) Stop words (articulated, essential) Demo: python, upnp Demo: python, upnppythonupnppythonupnp

63 PiQASso: Pisa Question Answering System “Computers are useless, they can only give answers” Pablo Picasso

64 PiQASso Architecture Sentence Splitter Sentence Splitter Indexer Query Formulation /Expansion Query Formulation /Expansion WordNet MiniPar ? Document collection MiniPar Type Matching Type Matching Relation Matching Relation Matching Answer Pars Answer Scoring Answer Scoring Popularity Ranking Popularity Ranking Answe r found? Answer Question analysis Answer analysis WNSense Question Classification Question Classification

65 Linguistic tools extracts lexical knowledge from WordNet classifies words according to WordNet top-level categories, weighting its senses computes distance between words based on is-a links suggests word alternatives for query expansion What metal has the highest melting point? subj lex-mod obj mod WNSenseMinipar [D. Lin] Example: Theatre Categorization: artifact 0.60, communication 0.40 Synonyms: dramaturgy, theater, house, dramatics Identifies dependency relations between words (e.g. subject, object, modifiers) Provides POS tagging Detects semantic types of words (e.g. location, person, organization) Extensible: we integrated a Maximum Entropy based Named Entity Tagger

66 Question Analysis What metal has the highest melting point? metal, highest, melting, point 2. Keyword extraction 1. Parsing 3. Answer type detection SUBSTANCE 4. Relation extraction 1.NL question is parsed 2.POS tags are used to select search keywords 3.Expected answer type is determined applying heuristic rules to the dependency tree 4.Additional relations are inferred and the answer entity is identified What metal has the highest melting point? subj lex-mod obj mod

67 Answer Analysis Tungsten is a very dense material and has the highest melting point of any metal. 1 Parsing …………. 2 Answer type check 3 Relation extraction SUBSTANCE … 4 Matching Distance Tungsten 6 Popularity Ranking ANSWER 1.Parse retrieved paragraphs 2.Paragraphs not containing an entity of the expected type are discarded 3.Dependency relations are extracted from Minipar output 4.Matching distance between word relations in question and answer is computed 5.Too distant paragraphs are filtered out 6.Popularity rank used to weight distances 5 Distance Filtering

68 Match Distance between Question and Answer Analyze relations between corresponding words considering: number of matching words in question and in answer number of matching words in question and in answer distance between words. Ex: moon matching with satellite distance between words. Ex: moon matching with satellite relation types. Ex: words in the question related by subj while the matching words in the answer related by pred relation types. Ex: words in the question related by subj while the matching words in the answer related by pred

69

70 Improving PIQASso

71 More NLP NLP techniques largely unsuccessful at information retrieval NLP techniques largely unsuccessful at information retrieval –Document retrieval as primary measure of information retrieval success Document retrieval reduces the need for NLP techniques –Discourse factors can be ignored –Query words perform word-sense disambiguation –Lack of robustness: NLP techniques are typically not as robust as word indexing

72 How these technologies help? Question Analysis Question Analysis –The tag of the predicted category is added to the query Named-Entity Detection: Named-Entity Detection: –The NE categories found in text are included as tags in the index What party is John Kerry in? (ORGANIZATION) John Kerry defeated John Edwards in the primaries for the Democratic Party. Tags: PERSON, ORGANIZATION

73 NLP Technologies Coreference Relations: Coreference Relations: –Interpretation of a paragraph may depend on the context in which it occurs Description Extraction: Description Extraction: –Appositive and predicate nominative constructions provide descriptive terms about entities

74 Represented as annotations associated to words, i.e. words in the same position as the reference Represented as annotations associated to words, i.e. words in the same position as the reference Coreference Relations How long was Margaret Thatcher the prime minister? (DURATION) The truth, which has been added to over each of her 11 1/2 years in power, is that they don't make many like her anymore. Tags: DURATION Colocated: her, MARGARET THATCHER

75 Description Extraction Identifies DESCRIPTION category Identifies DESCRIPTION category Allows descriptive terms to be used in term expansion Allows descriptive terms to be used in term expansion Famed architect Frank Gary… Tags: DESCRIPTION, PERSON, LOCATION Buildings he designed include the Guggenheim Museum in Bilbao. Colocation: he, FRANK GARY Who is Frank Gary? (DESCRIPTION) What architect designed the Guggenheim Museum in Bilbao? (PERSON)

76 NLP Technologies Question Analysis: Question Analysis: –identify the semantic type of the expected answer implicit in the query Named-Entity Detection: Named-Entity Detection: –determine the semantic type of proper nouns and numeric amounts in text

77 Will it work? Will these semantic relations improve paragraph retrieval? Will these semantic relations improve paragraph retrieval? –Are the implementations robust enough to see a benefit across large document collections and question sets? –Are there enough questions where these relationships are required to find an answer? Hopefully yes! Hopefully yes!

78 Preprocessing Paragraph Detection Paragraph Detection Sentence Detection Sentence Detection Tokenization Tokenization POS Tagging POS Tagging NP-Chunking NP-Chunking

79 Queries to a NE enhanced index text matches bush text matches PERSON:bush text matches LOCATION:* & PERSON: bin-laden text matches DURATION:* PERSON:margaret-thatcher prime- minister

80 Coreference Task: Task: –Determine space of entity extents: Basal noun phrases: –Named entities consisting of multiple basal noun phrases are treated as a single entity Pre-nominal proper nouns Possessive pronouns –Determine which extents refer to the same entity in the world

81 Paragraph Retrieval Indexing: Indexing: –add NE tags for each NE category present in the text –add coreference relationships –Use syntactically-based categorical relations to create a DESCRIPTION category for term expansion –Use IXE passage indexer

82 High Composability DocInfo PassageDoc Collection name date size name date size text boundaries text boundaries QueryCursor PassageQueryCursor next() Cursor next()

83 Tagged Documents QueryCursor QueryCursorTaggedWord QueryCursorWord select documents where select documents where –text matches bush –text matches PERSON:bush –text matches osama & LOCATION:*

84 Combination Searching passages on a collection of tagged documents Searching passages on a collection of tagged documents PassageQueryCursor > QueryCursor

85 Paragraph Retrieval Retrieval: Retrieval: –Use question analysis component to predict answer category and append it to the question –Evaluate using TREC questions and answer patterns 500 questions

86 System Overview NE Recognizer Coreference Resolution Documents IXE Search Question Analysis Question Paragraphs Description Extraction Paragraphs+ Sent. Splitter POS tagger Paragr. Splitter Tokenization IXE indexer IndexingRetrieval

87 Conclusion QA is a challenging task QA is a challenging task Involves state of the art techniques in various fields: Involves state of the art techniques in various fields: –IR –NLP –AI –Managing large data sets –Advanced Software Technologies


Download ppt "Search Engines & Question Answering Giuseppe Attardi Dipartimento di Informatica Università di Pisa."

Similar presentations


Ads by Google