Presentation is loading. Please wait.

Presentation is loading. Please wait.

Document Summarization

Similar presentations


Presentation on theme: "Document Summarization"— Presentation transcript:

1 Document Summarization
Vinayak Gagrani Neeraj Toshniwal Abhishek Kabra Guide Pushpak Bhattacharya

2 Outline Introduction Single Document Summarization
Multiple Document Summarization Application Evaluation Conclusion

3 Introduction What is Summary? Text produced from one or more texts
Conveys important information in the original texts, and that is no longer than half of the original texts. 3 important aspects of summary are: Summaries should be short Summaries should preserve important information Summaries may be produced from single/multiple documents

4 Common terms in summarization dialect
Extraction Procedure of identifying important sections of text and producing verbatim Abstraction Aim to produce material in a new way Fusion Combining extracted parts coherently Compression Aims at throwing out unimportant sections of text

5 Single Document Summarization
Early Works Machine Learning Methods Naïve-Bayes Methods Rich Features and Decision Trees Deep Natural Language Analysis Methods Lexical Chaining Rhetorical Structure Theory (RST)

6 Early Works Luhn, 1958 Edmundson, 1969
Summarization based on measuring significance of words depending on its frequency Deriving significance factor of sentence, based on number of significance words in that sentence Edmundson, 1969 Word frequency and positional importance were incorporated Presence of cue words, and skeleton of the document were also incorporated

7 Naïve Bayes Method Classifier based on applying Bayes theorem with strong independence assumption s-particular sentence S-set of sentences that make up the summary F1…, Fk -the features Assuming independence of features: P(s ε S | F1,F2….Fk)= 𝑖=1 𝑘 P(Fi |s ∈ S). P(s ∈ S) 𝑖=1 𝑘 P(Fi ) Evaluation is done by analyzing its match with the human extracted document summary

8 Naïve Bayes Method Term frequency-inverse document frequency
Increases proportionally to the number of times a word appears in the document offset by the frequency of the word in the corpus Takes into account that certain words are more common than others. For e.g.. “the”, “is” etc. Idf(t,D)= log |𝐷| | 𝑑∈𝐷:𝑡∈𝑑 | |D|: total number of documents in the corpus | 𝑑∈𝐷:𝑡∈𝑑 |: number of documents where the term t appears i.e. tf(t,d) ≠ 0

9 Rich Features and Decision Trees
Weighing sentences based on their position Arises from the idea that texts generally follow a predictable discourse structure Sentence position yield was calculated against the topic keywords later Sentence position were then ranked by average yield to produce Optimal Position Policy for topic positions for the genre Later, sentence extraction problem was modeled using decision trees assumption that features are independent broke away

10 Deep Natural Language Analysis Methods
Techniques aimed at modeling the text’s discourse structure Use of heuristics to create document extracts Lexical Chaining independent of the grammatical structure of the text list of words that captures a portion of the cohesive structure of the text sequence of related words in the text, spanning short or long distances technique used to identify the central theme of a document

11 Forms of Cohesion Ellipsis Substitution
Words are omitted when the phrase needs to be repeated Example: A: Where are you going? B: To town. Substitution Word is not omitted but replaced by another A: Which ice-cream would you like? B: I would like the pink one.

12 Forms of Cohesion Conjunction Repetition Reference
Relationship between two clauses Few of them are: “and”, “then”, “however” etc. Repetition Mentioning of the same word again Reference Anaphoric reference Refers to someone/something that has been previously identified Cataphoric reference Forward referencing . Example: Here he comes….It’s Brad Pitt

13 Lexical chaining Example:- John had mud pie for dessert. Mud pie is made of chocolate. John really enjoyed it. Steps involved in lexical chaining: a) Selecting a set of candidate words. b) For each candidate word, finding an appropriate chain relying on a relatedness criterion among members of the chain c) If it is found, inserting the word in the chain and updating it accordingly

14 Lexical Chaining relatedness measure-Wordnet Distance.
Weights assigned to chains based on their length and homogeneity Determining the strength of a lexical chain by taking in consideration the distribution of elements in the chain throughout the text Corresponds to the significance of the textual context it embodies. Provides a basis identifying the topical units in a document which are of great importance in document summarization.

15 Rhetorical Structure Theory(RST)
two non-overlapping pieces of text spans: the nucleus and the satellite Nuclei expresses what is more essential to the writer's purpose than the satellite Example: claim followed by evidence for the claim. RST posits an "Evidence" relation between the two spans. claim is more essential to the text than the particular evidence claim span a nucleus and the evidence span a satellite Nucleus is independent of the satellite but not vice versa

16 Rhetorical Structure Theory(RST)

17 Multiple Document Summarization
Need and Encouragement Extraction of single summary from multiple documents started in mid 1990s Most of the application in news article Google news (news.google.com) Columbia news blaster (newsblaster.cs.columbia.edu) News in Essence (NewsInEssence.com) Multiple source of information which are :- supplementary to each other overlapping in content even contradictory at time

18 Early Work Extended template driven message understanding system
Abstractive System, rely heavily on internal NLP tools Earlier considered as knowledge of Language Interpretation Generation Extractive Techniques have been applied - Similarity measures between sentences identify common theme through clustering - select one sentence to represent each cluster generate composite sentence from each cluster Summarization differs on what the final goal is MEAD : works based on extraction techniques on general domains SUMMONS : build a briefing highlighting difference and updates on news report

19 Abstractions and Information Fusion
SUMMONS is the first example of multi-document summarization Considers event about a narrow domain news articles about terrorism It produces a briefing merging relevant information about event and their evolution over time It reads a database built by template based message understanding system Concatenation of two systems : Content Planner and Linguistic Generator

20 SUMMONS - processing the text (Content Planner)
Content Planner : selects information to include in summary through combination of input templates It uses summary operators - set of heuristics that perform operations like : change of perspective, contradiction, refinement Linguistic Generator :selects the right words to express the information in grammatical and coherent text. Uses connective phrases to synthesize summary, adapting language generation tools like FUF/SURGE

21 Theme based approach - McKeown et al., Barzilay et al.
Themes - set of similar text units (Paragraphs) - Clustering Problem Text is mapped to vector of features including single words weighted by their TF-IDF scores, noun, pronoun, semantic classes of verbs For each pair of paragraphs a vector is computed which represents matches on different features. Decision rules learnt from data classify each pair as similar or dissimilar. An algorithm then places the most related paragraphs in same theme Information Fusion - which sentences of the theme should be included in the final summary.

22 Information Fusion Algorithm - compares and intersects predicate argument structures of the phrases within each theme to find which are repeated often enough to be included in summary Sentenced are parsed using Collins' statistical parser converted into dependency tree – captures predicate- argument structure, identify functional roles. Comparison algorithm traverses the tree recursively, adding identical nodes to output tree. Once full phrase are found, they are marked to be included in summary. Once summary content is decided, a grammatical text is generated using FUF/SURGE language generating system.

23 “McVeigh, 27,was charged with the bombing”
Decision Tree “McVeigh, 27,was charged with the bombing”

24 Topic-Driven Summarization
MMR - Maximal Marginal Relevance introduced by Carbonell and Goldstein Rewards relevant sentences and penalizes redundant ones by considering a linear combination of two similarity measures. Q - query or user profile, R - Ranked list of documents, S - already selected documents . Select a document one at a time and add them to S. For each document in Di in R\S, MR(Di) = a * Sim1(Di,Q) - (1-a) * max Di in S Sim2(Di,Dj), where a lies in [0,1] Document getting maximum MR(Di) is selected until maximum number is reached or threshold is reached, a controls the relative importance between relevance and redundancy. Sim1 and Sim2 are similarity measures ( cosine similarity measure )

25 Graph Spreading Activation
Content is denoted as entities and relations as nodes and edges of a graph.   Rather than extracting sentences, they detect salient regions of the graph.  Topic Driven : topic is denoted by entry nodes in graph. Graph : Each node is single occurrence of word.   Different kind of links – Adjacency links, Same links, Alpha Links and Phrase links, Name and Coref Links 

26 Graph Spreading Activation
Topic nodes are identified through stem comparison and marked as entry node.   Spreading activation: search for semantically related text is propagated from these to other nodes of the graph.  Weight of neighboring node depends on node links traveled and is exponentially decaying function of the distance.  Pair of document graph: identify common nodes and difference nodes. Highlight sentences having higher common and different scores.  User is able to specify the maximal number to control the output. 

27 Centroid-based Summarization
It does not use any language generation module. Easily scalable and domain-independent Topic Detection - Group together news articles that describe the same event. An agglomerative clustering algorithm is used, operates on TF- IDF vector representations, successively adding documents to clusters and re computing the centroids according to 𝑐𝑗= 𝑑 ∈ 𝐶𝑗 𝑑~ 𝐶𝑗 cj is the centroid of the j-th cluster, Cj the set of documents that belong to that cluster Centroids can thus be considered as pseudo-documents that include those words whose TF-IDF scores are above a threshold in their cluster.

28 Centroid-based Summarization
Second Stage - Identify sentences that are central to topic of the entire cluster. Two metrics similar to MMR(but not query dependent) are defined by Radev et al., 2000 Cluster-based relative utility (CBRU) - how relevant a particular sentence to general topic of cluster Cross-sentence Informational subsumption (CSIS) - measure of redundancy among sentences Given a cluster segmented into n sentences, and compression rate R, we select nR sentences in order of appearance in chronologically arranged documents Addition of the three scores minus redundancy penalty(Rs) for sentence that overlaps highly ranked sentence is the final score for each sentence Centroid Value (Ci) sum of centroid values of all the words in sentence Positional Value(Pi) makes leading sentences more important First sentence Overlap (Fi) - inner product of word occurrence vector of sentence I and that of 1st sentence of document

29 Application Google News: Ultimate research Assistant:
news aggregator, selecting most up-to-date(within the past 30 days) information from thousands of publications by an automatic aggregation algorithm Different versions available for more than 60 regions in 28 languages Ultimate research Assistant: performs text mining on Internet search results make it easier for the user to perform online research by organizing the output. Type name of a topic and it will search the web for highly relevant resources, and organize the search results

30 Application Shablast iResearch Reporter – Universal search engine
Produces multi-document summaries from the top 50 results returned by Microsoft's Bing search engine for a set of keywords. iResearch Reporter – Commercial Text Extraction and Text Summarization system Produces categorized, easily-readable natural language summary reports covering multiple documents retrieved by entering user query in google search engine

31 Application

32 Evaluation A difficult task
Absence of a standard human or automatic evaluation metric makes difficult to compare different systems and establish a baseline Manual evaluation not feasible Need for an evaluation metric having high correlation with human scores human and automatic evaluation: Comparison of automatic generated summaries with manually written "ideal" summaries decomposition of text into sentences Rating between 1-4 to system unit(SU) which shares content with Model unit(MU) corresponding to ideal summaries

33 Evaluation ROUGE Information-theoretic Evaluation of Summaries
based only on content overlap can determine if the same general concepts are discussed between an automatic summary and a reference summary cannot determine if the result is coherent or the sentences flow together in a sensible manner Better in case of single document summarization Information-theoretic Evaluation of Summaries Central idea is to use a divergence measure between a pair of probability distributions First distribution is derived from automatic summary Second from a set of reference summaries Suits both the single document and multi document summarization scenarios

34 Conclusion Need to develop efficient and accurate summarization systems due to enormous rate of information growth Still a lot of research going on this field especially in evaluation techniques Multi document summarization is more in use as compared to single-document summarization Extractive techniques are employed usually rather than abstractive techniques as they are easy to employ and have produced satisfactory results

35 References A survey on Automatic Summarization – Dipanjan Das and Andre F.T. Martins (http://www.cs.cmu.edu/~afm/Home_files/Das_Marti ns_survey_summarization.pdf) Wikipedia Relevance of cluster size in MMR Based summarizer (http://www.cs.cmu.edu/~madhavi/publications/Gan apathiraju_11-742Report.pdf)


Download ppt "Document Summarization"

Similar presentations


Ads by Google