DYNAMIC ELEMENT RETRIEVAL IN A STRUCTURED ENVIRONMENT MAYURI UMRANIKAR

CONTENTS Introduction Retrieval Environment - The Vector Space Model - INEX Environment - Flexible Retrieval System Method Used for Retrieval - Document Tree – Construction - Ranking of Elements - Output Experiments Conclusions

INTRODUCTION Extensible Markup Language (XML) preferred for representing documents and due to increase of documents, issue of element retrieval arises Focus on retrieval of relevant elements rather than entire document INEX – INitiative for Evaluation of XML Retrieval Flexible Mechanisms Different Approaches Term Weighting

RETRIEVAL ENVIRONMENT 2 Factors – Issues when focus moves from documents to components and Salton’s Vector Space Model Vector Space Model – Weight number of times a term occurs in the document Fox’s Extended Vector Space Model – Incorporation of objective identifiers Document vector consists of subvectors Contain text independently indexed, weighted, searched and retrieved Term Weighting – weighting within subjective vectors Smart Experimental Retrieval System

INEX ENVIRONMENT Content Only (CO) –ignore document structure, like typical queries, specify only content of search Content and Structure (CAS) – explicitly refer to structure, exhaustive and specific CO query directly to user, CAS additional filtering and search of body portion CAS returns rank ordered list of elements INEX-EVAL – uses measures of recall and precision ( fig, exhaustivity, specificity mapped to a single relevance) results are ranked

FLEXIBLE RETRIEVAL SYSTEM Smart Format – documents and topics translated, indexed as extended vectors Subjective vectors – contain content bearing terms Objective vectors – serve as filters on result returned by CAS queries Extended vector – subjective vector, terms having a paragraph in body subvector Lnu-ltu weighting Dynamic flexible retrieval- tree representation, rank ordered list by lnu weights

METHOD FOR FLEXIBLE RETRIEVAL Input – Query Q given and paragraph, retrieve rank ordered list, terminal modes N top ranked paragraphs as input selected Set of paragraphs used to identify documents – elements generated and returned as output Document Tree – Needs information of structure Terminal nodes Pre-order traversal Terminal nodes found in paragraph index

SIMPLE XML DOCUMENT AND ITS SCHEMA

CONSTRUCTION OF DOCUMENT TREE For query Q, n top ranked paras used to build trees Leaf elements or terminal nodes - paragraph nodes Each leaf represented by term-freq weighted frequency vector 1 st – gather all leaf nodes, terminal nodes done 2 nd – merge children vectors for parents Document schema determine merging Parent – unique terms of children, term –freq weighted parent vector( has content of children) Process in recursive manner done

RANKING OF ELEMENTS Set of elements of document tree generated Problem- structured retrieval; rank ordered list of elements Method used – All-element index( separate representation for each element of each document and weighting information) Lnu weights - elements variable length, do not require global frequency Normalization and length – failing results in biased values Pivot – document length probability of relevance= probability of retrieval Slope- amount of tilting Pivoted Normalization – reduces difference Lnu term weights: ((1+log(term_freq))/ (1+log(avg_term_freq)))/((1- slope)+slope*((no_unique_terms)/pivot)

Ltu weighting – N collection size, nk no of elements ((1+log(term_freq))/log(N/nk))/ ((1-slope)+slope*(no_unique_terms)/pivot)) N,nk element dependent, should be known through indexing We move up; N – count elements of each type Nk – inverted file entry in paragraph index, mapping identifiers and xpaths (given)

OUTPUT OF FLEXIBLE RETRIEVAL Select another leaf node, gather siblings, construct document tree, calculate Lnu term weights, Ltu weighted query; produce another rank ordered list After n top ranked exhausted, last list produced, merge lists Single set of elements rank ordered – correlation Q Comparison – flexible retrieval & all-element index identical – set of n paragraphs i/p to flexible retrieval have all paragraphs same values used for Lnu-ltu

ALGORITHM

EXPERIMENTS Paragraph – result; set of extended vectors representing paragraph CO – subvector represents subjective portion, body subvector important (content of element and not type) contained in body Tree Representation

FACTORS OF INTEREST Slope, pivot for Lnu-ltu Effective structure retrieval Can be determined – empirically, applied from one collection to other; Generic N- no of paragraphs input, sets upper bound on number per query Actual trees depend on number of paragraphs having same group or same document

EXPERIMENTS DONE All-element and dynamic/flexible retrieval experiments and results - body-only retrieval Correlation between element and query vector produced – correlation of body elements only Table 1

RESULTS Tables

Result equivalent Flexible more efficient – file space Time required for indexing is half Dynamic- Per query basis cost more – n; total trees not exact required specified Another factor – value of nk

DISCUSSIONS AND CONCLUSIONS Flexible retrieval dynamically, rank ordered list of elements, single indexing at level - basic indexing node (paragraph) Basic functions- SMART; extended vector model Results – flexible capabilities Attempt to incorporate other subvectors, internal node, weight INEX – exhaustivity and specificity; results exhaustive; specificity research going on; results are reflection It is the better way of retrieval than all-indexing

THANK YOU!!!