Query Languages

Information Retrieval Concerned with the: Representation of Storage of Organization of, and Access to Information items.

Recap 1 Important points –Data retrieval vs. information retrieval –Users specify their needs via an intermediary language. –Documents are represented by an abstraction of their content. –Traditional model vs. berry-picking model –Evaluation (precision/recall, single-value measures, human measures) –Task characteristics (question answering, open- ended analysis, ad-hoc vs. filtering) –Collection characteristics (size, document relations)

Recap 2 Important points –Content types (text, descriptive/semantic metadata, multimedia) –Metadata (formats & sets) –Information Theory (entropy) –Models of symbol distribution (Zipf’s law, Heap’s law) –Distance measures (Hamming distance, Levenshtein distance) –Markup languages (SGML, HTML, XML) –Multimedia formats (header + data)

Query Languages Query language determines which queries can be formulated –User-oriented languages –System-oriented languages (protocols) Language dependent on underlying information retrieval model Systems may enhance the query using –Word expansion (thesaurus & stemming) –removing stopwords

Keyword-Based Querying A query is composed of keywords –Documents containing keywords are returned –Intuitive, easy to express, fast ranking –Single-word and multi-word queries Classification of keyword-based queries –single-word queries –context queries –Boolean queries –natural language queries

Single-Word Queries Word query –the most elementary query that can be formulated –a word is a sequence of letters surrounded by separators –in many models, words are only types of queries allowed Result of word queries –the set of documents containing at least one of the words of the query –the resulting documents are ranked according to a degree of similarity to the query use term frequency and inverse document frequency

Context Queries Phrase query –a sequence of single-word queries –ignore separators and uninteresting words example: “…enhance the retrieval…” –ranked in a fashion analogous to single words Proximity query –a phrase query with a maximum allowed distance (character or word) between words in the query example: distance = 4 “enhance the power of retrieval” –physical proximity has semantic value: the words in the same paragraph are related in some way

Boolean Queries Boolean query –composed of atoms (basic queries) that retrieve documents, and of Boolean operators which work on their operands (sets of documents) Query syntax tree –compositional scheme –leaves: basic queries –internal nodes: operators AND OR translation syntaxsyntactic

Boolean Queries Operators in Boolean queries –e1 OR e2 : selecting all docs satisfying e1 or e2 –e1 AND e2 : selecting all docs satisfying both e1 and e2 –e1 BUT e2 : selecting all docs satisfying e1 but not e2 Classic Boolean system –no ranking of the retrieved docs –does not allow partial matching –alternative: fuzzy Boolean set of operators meaning of AND and OR can be relaxed (e.g., appearing in some operands)

Natural Language Natural language query –blurring the distinction between AND and OR → query becomes an enumeration of words and context queries –higher ranking is assigned to those documents matching more parts of the query Characteristics –retrieving all the documents close to the query –a complete document can be used as a query → leads to the use of relevance feedback techniques (user selects a document from the result, and submits it as a new query) –example system: AskJeeves

Query Languages: Patterns & Structures

Pattern Matching Pattern –a set of syntactic features that must occur in a text segment Types of patterns –Words: string (sequence of characters) in the text –Prefixes: string forming the beginning of a text word (e.g., comput → computer, computation) –Suffixes: string forming the termination of a text word (e.g., ters → computers, painters) –Substrings: string appeared within a text word (allowed word separators) (e.g., tal → talk, metallic & any flow → many flowers)

Pattern Matching (cont.) More Types of Patterns –Ranges: a pair of strings matched any word lying between them in lexicographical order (e.g., held to hold → hoax, hissing) –Allowing errors: retrieving word similar to given word (e.g., flower → flo wer [edit distance = 1]) –Regular expressions: general patterns built up by simple strings and operators (e.g., pro (blem | tein) (s | ε) (0 | 1 | 2)* ) → Problem02, proteins –Extended patterns: classes of characters, conditional expressions, wild characters, combinations

Structural Queries Structural query –mixing contents and structure in queries content constraints (words, phrases, patterns) structural constraints (containment, proximity) and restrictions on structural elements (chapters, sections) Type of structures of text –form-like fixed structure –hypertext structure –hierarchical structure

Structural Queries (cont.) Types of structures formhypertexthierarchical

Fixed Structure Traditional restrictions –documents had a fixed set of fields –each field had some text inside –only rarely the fields appear in any order or repeat –fields were not allowed to nest or overlap –retrieval: specifying a given basic pattern to be found only in a given field Characteristics –reasonable to retrieve text collection having a fixed structure (e.g. mail archive) → inadequate to represent the hierarchical structure such as HTML docs –expansion to relational DB model

Hypertext Hypertext (navigational) –a directed graph where the nodes hold some text and the links represent connections between nodes or between positions inside the nodes Browsing / Searching in hypertext –retrieval from a hypertext: browsing (traversing the hypertext nodes following link → navigational activity) –even in web, one can search by the text contents of the nodes, but not by their structural connectivity –some search engines now allow searching for specific source or destination anchors (but not general structure + content queries)

Hierarchical Structure Hierarchical structure –an intermediate structuring model lying between fixed structure and hypertext –represents a recursive decomposition of the text –a natural model for many text collections, e.g., books, articles, legal documents, structured programs, etc. Hierarchical models –PAT Expressions, Overlapped Lists, List of References, Proximal Nodes, Tree Matching Issues in hierarchical models –static or dynamic structure, restrictions on the structure, integration with text, query language

Query Protocols Query protocols –query language used to query text database –standards intended not for human use but for querying library systems and querying CD-ROMs Some important query protocols –Z39.50 –WAIS –CCL –CD-RDx –SFQL