1. CS 430: Information Discovery
Lecture 9
Vector Methods

2. Course Administration
• No discussion class tomorrow
• Assignment 2 will be posted shortly
• Please hand in laptop questionnaires

3. SMART System
An experimental system for automatic information retrieval
• automatic indexing to assign terms to documents and queries
• collect related documents into common subject classes
• identify documents to be retrieved by calculating
similarities between documents and queries
• procedures for producing an improved search query
based on information obtained from earlier searches
Gerald Salton and colleagues
Harvard
Cornell

4. Vector Space Methods
Problem: Given two text documents, how similar are they?
(One document may be a query.)
Vector space methods that measure similarity do not assume exact matches.
Benefits of similarity measures rather than exact matches
Encourage long queries, which are rich in information. An abstract should be very similar to its source document.
Accept probabilistic aspects of writing and searching. Different words will be used if an author writes the same document twice.

5. Vector space revision
x = (x1, x2, x3, ..., xn) is a vector in an n-dimensional vector space
Length of x is given by (extension of Pythagoras's theorem)
|x|2 = x12 + x22 + x xn2
If x1 and x2 are vectors:
Inner product (or dot product) is given by
x1.x2 = x11x21 + x12x22 + x13x x1nx1n
Cosine of the angle between the vectors x1 and x2:
cos () =
x1.x2 |x1| |x2|

6. Vector Space Methods: Concept
n-dimensional space, where n is the total number of different terms used to index a set of documents.
Each document is represented by a vector, with magnitude in each dimension equal to the (weighted) number of times that the corresponding term appears in the document.
Similarity between two documents is the angle between their vectors.

7. Three terms represented in 3 dimensions t1

8. Example 1: Incidence array
terms in d1 -> ant ant bee
terms in d2 -> bee hog ant dog
terms in d3 -> cat gnu dog eel fox
terms ant bee cat dog eel fox gnu hog length
d 2
d 4
d 5
Weights: tij = 1 if document i contains term j and zero otherwise

9. Example 1 (continued) Similarity of documents in example: d1 d2 d3
Similarity measures the occurrences of terms, but no other characteristics of the documents.

10. Example 2: frequency array
terms in d1 -> ant ant bee
terms in d2 -> bee hog ant dog
terms in d3 -> cat gnu dog eel fox
ant bee cat dog eel fox gnu hog length
d 5
d 4
d 5
Weights: tij = frequency that term j occurs in document i

11. Example 2 (continued) Similarity of documents in example: d1 d2 d3
Similarity depends upon the weights given to the terms.

12. Vector similarity computation
Documents in a collection are assigned terms from a set of n terms
The term assignment array T is defined as
if term j does not occur in document i, tij = 0
if term j occurs in document i, tij is greater than zero
(the value of tij is called the weight of term j in document i)
Similarity between di and dj is defined as
 tiktjk
|di| |dj| n
cos(di, dj) =
k=1

13. Simple use of vector similarity
Threshold
For query q, retrieve all documents with similarity more than 0.50
Ranking
For query q, return the n most similar documents ranked in order of similarity

14. Contrast with Boolean searching
With Boolean retrieval, a document either matches a query exactly or not at all
• Encourages short queries
• Requires precise choice of index terms
• Requires precise formulation of queries (professional training)
With retrieval using similarity measures, similarities range from 0 to 1 for all documents
• Encourages long queries to have as many dimensions as possible
• Benefits from large numbers of index terms
• Benefits from queries with many terms, not all of which need
match the document

15. Document vectors as points on a surface
• Normalize all document vectors to be of length 1
• Then the ends of the vectors all lie on a surface
with unit radius
• For similar documents, we can represent parts of
this surface as a flat region
• Similar document are represented as points that are
close together on this surface

16. Relevance feedback (concept)
hits from original search x x o  x x  o o
x documents identified as non-relevant
o documents identified as relevant
 original query
reformulated query 

17. Document clustering (concept)x x x x x x x x x x x x x x x x x x x
Document clusters are a form of automatic classification.
A document may be in several clusters.