Adversarial Information System Tanay Tandon Web Enhanced Information Management April 5th, 2011

Agenda What is Adversarial Information Retrieval? What is the goal of Adversarial Information Retrieval? Issues with First generation engines Resolution Future Improvements

What is Adversarial IR? Gathering, Indexing, Retrieving and Ranking Information. Subset of the information has been manipulated maliciously. Financial Gain.

What is the Goal of Adversarial IR? Detect bad/unauthentic sites. Improve precision on search engines by eliminating such unauthentic sites.

Issues with First Generation Engines What is Term Frequency? First generation engines relied heavily on "term frequency" to determine the page rank. Increase the page rank by repeating the same word over and over again.

Search Engine Spamming Link Spam Link-bombing Spam Blogs Comment Spam Keyword Spam Malicious Tagging

Google Bombing

Trust Rank Observation Good Pages tend to link good pages. Algorithm -- Select a small subset of pages and let a human classify them -- Propagate the goodness of pages.

Propagation Trust function T -- T(p) returns the probability that page p is good page. Initial values -- T(p) = 1, if p was found to be a good page. -- T(p) = 0, if p was found to be a spam page. Iterations: -- Propagate trust following out-links. -- only a fixed number of iterations M.

Propagation Issues and Resolution Problems with propagation – Pages reachable from good seeds might not be good. – the further away we are from good seed pages, the less certain we are that a page is good. Solution -- Reduce trust as we move further away from the good seed pages (trust attenuation).

Hubs and Authorities A hub is good if it belongs to good authority. An authority is good if good hubs point to it. Weights given to pages must keep track of the authenticity.

Further Improvements Seed Weighting Instead of assigning equal weights to each seed assign a weight proportional to its quality/importance. Seed Filtering Filtering out low quality pages that may exist in topic directories. Finer Topics Lower Layer of Topic Directories