The influence of search engines on preferential attachment Dan Li CS3150 Spring 2006

The paper The influence of search engines on preferential attachment Soumen Chakrabarti, Alan Frieze and Juan Vera

Background The evolution of social networks through time Web graph Models Preferential Attachment Copying Model

Background Evolution of the Web Power-law Preferential attachment( Barabasi and Albert) Copying Model  The author of a newborn page u picks a random reference page v from the web, and with some probability, copies out- links from v to u.  Power-law: power ~ 2 Organic Evolution NO POWERFUL CENTRAL ENTIRY!

The New Problem How the page authors find existing pages and create links to them? Highly popular search engines limit the attention of the page authors to a small set of celebrity pages. Page authors frequently use search engines to locate pages, and include the HOT pages they visit (with probability p)

The New Problem The evolution of the Web graph has been influenced permanently and pervasively by the existence of search engines. A search engine ranks a page highly, Authors find the page more often, some of them link to it, raising its in-degree and Pagerank, which leads to a further improvement or entrenchment of its rank.

The Results in This Paper The celebrity nodes eventually accumulate a constant fraction of all links created with high probability The degree of the other nodes still follow a power- law distribution with a steeper power:

The New Model Modeling how the web graphs evolves if the author use search engine to decide on links that they insert into new pages. How the degree distribution deviates from the traditional model

The New Model Undirected Web Graph Query to the Search Engine is fixed The search Engine returns a fix number of URLs ordered by their degree at the previous time-step Limit the analysis to one topic at a time with out loss of generality?? Comments: A new page may involve multiple topics at the same time and include different number of links for each topic.

The New Model Growth process: Generates a sequence of graphs G t, t =1,2,3,… At time t, the Graph G t = (V t, E t ) has t vertices and mt edges. Parameters: p: a probability N: maximum number of celebrity nodes listed by the search engine

The New Model – Comments Comments: The number of links each new page creates is fixed? Is this real? How does this affect the results? Intuitively, the page author may not have a number in mind of how many links he wants to include, he will only determine whether a link will be included based on the content of that link

Some Notations in the new model

Formal Definition of Process P

The New Model In both cases y i is selected by preferential attachment within the target subset of old nodes, i.e. for x in U

The New Model - Comments The m random edges may have duplicate vertices. For different i, the same vertex may be selected! When t is smaller than m, we have a lot of loops. Should we not start from one vertex? Instead, we can start from m vertices or N vertices and the initial web graph is created at random. With high probability, the oldest links become celebrity page. What happens in the real world? A page becomes hot not only by random, but also due to its contents, can we model this??

The simulation results Very different from the standard preferential attachment! The celebrities is far from the Power-Law straight line in log-log plot. As p increases, the power increases as well! PSimulated powerComputed power P = P = P = The celebrities command a constant fraction of the total degree over all nodes, this fraction grows with p.

The simulation results

Results

Theorem 1

Interpretations Celebrities capture a large? (depends on the constant) fraction of links. Non-celebrities follow a power-law degree distribution with a power steeper than in preferential attachment.

The Proof The celebrity list becomes fixed whp after some time t f Once the celebrity list is fixed, process P looks very similar to an analogous process P * : In each step, P * takes the N oldest vertices as S t, instead of the N largest-degree vertices. This is quite reasonable, basically, the oldest vertices have higher degree, since they have longer time to be included

Coupling G t and G t *

Analysis of the degree distribution of G t *

Basic Proof to Lemma 2 Finding recurrence of Finding a similar recurrence:

Lemma 3

Basic Proof to Lemma 3

The celebrity list get fixed WHP, adding m edges to a single non-celebrity will not make it a celebrity. The total degree of celebrities is concentrated to a constant fraction of all edges ever added to the graph

List-fixing Lemma

Proof to Lemma 4

Lemma 5

Lemma 6 With low degree, the celebrity has low degree

Lemma 7 With low probability, the non-celebrity has high degree

Lemma 8 With low probability, the gap will keep small

Proof of Theorem 1 Lst t f to be the last time that S t changes in the process P

Proof of Theorem 1 cont.

Conclusions Modeling the influence of a search engine within the preferential attachment framework leads to a qualitative change in the familiar power-law degree distribution. Each of a clot of celebrities captures a constant fraction of the total degree of the graph, and the degree of the remaining nodes follow a steeper power law.

Is this Model real? The model differs from the reality. Edges are undirected? Outlinks are not modified after creation Pages do not die No topic-based clustering

Comments This model is used on to one topic There may be interactions between topics The author may include links for different topics into the same page The number of links on a page is fixed, which is not the real case

Thank you!