PageRank Algorithm

Prepared By: Mai Mustafa

Contents:• Background• Introduction to PageRank• PageRank Algorithm• Power iteration method• Examples using PageRank and iteration • Exercises• Pseudo code of PageRank algorithm• Searching with PageRank• Application using PageRank• Advantages and disadvantages of PageRank algorithm• References

Background

PageRank was presented and published by Sergey Brin and Larry Page at the Seventh International World Wide Web Conference (WWW7) in April 1998.

The aim of this algorithm is track some difficulties with the content-based ranking algorithms of early search engines which used text documents for webpages to retrieve the information with no explicit relationship of link between them.

Introduction to PageRank

• PageRank is an algorithm uses to measure the importance of website pages using hyperlinks between pages.• Some hyperlinks point to pages to the same site (in link)

and others point to pages in other Web sites(out link).• PageRank is a “vote”, by all the other pages on the Web,

about how important a page is.• A link to a page counts as a vote of support

PageRank Algorithm

The main concepts: • In-links of page i : These are the hyperlinks that point to page i from other

pages. Usually, hyperlinks from the same site are not considered.• Out-links of page i : These are the hyperlinks that point out to other pages

from page i .

The following ideas based on rank prestige are used to derive the PageRank algorithm:• A hyperlink from a page pointing to another page is an implicit conveyance

of authority to the target page. Thus, the more in-links that a page i receives, the more prestige the page i has.

• Pages that point to page i also have their own prestige scores. A page with a higher prestige score pointing to i is more important than a page with a lower prestige score pointing to i .

Cont. PageRank AlgorithmTo formulate the above ideas, we treat the Web as a directed graph G = (V, E), where V is the set of vertices or nodes, i.e., the set of all pages, and E is the set of directed edges in the graph, i.e., hyperlinks. Let the total number of pages on the Web be n (i.e., n = |V|).

The PageRank score of the page i (denoted by P(i)) is defined by:

1-

Oj is the number of out-links of page j.

Cont. PageRank AlgorithmMathematically, we have a system of n linear equations (1) with n unknowns. We can use a matrix to represent all the equations. Let P be a n-dimensional column vector of PageRank values

Let A be the adjacency matrix of our graph with 2-

We can write the system of n equations with 3-

Cont. PageRank Algorithmthe above three conditions come from Markov chains Model, in it; each Web page in the Web graph is regarded as a state. A hyperlink is a transition, which leads from one state to another state with a probability. Thus, this framework models Web surfing as a stochastic process. It models a Web surfer randomly surfing the Web as a state transition in the Markov chain .so on, this three conditions are not satisfied. Because First of all, A is not a stochastic matrix. A stochastic matrix is the transition matrix for a finite Markov chain whose entries in each row are nonnegative real numbers and sum to 1. This requires that every Web page must have at least one out-link. This is not true on the Web because many pages have no out-links, which are reflected in transition matrix A by some rows of complete 0’s. Such pages are called dangling pages (nodes).

Cont. PageRank Algorithm

We can see that A is not a stochastic matrix because the fifth row is all 0’s, that is, page 5 is a dangling page.We can fix this problem by adding a complete set of outgoing links from each such page i to all the pages on the Web. Thus, the transition probability of going from i to every page is 1/n, assuming a uniform probability distribution. That is, we replace each row containing all 0’s with e/n, where e is n-dimensional vector of all 1’s.

Cont. PageRank AlgorithmAnother problems: A is not irreducible, which means that the Web graph G is not strongly connected. And to be strongly connected it must have a path from u to v. (if there is a non-zero probability of transitioning from any state to any other state).A is not aperiodic. A state i in a Markov chain being periodic means that there exists a directed cycle that the chain has to traverse. To be aperiodic all paths leading from state i back to state i have a length that is a multiple of k. It is easy to deal with the above two problems with a single strategy. We add a link from each page to every page and give each link a small transition probability controlled by a parameter d, it is used to model the probability that at each page the surfer will become unhappy with the links and request another random page.The parameter d, called the damping factor, can be set to a value between 0 and 1.Always d = 0.85.

Cont. PageRank Algorithm• The PageRank model:• (t: Transpose)

• The PageRank formula for each page i :

TA

Power iteration method:The PageRank algorithm must be able to deal with billions of pages, meaning incredibly immense matrices; thus, we need to find an efficient way to calculate the eigenvector of a square matrix with a dimension in the billions. Thus, the best option for calculating the eigenvector is through the power method. The power method is a simple and easy to implement algorithm. Additionally, it is effective in that it is not necessary to compute a matrix decomposition, which is near-impossible for matrices containing very few values, such as the link matrix we receive. The power method does have downsides, however, in that it is only able to find the eigenvector of the largest absolute-value eigenvalue of a matrix. Also, the power method must be repeated many times until it converges, which can occur slowly. Fortunately, as we are working with a stochastic matrix, the largest eigenvalue is guaranteed to be 1. Since this is the eigenvector we are searching for, the power method will return the importance vector we are looking for. Additionally, it has been proven that the speed of convergence for the Google PageRank matrix is slower the closer α gets to 0. Since we have set d to be equal to 0.15, we can expect the speed of convergence to be approximately 50 - 100 iterations, which is the number of iterations reported by the creators of PageRank to be necessary for returning sufficiently close values.

Simple example using PageRank with iteration

2 pages A,B:

• P(A)=(1-d)+d(pagerank(B)/1) P(A)=0.15+0.85*1=1

• P(B)=(1-d)+d(pagerank(A)/1) P(B)=0.15+0.85*1=1

When we calculate the PageRank of A and B is 1. now, we plug in 0 as the guess and calculate again: P(A)=0.15+0.85*0=0.15 P(B)=0.15+0.85*0.15=0.2775Continue the second iteration: P(A)=0.15+0.85*0.2775=0.3859 P(B)=0.15+0.85*0.3859=0.4780If we repeat the calculations, eventually the PageRank for both the pages converge to 1.

Another example using PageRank with iterationThree pages A,B And C• P(A)=(1-d)+d(pagerank(B)+pagerank(C)/1)• P(B)=(1-d)+d(pagerank(A)/2)• P(C)=(1-d)+d(pagerank(A)/2)

Begin with the initial value as 0:1st iteration:P(A)=0.15+0.85*0=0.15P(B)=0.15+0.85*(0.15/2)=0.21P(c)=0.15+0.85*(0.15/2)=0.212nd iteration:P(A)=0.15+0.85*(0.21*2)=0.51P(B)=0.15+0.85*(0.51/2)=0.37P(C)=0.15+0.85*(0.51/2)=0.37

Cont. example

3rd iteration:P(A)=0.15+0.85*(0.37*2)=0.78P(B)=0.15+0.85*(0.87/2)=0.48P(C)=0.15+0.85*(0.87/2)=0.48And so on.. After 20 iterationsP(A)=1.46P(B)=0.77P(C)=0.77The total PageRank =3, but we can see A has much larger proportion of the PageRank than B and C, because they are passing to A not to any other pages.

Exercise:Given A below, obtain P by solving Equation PageRank model directly.

first: we will represent the matrix as graph:

pdAedp T )1(

004

1000

2

10

4

1000

2

10000

3

1

02

1

4

10

2

1

3

1

02

1

4

110

3

1

00002

10

TA

Find first then find e: TA

6

1

6

1

6

1

6

1

6

1

6

16

1

6

1

6

1

6

1

6

1

6

16

1

6

1

6

1

6

1

6

1

6

16

1

6

1

6

1

6

1

6

1

6

16

1

6

1

6

1

6

1

6

1

6

16

1

6

1

6

1

6

1

6

1

6

1

e

002125.0000

425.002125.0000

425.00000283.0

0425.02125.00425.0283.0

0425.02125.085.00283.0

0000425.00

025.0025.0025.0025.0025.0025.0

025.0025.0025.0025.0025.0025.0

025.0025.0025.0025.0025.0025.0

025.0025.0025.0025.0025.0025.0

025.0025.0025.0025.0025.0025.0

025.0025.0025.0025.0025.0025.0

p

004

1000

2

10

4

1000

2

10000

3

1

02

1

4

10

2

1

3

1

02

1

4

110

3

1

00002

10

85.0

6

1

6

1

6

1

6

1

6

1

6

16

1

6

1

6

1

6

1

6

1

6

16

1

6

1

6

1

6

1

6

1

6

16

1

6

1

6

1

6

1

6

1

6

16

1

6

1

6

1

6

1

6

1

6

16

1

6

1

6

1

6

1

6

1

6

1

15.0p

And we know that d=0.85

025.0025.02375.0025.0025.0025.0

45.0025.02375.0025.0025.0025.0

45.0025.0025.0025.0025.0308.0

025.045.02375.0025.045.0308.0

025.045.02375.0875.0025.0308.0

025.0025.0025.0025.045.0025.0

p

Exercise2:Given A as in problem 1 in the last exercise, use the power iteration method to show the first 5 iterations of P.• First iteration:

• Second iteration:

0k

0.363

0.788

0.858

1.496

1.921

0.575

1

1

1

1

1

1

025.0025.02375.0025.0025.0025.0

45.0025.02375.0025.0025.0025.0

45.0025.0025.0025.0025.0308.0

025.045.02375.0025.045.0308.0

025.045.02375.0875.0025.0308.0

025.0025.0025.0025.045.0025.0

0k

0.333

0.487

0.467

1.647

2.102

0.966

0.363

0.788

0.858

1.496

1.921

0.575

025.0025.02375.0025.0025.0025.0

45.0025.02375.0025.0025.0025.0

45.0025.0025.0025.0025.0308.0

025.045.02375.0025.045.0308.0

025.045.02375.0875.0025.0308.0

025.0025.0025.0025.045.0025.0

* 01 kPk

• third iteration:

• Fourth iteration:

0.250

0.391

0.565

1.623

2.130

1.043

0.333

0.487

0.467

1.647

2.102

0.966

025.0025.02375.0025.0025.0025.0

45.0025.02375.0025.0025.0025.0

45.0025.0025.0025.0025.0308.0

025.045.02375.0025.045.0308.0

025.045.02375.0875.0025.0308.0

025.0025.0025.0025.045.0025.0

* 12 kPk

0.270

0.377

0.551

1.637

2.111

1.055

0.250

0.391

0.565

1.623

2.130

1.043

025.0025.02375.0025.0025.0025.0

45.0025.02375.0025.0025.0025.0

45.0025.0025.0025.0025.0308.0

025.045.02375.0025.045.0308.0

025.045.02375.0875.0025.0308.0

025.0025.0025.0025.045.0025.0

* 23 kPk

• Fifth iteration:

We would then continue this iterating until the values are approximately stable, and we would be able to determine the importance ranking using the resulting vector. With this, we can see that even with a small count of 5 iterations, our vector was already converging towards the eigenvector. Since this is the importance vector of our network, we can see that the PageRank importance ranking of our pages would thus be

2 > 3 > 1 > 4 > 5 > 6

0.267

0.382

0.563

1.623

2.118

1.047

0.270

0.377

0.551

1.637

2.111

1.055

025.0025.02375.0025.0025.0025.0

45.0025.02375.0025.0025.0025.0

45.0025.0025.0025.0025.0308.0

025.045.02375.0025.045.0308.0

025.045.02375.0875.0025.0308.0

025.0025.0025.0025.045.0025.0

* 34 kPk

Pseudo code of PageRank algorithm:

Searching with PageRankTwo search engines:• Title-based search engine• Full text search engine

• Title-based search engine Searches only the “Titles” Finds all the web pages whose titles contain all the query

words Sorts the results by PageRank Very simple and cheap to implement Title match ensures high precision, and PageRank ensures

high quality

• Full text search engine• Called Google• Examines all the words in every stored document and also

performs PageRank (Rank Merging)• More precise but more complicated

Cont. searching with PageRank

Application using PageRank

• the first and most obvious application of the PageRank algorithm is for search engines. As it was developed specifically by Google for use in their search engine, PageRank is able to rank websites in order to provide more relevant search results faster.

• applied PageRank algorithm is towards searching networks outside of the internet. this can be applied towards academic papers; by using citations as a substitute for links, PageRank can determine the most effective and referenced papers in an academic area.

• real-world application of the PageRank algorithm; for example, determining key species in an ecology. By mapping the relationships between species in an ecosystem, applying the PageRank algorithm allows the user to identify the most important species. Thus, being able to assign importance towards key animal and plant species in an ecosystem allows for easier forecasting of consequences such as extinction or removal of a species from the ecosystem.

Advantage and disadvantages of PageRank algorithm:

Advantages of PageRank:1. The algorithm is robust against Spam since its not easy for a

webpage owner to add in links to his/her page from other important pages.

2. PageRank is a global measure and is query independent.

Disadvantages of PageRank:3. it favors the older pages, because a new page, even a very good

one will not have many links unless it is a part of an existing site.4. It is very efficient to raise your own PageRank, is ’buying’ a link on

a page with high PageRank.

References:

• Comparative Analysis Of Pagerank And HITS Algorithms, by: Ritika Wason. Published in IJERT, October - 2012.• The top ten algorithms in data mining, by: Xindong wu

and vipin kumar.• Building an Intelligent Web: Theory and Practice, By

Pawan Lingras, Saint Mary.• Hyperlink based search algorithms-PageRank and

HITS, by: Shatakirti.