Piyush Kumar(Lecture 2: PageRank)

Welcome to COT5405

Quick Recap: Linear AlgebraMatrices

2 3 7

1 1 5A

1 3 1

2 1 4

4 7 6

B

Source: http://www.phy.cuhk.edu.hk/phytalent/mathphy/

3

Square matrices

When m = n, i.e.,

11 12 1

21 22 2

1 2

n

n

n n nn

a a a

a a aA

a a a

1.1 Matrices

4

Sums of matrices

1.2 Operations of matrices

1 2 3

0 1 4

A

2 3 0

1 2 5

BExample: if and

Evaluate A + B and A – B. 1 2 2 3 3 0 3 5 3

0 ( 1) 1 2 4 5 1 3 9

A B

1 2 2 3 3 0 1 1 3

0 ( 1) 1 2 4 5 1 1 1

A B

5

Scalar multiplication

1.2 Operations of matrices

1 2 3

0 1 4

AExample: . Evaluate 3A.

3 1 3 2 3 3 3 6 93

3 0 3 1 3 4 0 3 12

A

6

Properties

1.2 Operations of matrices

Matrices A, B and C are conformable,

A + B = B + A

A + (B +C) = (A + B) +C

(A + B) = A + B, where is a scalar

(commutative law)

(associative law)

Can you prove them?

(distributive law)

7

Matrix multiplication

1.2 Operations of matrices

If A = [aij] is a m p matrix and B = [bij] is a p n matrix, then AB is defined as a m n matrix C = AB, where C= [cij] with

1 1 2 21

...

p

ij ik kj i j i j ip pjk

c a b a b a b a b

1 2 3

0 1 4

A

1 2

2 3

5 0

BExample: , and C = AB. Evaluate c21.

1 21 2 3

2 30 1 4

5 0

21 0 ( 1) 1 2 4 5 22 c

for 1 i m, 1 j n.

8

Matrix multiplication

1.2 Operations of matrices

1 2 3

0 1 4

A

1 2

2 3

5 0

BExample: , , Evaluate C = AB.

11

12

21

22

1 ( 1) 2 2 3 5 181 2

1 2 2 3 3 0 81 2 32 3

0 ( 1) 1 2 4 5 220 1 45 0

0 2 1 3 4 0 3

c

c

c

c

1 21 2 3 18 8

2 30 1 4 22 3

5 0

C AB

9

Properties

1.2 Operations of matrices

Matrices A, B and C are conformable,

A(B + C) = AB + AC

(A + B)C = AC + BC

A(BC) = (AB) C

AB BA in general

AB = 0 NOT necessarily imply A = 0 or B = 0

AB = AC NOT necessarily imply B = C How

ever

Identity Matrix

Examples of identity matrices: and

1 0

0 1

1 0 0

0 1 0

0 0 1

11

The transpose of a matrix

The matrix obtained by interchanging the rows and columns of a matrix A is called the transpose of A (write AT).

Example:

The transpose of A is

1 2 3

4 5 6

A

1 4

2 5

3 6

TA

For a matrix A = [aij], its transpose AT = [bij], where bij = aji.

1.3 Types of matrices

12

If matrices A and B such that AB = BA = I, then B is called the inverse of A (symbol: A-

1); and A is called the inverse of B (symbol: B-1).

The inverse of a matrix

6 2 3

1 1 0

1 0 1

B

Show B is the the inverse of matrix A.

1 2 3

1 3 3

1 2 4

A

Example:

1 0 0

0 1 0

0 0 1

AB BA

Ans: Note that Can you show the details?

1.3 Types of matrices

13

Symmetric matrix

A matrix A such that AT = A is called symmetric, i.e., aji = aij for all i and j.

A + AT must be symmetric. Why?

Example: is symmetric.1 2 3

2 4 5

3 5 6

A

A matrix A such that AT = -A is called skew-symmetric, i.e., aji = -aij for all i and j.

A - AT must be skew-symmetric. Why?

1.3 Types of matrices

14

(AB)-1 = B-1A-1

(AT)T = A and (A)T = AT

(A + B)T = AT + BT

(AB)T = BT AT

1.4 Properties of matrix

3.15

The determinant of a 2 × 2 matrix:

Note:

1. For every square matrix, there is a real number associated with this matrix and

called its determinant

2. It is common practice to delete the matrix brackets

2221

1211

aa

aaA

12212211||)det( aaaaAA

2221

1211

aa

aa

2221

1211

aa

aa

Source: http://www.management.ntu.edu.tw/~jywang/course/

3.16

Historically, the use of determinants arose from the recognition of special patterns that occur in the solutions of linear systems:

Note:

1. a11a22 - a21a12≠0

2. x1 and x2 have the same denominator, and this quantity is called the

determinant of the coefficient matrix A

11 1 12 2 1

21 1 22 2 2

1 22 2 12 2 11 1 211 2

11 22 21 12 11 22 21 12

and

a x a x b

a x a x b

b a b a b a b ax x

a a a a a a a a

3.17

Ex. 1: (The determinant of a matrix of order 2)

21

32

24

12

42

30

Note: The determinant of a matrix can be positive, zero, or negative

)3(1)2(2 34 7

)1(4)2(2 44 0

)3(2)4(0 60 6

18

1.5 Determinants

1. If every element of a row (column) is

zero, e.g., , then |A| = 0.

2. |AT| = |A|

3. |AB| = |A||B|

determinant of a matrix = that of its transpose

The following properties are true for determinants of any order.

1 21 0 2 0 0

0 0

Eigenvalues and Eigenvectors

Ax = λx

Should not exist?

det(A − λI) = 0.

Fact: A and transpose(A) have the same eigenvalues. Why?

Task of search enginesCrawlBuild indices so that one can search

keywords efficiently.Rate the importance of pages.

One example is the simple algorithm named pagerank.

The basic ideaMimic democracy!Use the brains of all people collectively.

The basic ideaMimic democracy!Use the brains of all people collectively for

the ranking.

What’s wrong withcounting backlinks?

Should page 1 be rankedabove page 4?

Voting using backlinks?

But then we don’t want an individual to cast more than one vote?

Normalize?

Normalized Voting?

Link Matrix (for the given web):

Link Matrix (for the given web):

Most important node = 1?

DefinitionA square matrix is called column stochastic if

all of its entries are non-negative and the entries in each column sum to 1.

Lemma: Every column stochastic matrix has 1 as an eigenvalue.

Proof: A and A’ = transpose of A, have the same eigenvalues: Why?

Two shortcomingsNonunique Rankings.Dangling nodes : Nodes with no outgoing

edges.The matrix is no longer column stochastic.Can we transform it into one easily?

Nonunique Rankings

Not clear: Which linear combination should we pick for the ranking?

Nonunique Rankings

Nonunique Rankings

Modification of the Link Matrix

The value of m used by google (1998) was .15For any m between 0,1; M is column stochastic.M can be used to compute unambiguous importance scores(in the absence of dangling nodes)

For m = 1, the only normalized eigenvector with eigenvalue 1 is ?

Modification of the Link Matrix

Example 1For our first example graph, m = 0.15.

Example 2Still, m = 0.15.

Towards the proof

For real numbers

Proof by Contradiction? -> Let x be an eigenvector with mixed signs for the eigenvalue 1.

Towards the proof

A punchline

The Algorithm (aka Power Method)

c ?

One last lemma…

Why does it converge?

The main theorem

For figure 2:

First Example

Do we need any modifications to A?

Calculations

Another Example

Random Surfer ModelThe 85-15 Rule:

Assume that 85 per cent of the time the random surfer clicks a random link on the current page (each link chosen with equal probability)

15 percent of the time the random surfer goes directly to a random page (all pages on the web chosen with equal probability).

Random Surfer ModelCons

No one chooses links or pages with equal probability.

There is no real potential to surf directly to each page on the web.

The 85-15 (or any fixed) breakdown is just a guess. Back Button? Bookmarks?

Despite these flaws, the model is good enough that we have learnt a great deal about the web using it.

Related stuff to explore Random walks and Markov Chains. Random Graph construction using Random walks. Absorbing Markov Chains. Ranking with not too many similar items at the top. Dynamical Systems point of view. Equilibrium or Stationary Distributions. Rate of convergence. Perron-Frobenius Theorem Intentional Surfer model.

Markov Chain Slides:

http://www.math.dartmouth.edu/archive/m20x06/public_html/Lecture13.pdfhttp://www.math.dartmouth.edu/archive/m20x06/public_html/Lecture14.pdfhttp://www.math.dartmouth.edu/archive/m20x06/public_html/Lecture15.pdf

Homework 1Implementation: Parse wikipedia pages and find

pageranks of top 1000 pages of the given input. (TBA)Theory: Solve Exercises in the given paper. (Online)

There are 24 questions in total in the paper (including subproblems marked with a filled disc, Example, problem 6 has 3 subproblems).

Pick the first two characters of your fsu.edu email address. Example “pk” for pkumar@fsu.edu. (all lowercase)

Represent in hex : “706B” = x = Your hex number goes here.Calculate f1 = ((x mod 3D) mod 18)+1Calculate f2 = (f1 + 12) mod 18Solve those two exercises in the paper. Write the problems you

solve (including problem numbers) and the solution in Latex. Submit.