Matrix Factorization and Matrix Factorization and its applicationsits applicationsBy Zachary16th Nov, 2010

OutlineOutlineExpression power of matrixVarious matrix factorization

methodsApplication of matrix

factorization

What can matrix What can matrix represent?represent?System of equationsUser rating matrixImageMatrix structure in graph theory

◦Adjacent matrix◦Distance matrix

Different matrix factorization Different matrix factorization methodsmethodsLU decompositionSingular Value

Decomposition(SVD)Probabilistic Matrix

Factorization(PMF)Non-negative Matrix

Factorization(NMF)

Application of matrix Application of matrix factorizationfactorizationLU decomposition

◦Solving system of equationsSVD decomposition

◦Low rank matrix approximation◦Pseudo-inverse

Application of matrix Application of matrix factorizationfactorizationPMF

◦Recommendation systemNMF

◦Learning the parts of objects

PMFPMFConsider a typical

recommendation problem◦Given a n by m matrix R with some

entries unknown n rows represent n users m columns represent m movies Entry represent the ith user’s rating on

the jth movie◦We are interested in the unknown

entries’ possible values i.e. Predict users’ ratings

ijR

PMFPMFWe can model the problem as R=U’V

◦U (k by n) is the latent feature matrix for users How much the user likes action movie? How much the user likes comedy movie?

◦V (k by m) is the latent feature matrix for movies To what extent is the movie an action movie? To what extent is the movie a comedy movie?

PMFPMFIf we can learn U and V from

existing ratings, then we can compute unknown entries by multiplying these two matrices.

Let’s consider a probabilistic approach.

PMFPMF

PMFPMFWe want to maximize

Equivalent to minimizing

Can be solved using steepest descent method

Extension to PMFExtension to PMFWe can augment the model as

long as we have additional data matrix that share comment latent feature matrix

NMFNMFConsider the following problem

◦M = 2429 facial images◦Each image of size n = 19 by 19 = 361◦Matrix V = n by m is the original dataset◦We want to approximate V by two lower

rank matrix W (n by 49) and H (49 by m) V ~ WH Constraints

All entries of W and H are non-negative

NMFNMFHow well can W and H

approximate VHow can we interpret the result

NMFNMFAssumption

◦ ◦ ◦Maximize logarithm likelihood and

we get the objective function

Criticize of NMFCriticize of NMFNMF doesn’t always

give parts based resultSparseness constraints

For more information, refer to “Non-negative matrix factorization with sparseness constrains”

Questions?Questions?Thank you