analysis meets graphs · 2019-02-28 · functional analysis compressive sensing stochastic...
TRANSCRIPT
Analysis meets Graphs
Matthew Begue
Norbert Wiener CenterDepartment of Mathematics
University of Maryland, College Park
Why study graphs?
Graph theory has developed into a useful tool in appliedmathematics.Vertices correspond to different sensors, observations, or datapoints. Edges represent connections, similarities, or correlationsamong those points.
Wikipedia graph
Graphs in pure mathematics too!
Why study Analysis?
Why study Analysis?
Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis
Why study Analysis?
Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis
Why study Analysis?
Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis
Why study Analysis?
Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis
Why study Analysis?
Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis
Why study Analysis?
Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis
Why study Analysis?
Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis
Why study Analysis?
Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis
Why study Analysis?
Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis
Why study Analysis?
Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis
Why study Analysis?
Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis
Why study Analysis?
Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis
Why study Analysis?
Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis
Why study Analysis?
Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis
Why study Analysis?
Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis
Why study Analysis?
Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis
Analysis toolbox
TranslationConvolutionModulationFourier Transform
Analysis toolbox
TranslationConvolutionModulationFourier Transform
Analysis toolbox
TranslationConvolutionModulationFourier Transform
Analysis toolbox
TranslationConvolutionModulationFourier Transform
Outline
1 Fourier Transform
2 Modulation
3 Convolution
4 Translation
Motivation
In the classical setting, the Fourier transform on R is given by
f (ξ) =
∫R
f (t)e−2πiξt dt = 〈f ,e2πiξt〉.
This is precisely the inner product of f with the eigenfunctions ofthe Laplace operator.
Motivation
In the classical setting, the Fourier transform on R is given by
f (ξ) =
∫R
f (t)e−2πiξt dt = 〈f ,e2πiξt〉.
This is precisely the inner product of f with the eigenfunctions ofthe Laplace operator.
Graph Laplacian
DefinitionThe pointwise formulation for the Laplacian acting on a functionf : V → R is
∆f (x) =∑y∼x
f (x)− f (y).
For a finite graph, the Laplacian can be represented as a matrix.Let D denote the N × N degree matrix, D = diag(dx ).Let A denote the N × N adjacency matrix,
A(i , j) =
{1, if xi ∼ xj0, otherwise.
Then the unweighted graph Laplacian can be written as
L = D − A.
Spectrum of the Laplacian
L is a real symmetric matrix and therefore has nonnegativeeigenvalues {λk}N−1
k=0 with associated orthonormal eigenvectors{ϕk}N−1
k=0 .If G is finite and connected, then we have
0 = λ0 < λ1 ≤ λ2 ≤ · · · ≤ λN−1.
Easy to show that ϕ0 ≡ 1/√
N.
Data Sets - Minnesota Road Network
−98 −97 −96 −95 −94 −93 −92 −91 −90 −8943
44
45
46
47
48
49
50
(a) λ3
−98 −97 −96 −95 −94 −93 −92 −91 −90 −8943
44
45
46
47
48
49
50
(b) λ4
−98 −97 −96 −95 −94 −93 −92 −91 −90 −8943
44
45
46
47
48
49
50
(c) λ5
−98 −97 −96 −95 −94 −93 −92 −91 −90 −8943
44
45
46
47
48
49
50
(d) λ6
−98 −97 −96 −95 −94 −93 −92 −91 −90 −8943
44
45
46
47
48
49
50
(e) λ7
−98 −97 −96 −95 −94 −93 −92 −91 −90 −8943
44
45
46
47
48
49
50
(f) λ8
Figure : Eigenfunctions corresponding to the first six nonzero eigenvalues.Minnesota road graph (2642 vertices)
Data Sets - Sierpinski gasket graph approximation
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
(a) λ3
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
(b) λ4
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
(c) λ5
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
(d) λ6
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
(e) λ7
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
(f) λ8
Figure : Eigenfunctions corresponding to the first six nonzero eigenvalues.Level-8 graph approximation to Sierpinski gasket (9843 vertices)
Data Sets - Sierpinski gasket graph approximation
−1
−0.5
0
0.5
1
0
0.5
1
1.5
2
−0.02
−0.015
−0.01
−0.005
0
0.005
0.01
0.015
0.02
(a) λ3
−1
−0.5
0
0.5
1
0
0.5
1
1.5
2
−0.02
−0.015
−0.01
−0.005
0
0.005
0.01
(b) λ4
−1
−0.5
0
0.5
1
0
0.5
1
1.5
2
−0.02
−0.015
−0.01
−0.005
0
0.005
0.01
0.015
(c) λ5
−1
−0.5
0
0.5
1
0
0.5
1
1.5
2
−0.02
−0.015
−0.01
−0.005
0
0.005
0.01
0.015
0.02
(d) λ6
−1
−0.5
0
0.5
1
0
0.5
1
1.5
2
−0.02
−0.015
−0.01
−0.005
0
0.005
0.01
0.015
0.02
(e) λ7
−1
−0.5
0
0.5
1
0
0.5
1
1.5
2
−0.02
−0.015
−0.01
−0.005
0
0.005
0.01
0.015
0.02
(f) λ8
Figure : Eigenfunctions corresponding to the first six nonzero eigenvalues.Level-8 graph approximation to Sierpinski gasket (9843 vertices)
Graph Fourier Transform
DefinitionThe graph Fourier transform is defined as
f (λl ) = 〈f , ϕl〉 =N∑
n=1
f (n)ϕ∗l (n).
Notice that the graph Fourier transform is only defined on values ofσ(L).The inverse Fourier transform is then given by
f (n) =N−1∑l=0
f (λl )ϕl (n).
Outline
1 Fourier Transform
2 Modulation
3 Convolution
4 Translation
Graph Modulation
In Euclidean setting, modulation is multiplication of a Laplacianeigenfunction.
DefinitionFor any k = 0,1, ...,N − 1 the graph modulation operator Mk , isdefined as
(Mk f )(n) =√
Nf (n)ϕk (n).
On R,
modulation in the time domain = translation in the frequencydomain
Mξf (ω) = f (ω − ξ).
Example - Classical Setting
Example - Movie
G =SG6f (λl ) = δ2(l) =⇒ f = ϕ2.
−1
−0.5
0
0.5
1
0
0.5
1
1.5
2
−0.01
−0.005
0
0.005
0.01
0.015
0.02
Example - Movie
G =Minnesotaf (λl ) = δ2(l) =⇒ f = ϕ2.
−98 −97 −96 −95 −94 −93 −92 −91 −90 −8943
44
45
46
47
48
49
50
Outline
1 Fourier Transform
2 Modulation
3 Convolution
4 Translation
Graph Convolution - Motivation and Definition
Classically, for signals f ,g ∈ L2(R) we define the convolution as
f ∗ g(t) =
∫R
f (u)g(t − u) du.
However, there is no clear analogue of translation in the graphsetting. So we exploit the property
( f ∗ g)(ξ) = f (ξ)g(ξ),
and then take inverse Fourier transform.
DefinitionFor f ,g : V → R, we define the graph convolution of f and g as
f ∗ g(n) =N−1∑l=0
f (λl )g(λl )ϕl (n).
Outline
1 Fourier Transform
2 Modulation
3 Convolution
4 Translation
Graph Translation
For signal f ∈ L2(R), the translation operator, Tu, can be thoughtof as a convolution with δu.On R we can calculateδu(k) =
∫R δu(x)e−2πikx dx = e−2πiku(= ϕ∗
k (u)).Then by taking the convolution on R we have
(Tuf )(t) = (f∗δu)(t) =
∫R
f (k)δu(k)ϕk (t) dk =
∫R
f (k)ϕ∗k (u)ϕk (t) dk
DefinitionFor any f : V → R the graph translation operator, Ti , is defined to be
(Ti f )(n) =√
N(f ∗ δi )(n) =√
NN−1∑l=0
f (λl )ϕ∗l (i)ϕl (n).
Example - Classical Setting
Translation in the time domain = Modulation in the frequency domain
Example - Movie
G =Minnesotaf = 11
−98 −97 −96 −95 −94 −93 −92 −91 −90 −8943
44
45
46
47
48
49
50
0
0.2
0.4
0.6
0.8
1
0 1 2 3 4 5 6−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
Example - Movie
G =Minnesotaf (λl ) = e−5λl
−98 −97 −96 −95 −94 −93 −92 −91 −90 −89
43
44
45
46
47
48
49
50
−0.1
−0.05
0
0.05
0 1 2 3 4 5 6 7−0.02
0
0.02
0.04
0.06
0.08
0.1
0.12
Example - Movie
G =SG6f (λl ) = e−5λl
−1
−0.5
0
0.5
1
0
0.5
1
1.5
2
−0.1
−0.08
−0.06
−0.04
−0.02
0
0.02
0.04
0.06
0.08
0.1
0 1 2 3 4 5 6−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
Example - Movie
G =Minnesotaf ≡ 1
−98 −97 −96 −95 −94 −93 −92 −91 −90 −8943
44
45
46
47
48
49
50
−5
0
5
10
0 1 2 3 4 5 6−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
Nice Properties of Graph Translation
Theorem
For any f ,g : V → R and i , j ∈ {1,2, ...,N} then1 Ti (f ∗ g) = (Ti f ) ∗ g = f ∗ (Tig).2 TiTj f = TjTi f .
Not-so-nice Properties of Translation operator
The translation operator is not isometric
‖Ti f‖`2 6= ‖f‖`2
In general, the set of translation operators {Ti}Ni=1 do not form a
group like in the classical Euclidean setting.
TiTj 6= Ti+j
In general, Can we even hope for TiTj = Ti•j for some semigroupoperation, • : {1, ...,N} × {1, ...,N} → {1, ...,N}?
Theorem (B. & O.)
Given a graph, G, with eigenvector matrix Φ = [ϕ0| · · · |ϕN−1]. Graphtranslation on G is a semigroup, i.e. TiTj = Ti•j for some semigroupoperator • : {1, ...,N} × {1, ...,N} → {1, ...,N}, only if Φ = (1/
√N)H,
where H is a Hadamard matrix.
Not-so-nice Properties of Translation operator
The translation operator is not isometric
‖Ti f‖`2 6= ‖f‖`2
In general, the set of translation operators {Ti}Ni=1 do not form a
group like in the classical Euclidean setting.
TiTj 6= Ti+j
In general, Can we even hope for TiTj = Ti•j for some semigroupoperation, • : {1, ...,N} × {1, ...,N} → {1, ...,N}?
Theorem (B. & O.)
Given a graph, G, with eigenvector matrix Φ = [ϕ0| · · · |ϕN−1]. Graphtranslation on G is a semigroup, i.e. TiTj = Ti•j for some semigroupoperator • : {1, ...,N} × {1, ...,N} → {1, ...,N}, only if Φ = (1/
√N)H,
where H is a Hadamard matrix.
Not-so-nice Properties of Translation operator
The translation operator is not isometric
‖Ti f‖`2 6= ‖f‖`2
In general, the set of translation operators {Ti}Ni=1 do not form a
group like in the classical Euclidean setting.
TiTj 6= Ti+j
In general, Can we even hope for TiTj = Ti•j for some semigroupoperation, • : {1, ...,N} × {1, ...,N} → {1, ...,N}?
Theorem (B. & O.)
Given a graph, G, with eigenvector matrix Φ = [ϕ0| · · · |ϕN−1]. Graphtranslation on G is a semigroup, i.e. TiTj = Ti•j for some semigroupoperator • : {1, ...,N} × {1, ...,N} → {1, ...,N}, only if Φ = (1/
√N)H,
where H is a Hadamard matrix.
Not-so-nice Properties of Translation operator
The translation operator is not isometric
‖Ti f‖`2 6= ‖f‖`2
In general, the set of translation operators {Ti}Ni=1 do not form a
group like in the classical Euclidean setting.
TiTj 6= Ti+j
In general, Can we even hope for TiTj = Ti•j for some semigroupoperation, • : {1, ...,N} × {1, ...,N} → {1, ...,N}?
Theorem (B. & O.)
Given a graph, G, with eigenvector matrix Φ = [ϕ0| · · · |ϕN−1]. Graphtranslation on G is a semigroup, i.e. TiTj = Ti•j for some semigroupoperator • : {1, ...,N} × {1, ...,N} → {1, ...,N}, only if Φ = (1/
√N)H,
where H is a Hadamard matrix.
Why should we care?
With these time-frequency operators generalized tovertex-frequency operators, we are able to convert many niceresults and theories from harmonic analysis to the graph setting.
Wavelets and Wavelet TransformWindowed/Short-Time Fourier Transform (STFT)Windowed Fourier Frames
Why should we care?
Why should we care?
Thank you!