polynomial regression on riemannian manifoldsjacob/pubs/frg2012.pdf · polynomial regression on...

Polynomial Regression onRiemannian Manifolds

Jacob Hinkle, Tom Fletcher, Sarang Joshi

May 11, 2012

arxiv:1201.2395

Nonparametric RegressionNumber of parameters tied to amount of data present

Example: kernel regression on images using diffeomorphisms(Davis2007)

Polynomial Regression on Riemannian Manifolds 2

Parametric RegressionSmall number of parameters can be estimated more efficiently

Fletcher 2011

Geodesic regression (Niethammer2011, Fletcher2011) hasrecently received attention.

Polynomial Regression

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

−0.01

−0.005

Independent Variable

Polynomials provide a more flexible framework for parametricregression on Riemannian manifolds

Riemannian PolynomialsAt least three ways to define polynomial in Rd

Algebraic: γ(t) = c0 + 11!c1t+ 1

2!c2t2 + · · ·+ 1

k!cktk

Variational: γ = argminϕ∫ T0 |(ddt

) k+12 ϕ(t)|2dt s.t. BC/ICs

Differential:(ddt

)k+1γ(t) = 0 s.t. initial conditions

)iγ(0) = ci

Covariant derivative: replace ddt of vectors with ∇γ̇

2!c2t2 + · · ·+ 1

k!cktk

Differential:(ddt

)iγ(0) = ci

2!c2t2 + · · ·+ 1

k!cktk

Differential:(ddt

)iγ(0) = ci

2!c2t2 + · · ·+ 1

k!cktk

Differential:(ddt

)iγ(0) = ci

2!c2t2 + · · ·+ 1

k!cktk

Differential:(ddt

)iγ(0) = ci

Geodesic (k = 1) has both formsγ = argminϕ

∫ T0 |ϕ̇(t)|2dt

∇γ̇ γ̇ = 0 s.t. initial conditions γ(0), γ̇(0)

Well-studied (Fletcher, Younes, Trouve, …)

2!c2t2 + · · ·+ 1

k!cktk

Differential:(ddt

)iγ(0) = ci

Cubic spline satisfies (Noakes1989, Leite, Machado,…)γ = argminϕ

∫ T0 |∇ϕ̇ϕ̇(t)|2dt

Euler-Lagrange equation: (∇γ̇)3γ̇ = R(γ̇,∇γ̇ γ̇)γ̇

Shape splines (Trouve-Vialard)

2!c2t2 + · · ·+ 1

k!cktk

Differential:(ddt

)iγ(0) = ci

k-order polynomial satisfies(∇γ̇)k γ̇ = 0subject to initial conditions γ(0), (∇γ̇)iγ̇(0), i = 0, . . . , k − 1

Introduced via rolling maps by Jupp&Kent1987Studied by Leite (2008), in rolling map setting

2!c2t2 + · · ·+ 1

k!cktk

Differential:(ddt

)iγ(0) = ci

2!c2t2 + · · ·+ 1

k!cktk

Differential:(ddt

)iγ(0) = ci

Rolling maps

Leite 2008

Unroll curve α on manifold to curve αdev on Rd without twistingor slipping. Then

(∇α̇)kα̇ = 0 ⇐⇒(d

)kα̇dev = 0

Unknown whether this satisfies a variational principle

Rolling maps

Leite 2008

Unroll curve α on manifold to curve αdev on Rd without twistingor slipping. Then

(∇α̇)kα̇ = 0 ⇐⇒(d

)kα̇dev = 0

Unknown whether this satisfies a variational principlePolynomial Regression on Riemannian Manifolds 6

Riemannian Polynomials

Generate via forward evolution of linearizedsystem of first-order covariant ODEsForward Polynomial Evolution

repeatw ← v1for i = 1, . . . , k − 1 dovi ← ParallelTransportγ(∆tw, vi + ∆tvi+1)

end forvk ← ParallelTransportγ(∆tw, vk)γ ← Expγ(∆tw)t← t+ ∆t

until t = T

Parametrized by ICs:γ(0) positionv1(0) velocityv2(0) accelerationv3(0) jerk

(∇γ̇)kγ̇ = 0 becomes linearized system

γ̇ = v1

∇γ̇vi = vi+1 i = 1, . . . , k − 1

∇γ̇vk = 0.

Want to find initial conditions for this ODE that minimize

E(γ) =

N∑i=1

gi(γ(ti))

Lagrange multiplier (adjoint) vector fields λi along γ:

E∗(γ, {vi}, {λi}) =

N∑i=1

gi(γ(ti)) +

0〈λ0, γ̇ − v1〉dt

k−1∑i=1

0〈λi,∇γ̇vi − vi+1〉dt+

0〈λk,∇γ̇vk〉dt

Euler-Lagrange for {λi} gives forward system.Vector field integration by parts:∫ T

0〈λi,∇γ̇vi〉dt = [〈λi, vi〉]T0 −

0〈∇γ̇λi, vi〉dt

Lagrange multiplier (adjoint) vector fields λi along γ:

E∗(γ, {vi}, {λi}) =

N∑i=1

gi(γ(ti)) +

0〈λ0, γ̇ − v1〉dt

k−1∑i=1

0〈λi,∇γ̇vi − vi+1〉dt+

0〈λk,∇γ̇vk〉dt

Euler-Lagrange for {λi} gives forward system.Vector field integration by parts:∫ T

0〈λi,∇γ̇vi〉dt = [〈λi, vi〉]T0 −

Rewrite using integration by parts

E∗(γ, {vi}, {λi}) =

N∑i=1

gi(γ(ti)) +

0〈λ0, γ̇ − v1〉dt

k−1∑i=1

[〈λi, vi〉]T0 −k−1∑i=1

−k−1∑i=1

0〈λi, vi+1〉dt

+ [〈λk, vk〉]T0 −∫ T

0〈∇γ̇λk, vk〉dt

So variation w.r.t. {vi} gives

δviE∗ = 0 = −∇γ̇λi − λi−1

δvi(T )E∗ = 0 = λi(T )

δvi(0)E∗ = −λi(0)

Rewrite using integration by parts

E∗(γ, {vi}, {λi}) =

N∑i=1

gi(γ(ti)) +

0〈λ0, γ̇ − v1〉dt

k−1∑i=1

[〈λi, vi〉]T0 −k−1∑i=1

−k−1∑i=1

0〈λi, vi+1〉dt

+ [〈λk, vk〉]T0 −∫ T

0〈∇γ̇λk, vk〉dt

So variation w.r.t. {vi} gives

δviE∗ = 0 = −∇γ̇λi − λi−1

δvi(T )E∗ = 0 = λi(T )

δvi(0)E∗ = −λi(0)

Variation with respect to the curve γ:Let {γs : s ∈ (−ε, ε)} a smooth family of curves, with:

γ0 = γ

W (t) :=d

dsγs(t)|s=0

Extend vi, λi away from curve via parallel transport:

∇W vi = 0

∇Wλi = 0

Then∫ T

0〈δγE∗(γ, {vi}, {λi}),W 〉dt =

dsE∗(γs, {vi}, {λi})|s=0

For any smooth family of curves γs(t), we have[d

dsγs(t),

dtγs(t)

]= [W, γ̇s] = 0

∇W γ̇ = ∇γ̇W.

We also need the Leibniz rule

ds〈X,Y 〉|s=0 = 〈∇WX,Y 〉+ 〈X,∇WY 〉,

And the Riemann curvature tensor

R(X,Y )Z = ∇X∇Y Z −∇Y∇XZ −∇[X,Y ]Z

∇W∇γ̇Z = ∇γ̇∇WZ +R(W, γ̇)Z

For first term, T1 =∫ T0 〈λ0, γ̇s〉dt

dsT1(γs)|s=0 =

0〈λ0, γ̇s〉dt|s=0

0〈∇Wλ0, γ̇s〉+ 〈λ0,∇W γ̇s〉dt|s=0

0〈0, γ̇s〉+ 〈λ0,∇γ̇W 〉dt|s=0

= [〈λ0,W 〉]T0 −∫ T

0〈∇γ̇λ0,W 〉dt

Variation of this term with respect to γ is

δγ(t)T1 = −∇γ̇λ0δγ(T )T1 = 0 = λ0(T )

δγ(0)T1 = −λ0(0)

Now do the same with another term Ti

dsTi(γs)|s=0 =

0〈λi,∇γ̇vi〉dt

0〈∇Wλi,∇γ̇vi〉+ 〈λi,∇W∇γ̇vi〉dt

0〈λi,∇γ̇∇W vi +R(W, γ̇)vi〉dt

0〈R(λi, vi)γ̇,W 〉dt

where we used Bianchi identities to rearrange the curvatureterm. So

δγ(t)Ti = R(λi, vi)γ̇

Combine all terms to get adjoint equations

∇γ̇λ0 =

N∑i=1

δ(t− ti)(grad gi(γ(t))) +

k∑i=1

R(λi, vi)v1

∇γ̇λi = −λi−1

Initialization for λi at t = T is

λi(T ) = 0,

Parameter gradients are

δγ(0)E = −λ0(0)

δvi(0)E = −λi(0)

Typically, gi(γ) = d(γ, yi)2, so that

(grad gi(γ)) = −Logγ yi

∇γ̇λ0 =

N∑i=1

k∑i=1

R(λi, vi)v1

∇γ̇λi = −λi−1

λi(T ) = 0,

δγ(0)E = −λ0(0)

δvi(0)E = −λi(0)

∇γ̇λ0 =

N∑i=1

k∑i=1

R(λi, vi)v1

∇γ̇λi = −λi−1

λi(T ) = 0,

δγ(0)E = −λ0(0)

δvi(0)E = −λi(0)

Algorithmrepeat

Integrate γ, {vi} forward to t = TInitialize λi(T ) = 0, i = 0, . . . , kIntegrate {λi} via adjoint equations back to t = 0Gradient descent step:γ(0)n+1 = Expγ(0)n(ελ0(0))

vi(0)n+1 = ParTransγ(0)n(ελ0(0), vi(0)n + ελi(0))until convergence

Special Case: Geodesic (k = 1)

Adjoint system is

∇γ̇λ0 =

N∑i=1

δ(t− ti)(grad gi(γ(t))) +R(λ1, v1)v1

∇γ̇λ1 = −λ0

Between data points this is

(∇γ̇)2λ1 = −R(λ1, γ̇)γ̇

This is the Jacobi equation, λ1 is a Jacobi field.

Kendall Shape Space

Space of N landmarks in d dimensions,RNd, modulo translation, scale, rotationPrevents skewed statistics due tosimilarity transformed datad = 2, complex projective space CPN−2

Kendall Shape Space Geometry(d = 2)

Center point-set and scale so that∑N

i=1 |xi|2 = 1 (resultingobject is called a preshape)Preshapes lie on sphere S2N−1, represented as vectors in(R2)N = CN

Riemannian submersion from preshape to shape space:vertical direction holds rotations of R2

Exponential and log map available in closed form (for d = 2)

Covariant derivative in shape space in terms of preshape(O’Neill1966):

∇∗X∗Y∗ = H∇XY

Vertical direction is JN , where N is outward unit normal at thepreshape, J is almost complex structure on CN .

So parallel transport in small steps in upstairs space then dohorizontal projection.

Curvature on preshape sphere S2N−1, R(X,Y )Z is:

R(X,Y )Z = 〈X,Z〉Y − 〈Y,Z〉X

For curvature, need first fundamental form A. For horizontal vf’sX,Y ,

AXY =1

2V[X,Y ]

Curvature downstairs is

〈R∗(X∗, Y∗)Z∗, H〉 = 〈R(X,Y )Z,H〉+ 2〈AXY,AZH〉 − 〈AY Z,AXH〉 − 〈AZX,AYH〉

First fundamental form is (O’Neill)

AXY = 〈X,JY 〉JN

Adjoint of AZ :

〈AXY,AZH〉 = 〈−J〈X, JY 〉Z,H〉

Curvature then is

R∗(X∗, Y∗)Z∗ = R(X,Y )Z − 2J〈X, JY 〉Z + J〈Y, JZ〉X + J〈Z, JX〉Y

Bookstein Rat Calivarium Growth

8 landmark points18 subjects8 ages

−0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.5 0.6

−0.2

−0.1

−0.32 −0.3 −0.28 −0.26 −0.24 −0.22 −0.2

−0.26

−0.24

−0.22

−0.2

−0.18

−0.16

−0.14

−0.15 −0.1 −0.05

0.22 0.24 0.26 0.28 0.3 0.32−0.22

−0.2

−0.18

−0.16

−0.14

−0.12

−0.1

0.44 0.46 0.48 0.5 0.52 0.54 0.56−0.2

−0.18

−0.16

−0.14

−0.12

−0.1

−0.08

−0.06

1 0.792 0.853 0.87

Corpus Collosum Aging (www.oasis-brains.org)

Fletcher 2011

N = 32 patientsAge range 18–9064 landmarks usingShapeWorks sci.utah.edu

1 0.122 0.133 0.21

−0.15 −0.1 −0.05 0 0.05 0.1 0.15−0.08

−0.06

−0.04

−0.02

ic−0.15 −0.1 −0.05 0 0.05 0.1 0.15

−0.08

−0.06

−0.04

−0.02

−0.15 −0.1 −0.05 0 0.05 0.1 0.15

−0.08

−0.06

−0.04

−0.02

Corpus Collosum Aging

−0.15 −0.1 −0.05 0 0.05 0.1 0.15 0.2

−0.08

−0.06

−0.04

−0.02

γ̇(0)∇γ̇ γ̇(0)

(∇γ̇)2γ̇(0)

Initial conditions are collinear, implying time reparametrization

Landmark Space

Space L of N points in Rd. Geodesic equations:

dtxi =

N∑j=1

γ(|xi − xj |2)αj

dtαi = −2

N∑j=1

(xi − xj)γ′(|xi − xj |)2αTi αj

Usually use Gaussian kernel

γ(r) = e−r/(2σ2)

x ∈ L and α ∈ T ∗xL is a covector (momentum)

Landmark Space

Space L of N points in Rd. Geodesic equations:

dtxi =

N∑j=1

γ(|xi − xj |2)αj

dtαi = −2

N∑j=1

(xi − xj)γ′(|xi − xj |)2αTi αj

Usually use Gaussian kernel

γ(r) = e−r/(2σ2)

x ∈ L and α ∈ T ∗xL is a covector (momentum)

Have simple formula for cometric gij (the kernel)Parallel transport in terms of covectors, cometric:

dtβ` =

in,j g

jm (αmβn − αnβm)− 1

2gmn,` αmβn

Curvature more complicated (Mario’s Formula):

2Rursv = −gur,sv − grv,us + grs,uv + guv,rs + 2Γrvρ Γusσ gρσ − 2Γrsρ Γuvσ g

+ grλ,ugλµgµv,s − grλ,ugλµgµs,v + guλ,rgλµg

µs,v − guλ,rgλµgµv,s

+ grλ,sgλµgµv,u + guλ,vgλµg

µs,r − grλ,vgλµgµs,u − guλ,sgλµgµv,r.

Landmark parallel transport in momenta (Younes2008):

dtβi = K−1

( N∑j=1

(xi − xj)T ((Kβ)i − (Kβ)j)γ′(|xi − xj |2)αj

−N∑j=1

(xi − xj)T ((Kα)i − (Kα)j)γ′(|xi − xj |2)βj

−N∑j=1

(xi − xj)γ′(|xi − xj |2)(αTj βi + αTi βj)

This is enough to integrate polynomials

For curvature, need Christoffel symbols and their derivatives:

(Γ(u, v))i = −N∑

(xi − xj)T (vi − vj)γ′(|xi − xj |2)(K−1u)j

−N∑

(xi − xj)T (ui − uj)γ′(|xi − xj |2)(K−1v)j

γ(|xi − xj |2)

N∑k=1

(xj − xk)γ′(|xj − xk|2)((K−1u)Tk (K−1v)j + (K−1u)Tj (K−1v)k)

Take derivative with respect to x, and combine using

R`ijk = Γ`ki,j − Γ`ji,k + Γ`jmΓmki − Γ`kmΓmji

((DΓ(u, v))w)i =N∑j=1

(wi − wj)T (ui − uj)γ′(|xi − xj |2)(K−1v)j

+ 2N∑j=1

(xi − xj)T (ui − uj)(xi − xj)T (wi − wj)γ′′(|xi − xj |2)(K−1v)j

+N∑j=1

(xi − xj)T (ui − uj)γ′(|xi − xj |2)((d

dεK−1)v)j

+N∑j=1

(wi − wj)T (vi − vj)γ′(|xi − xj |2)(K−1u)j

+ 2N∑j=1

(xi − xj)T (vi − vj)(xi − xj)T (wi − wj)γ′′(|xi − xj |2)(K−1u)j

+N∑j=1

(xi − xj)T (vi − vj)γ′(|xi − xj |2)((d

dεK−1)u)j

− 2N∑j=1

(xi − xj)T (wi − wj)γ′(|xi − xj |2)N∑k=1

(xj − xk)γ′(|xj − xk|2)((K−1u)Tk (K−1v)j + (K−1u)Tj (K−1v)k)

−N∑j=1

γ(|xi − xj |2)N∑k=1

(wj − wk)γ′(|xj − xk|2)((K−1u)Tk (K−1v)j + (K−1u)Tj (K−1v)k)

− 2N∑j=1

γ(|xi − xj |2)N∑k=1

(xj − xk)(xj − xk)T (wj − wk)γ′′(|xj − xk|2)((K−1u)Tk (K−1v)j + (K−1u)Tj (K−1v)k)

−N∑j=1

γ(|xi − xj |2)N∑k=1

(xj − xk)γ′(|xj − xk|2)

× ((d

dεK−1u)Tk (K−1v)j + (K−1u)Tk (

dεK−1v)j + (

dεK−1u)Tj (K−1v)k + (K−1u)Tj (

dεK−1v)k)(

dεK−1)v

= −(K−1d

dεKK−1v)i

= −2(K−1N∑j=1

(xk − xj)T (wk − wj)γ′(|xk − xj |2)(K−1v)j

Landmark Regression ResultsSame Bookstein rat data. Procrustes alignment, no scaling.

−0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8

−0.2

−0.1

0.2 0.25 0.3 0.35 0.4 0.450.24

−0.2 −0.18 −0.16 −0.14 −0.12 −0.1 −0.08 −0.06 −0.04 −0.02 00.22

R2 = 0.92 geodesic, 0.94 quadraticPolynomial Regression on Riemannian Manifolds 31

Thank You!

polynomial regression on riemannian manifoldsjacob/pubs/frg2012.pdf · polynomial regression on...

Documents

local polynomial order in regression discontinuity...

polymath regression tutorial on polynomial fitting of...

polynomial regression

a machine learning approach to polynomial regression

piecewise-polynomial regression...

measurement-based color correction method for 3d … · 3d...

multivariate local polynomial regression with application to...

the use of polynomial regression analysis with indicator...

lecture 6: multiple linear regression, polynomial ... ·...

multivariate local polynomial regression with application...

multiresolution local polynomial regression: a new approach...

method using second-order polynomial regression analysis

alternatives to difference scores: polynomial regression and...

wepor: a weighted polynomial regression program

methodology and application of savitzky-golay moving average...

evolutionary polynomial regression-based models to

topic 16: multicollinearity and polynomial regression

fp — fractional polynomial regression - stata · pdf...

multiple linear and polynomial regression with statistical...

literature review for local polynomial regression€¦ ·...