harmonic networks - cs.ucl.ac. · pdf filescale-space harmonic analysis representation h-nets...
TRANSCRIPT
![Page 1: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/1.jpg)
HARMONIC NETWORKS: DEEP TRANSLATION AND ROTATION EQUIVARIANCE
Daniel E. Worrall, Stephan J. Garbin, Daniyar Turmukhambetov, and Gabriel J. Brostow
![Page 2: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/2.jpg)
OUR GROUP
![Page 3: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/3.jpg)
OUTLINE
• Brief intro: Invariance and Equivariance
• What has been done for rotations?
• Harmonic Networks and the Fourier Shift Theorem
• Appendix: Further theory and future works
![Page 4: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/4.jpg)
INVARIANCE
• Many tasks invariant to input transformations
Doggie DoggieDoggie
![Page 5: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/5.jpg)
INVARIANCE
![Page 6: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/6.jpg)
e.g. translation
Mapping independent of Tg, for all Tg
f(I) = f(Tg[I])Notational aside: (in our case)
Tg[I](x) = I(Mgx+ bg)
Tg[I](x) = I(x+ bg)
INVARIANCE WARNING: DIFFERENT NOTATION TO PAPER
![Page 7: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/7.jpg)
INVARIANCE: EXAMPLES
Var[p] = Var[Tgp]Variance: translations
Fourier spectrum modulus: translations����Z 1
�1I(x)e�◆!x dx
���� =����Z 1
�1I(x� b)e�◆!x dx
����
Pooling: smooth invertible mappings (typ. diffeomorphisms)Z
x2⌦I(x) dx =
Z
x2⌦I(Tgx) dx max
x2⌦I(x) = max
x2⌦I(Tgx)
spectrum[M] = spectrum[QMQ>] QQ> = Q>Q = I
Eigenspectrum: orthogonal transformations
![Page 8: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/8.jpg)
EQUIVARIANCE“The man swallowed the chicken”
versus “The chicken swallowed the man”
![Page 9: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/9.jpg)
EQUIVARIANCE: GENERAL DEFINITION
Sg[f(I)] = f(Tg[I])Mapping preserves algebraic structure of transformation
Sg = Id
Invariance is special case of equivariance
Different representations of same transformation
![Page 10: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/10.jpg)
EQUIVARIANCE: EXAMPLESFourier spectrum phase: translations
e
�◆!b
Z 1
�1I(x)e�◆!x dx =
Z 1
�1I(x� b)e�◆!x dx
Mean: translationsaEp[x] + b = Ep[ax+ b]
[f ⇤ g](x� b) = [Tb[f ] ⇤ g](x)Convolution (and correlation)
|◆!|nh(!) = d@n
x
h(!)
Fourier transform: differentiation (and pretty much any other linear transformation)
![Page 11: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/11.jpg)
[w ? Tbf ](x) =X
y2X
w(y � x)Tbf(y)
=X
y2X
w(y � x)f(y � b)
=X
y02X
w(y0 + b� x)f(y0)
= [w ? f ](x� b)
TRANSLATION EQUIVARIANCE OF Z-CORRELATION
Boundaries imply local equivariance
Source: https://github.com/vdumoulin/conv_arithmetic
![Page 12: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/12.jpg)
LACK OF ROTATION EQUIVARIANCE:
![Page 13: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/13.jpg)
ROTATION EQUIVARIANCE METHODS
Rotate in network Rotate input
Warped Convolutions (Henriques et al., 2017) Polar Transformer Nets (Esteves et al., 2017)
Exploiting Cyclic Symmetry (Dieleman et al., 2016)
Rotate filters Rotate activations Log-polar trans.Brute-forceRotation-Invariant Neoperceptron (Fasel et al., 2006) TI-Pooling (Laptev et al., 2016)
Roto-Translation Scattering(Oyallon et al., 2014) Group Equivariant CNNs(Cohen et al., 2016) Oriented Response Networks (Zhou et al., 2017) Steerable CNNs (Cohen et al., 2017)
Store full response Store max/mean responseRoto-Translation Invariant Scattering(Sifre et al., 2013) Rotation Equivariant Vector Fields (Marcos et al., 2017)
ALL DISCRETE ROTATIONS
![Page 14: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/14.jpg)
"Do you want the wrong answer to the right question or the right answer to the wrong
question? I think you want the former.”
DAVID BLEI
![Page 15: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/15.jpg)
PART II: HARMONIC NETWORKS
![Page 16: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/16.jpg)
THE FOURIER SHIFT THEOREM
Interestingly this generalises beyond 1D shifts to any transformation if considering groups
Equivariant Invariant
m 2 Z
Constraint on angular component of weights
Z ⇡
�⇡I(�� ✓)e�◆m� d� =
Z ⇡�✓
�⇡�✓I(�0)e�◆m(�0+✓) d�0
= e�◆m✓
Z ⇡
�⇡I(�0)e�◆m�0
d�0
![Page 17: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/17.jpg)
CIRCULAR HARMONICS
Constrained
Example with Gaussian radial component
Live in orthogonal spaces, so don’t interactNote: the sign
flip in the angular
component
Free
Wm(r;�) = R(r)e◆(m�+�)
![Page 18: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/18.jpg)
CIRCULAR HARMONIC WEIGHTS: IN PRACTICAL TERMS
Wm ? I = (WRem ? IRe �W Im
m ? IIm) + ◆(WRem ? IIm +W Im
m ? IRe)
Real Imaginary
![Page 19: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/19.jpg)
CIRCULAR HARMONIC WEIGHTS
K K
![Page 20: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/20.jpg)
CIRCULAR HARMONICS
X-correlation w. circular harmonics returns orientation field
![Page 21: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/21.jpg)
Wm2 ? g([Wm1 ? T✓I])
CIRCULAR HARMONIC ALGEBRASo far we have a mapping
But what about: 1) nonlinearities? 2) subsequent layers?
W? : RM⇥N⇥K ! CM⇥N⇥1
![Page 22: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/22.jpg)
NONLINEARITIESTo maintain linear equivariance apply nonlinearities to invariant magnitude only g(|F |e◆\F ) = g(|F |)e◆\F
�(|F |+ c)ReLU(|F |+ c) LeakyReLU(|F |+ c)
![Page 23: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/23.jpg)
CIRCULAR HARMONIC ALGEBRA
?
![Page 24: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/24.jpg)
CIRCULAR HARMONIC ALGEBRA
Sum orders along chained cross-correlations
Wm2 ? g([Wm1 ? T✓I]) = Y e◆(m1+m2)✓
![Page 25: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/25.jpg)
CIRCULAR HARMONIC ALGEBRA
?
???
?
??
?
?
![Page 26: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/26.jpg)
CIRCULAR HARMONIC ALGEBRAAt a layer, we sum over incoming feature maps, but need to pay special attention to rotation order
![Page 27: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/27.jpg)
CIRCULAR HARMONIC ALGEBRAAt a layer, we sum over incoming feature maps, but need to pay special attention to rotation order
![Page 28: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/28.jpg)
CIRCULAR HARMONIC ALGEBRA
The Equivariance Condition
X
p2Pi
mp = i
Path through i
Rotation order of filterConstant at
feature map i
My conjecture: For Lie Groups, streams identified with basis vectors of Lie Algebra, and harmonic filters act as raising/lowering operators, jumping between basis vectors
![Page 29: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/29.jpg)
CIRCULAR HARMONIC ALGEBRA
![Page 30: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/30.jpg)
CIRCULAR HARMONIC ALGEBRA
![Page 31: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/31.jpg)
SOME EXPERIMENTS
20
20
20
20
20
20
CNN
8
8
16
16
35
35
H-Net
m=0
m=110
8
8
H-DSN
m=0 m=1
16
16
32
32
64
64
64
64
1
1
1
1
1
1
DSN
1
1
1
1
1
1
32
32
64
64
128
128
256
256
256
256
Brute force
Rotation + Scale
ORN-8 [30] 1.37 ~1MTI-Pooling [13] 1.21 ~1MPTN-CNN-B++ 0.89 ~254k
Recent results
![Page 32: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/32.jpg)
SOME EXPERIMENTS
CNNCNN + data aug.H Net (ours)
0.880.900.920.940.960.981.00
2000 4000 6000 8000 10000 12000Training set sizeN
orm
aliz
ed te
st a
ccur
acy
![Page 33: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/33.jpg)
SOME EXPERIMENTS
20
20
20
20
20
20
CNN
8
8
16
16
35
35
H-Net
m=0
m=110
8
8
H-DSN
m=0 m=1
16
16
32
32
64
64
64
64
1
1
1
1
1
1
DSN
1
1
1
1
1
1
32
32
64
64
128
128
256
256
256
256
Dynamic Steerable ResNet 0.732 0.751Recent results
![Page 34: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/34.jpg)
SOME EXPERIMENTS
![Page 35: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/35.jpg)
SOME EXPERIMENTS
![Page 36: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/36.jpg)
SOME EXPERIMENTS
![Page 37: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/37.jpg)
SOME EXPERIMENTS
![Page 38: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/38.jpg)
LIMITATIONS
• Filters too basic
• No stability guarantee for noise/high freq. deformations
• Do we really need higher order streams?
• What about other kinds of transformation?
Working on all of these
![Page 39: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/39.jpg)
REFERENCESRotate filters- Deep Roto-Translation Scattering for Object Classification (Oyallon et al., 2014) - Group Equivariant Convolutional Networks (Cohen et al., 2016) - Rotation Equivariant Vector Field Network (Marcos et al., 2017) - Oriented Response Networks (Zhou et al., 2017) - Steerable CNNs (Cohen et al., 2017) - Roto-Translation Invariant Scattering (Sifre et al., 2013)
Rotate activations
Rotate input- Rotation-Invariant Neoperceptron (Fasel et al., 2006) - TI-POOLING: Transformation-invariant Pooling for Feature Learning in Convolutional
Neural Networks (Laptev et al., 2016) - Warped Convolutions: Efficient Invariance to Spatial Transformations (Henriques et al.,
2017) - Polar Transformer Networks (Esteves et al., 2017)
- Exploiting Cyclic Symmetry in Convolutional Neural Networks (Dieleman et al., 2016)
![Page 40: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/40.jpg)
PART III: THEORY
![Page 41: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/41.jpg)
THEORY: THERE IS A LOT
Group theory
Lie groups
Steerable function spaces
Scale-space theory
Gaussian scale-space
Harmonic Analysis
H-NetsRepresentation theory
![Page 42: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/42.jpg)
THE FOURIER SHIFT THEOREM ON LIE GROUPS
Theory is very involved, so this is just a brief outline
![Page 43: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/43.jpg)
GROUP THEORYA group G=(G, ●) is a set G and a binary
composition operator ●, with
1. Closure
2. Associativity
3. Identity element
4. Unique inverses
(f • g) • h = f • (g • h)
g • h 2 G
g • e = e • g = g
g • g�1 = g�1 • g = e
![Page 44: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/44.jpg)
GROUP THEORY: EXAMPLE
y · z 2 C \ {0}(x · y) · z = x · (y · z)
x · 1 = 1 · x = x
x
�1 =1
|x|e�◆\x
1. Closure 2. Associativity 3. Identity element 4. Unique inverses
(C \ {0}, ·)
![Page 45: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/45.jpg)
2D ROTATIONS FORM A GROUPThese are some different ways to describe rotations
SO(2)
([0, 2⇡),+mod 2⇡)
(C, ·)C = {z 2 C : |z|}C = {z 2 C : |z| = 1}
�{A 2 M2⇥2 : A>A = AA> = I, |A| = 1}, ·
�
In fact this group has a manifold structure: it is a Lie Group (Don’t worry if none of this makes sense—I only understand it every 2nd day)
There are an infinite number of representations to describe essentially the same thing
![Page 46: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/46.jpg)
PATCH IS FUNCTION ON SO(2)
![Page 47: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/47.jpg)
FOURIER TRANSFORMS ON LIE GROUPS
PreviouslyZ
f(�)e�◆m� d� e�◆m�e�◆m✓ = e�◆m(�+✓)
m(✓) m(�) = m(✓ • �)Homomorphism property
NowZ
f(�) m(�) d�
(We lose commutativity)Z
GT✓[f ](�) m(�) d� =
Z
Gf(✓�1�) m(�) d�
=
Z
✓�1Gf(�) m(✓�) d(✓�)
= m(✓)
Z
Gf(�) m(�) d�
Need left-invariant Haar measure
InvariantEquivariant
Extension of Fourier Shift Theorem
![Page 48: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/48.jpg)
INVARIANTS ON LIE GROUPSCan show that the irreducible representations are unitary so
kHk2 = hH|Hi = trace(H>H)
Invariant on group
In such a way we can proceed as before to define a harmonic network over a matrix Lie group
kFm{T✓f}k2 = k m(✓)Fm{f}k2
= h m(✓)Fm{f}| m(✓)Fm{f}i
= trace( ˜Fm{f}> ˜ m(✓)
> m(✓)| {z }
I
Fm{f})
= kFm{f}k2
kFm{T✓f}k2 = k m(✓)Fm{f}k2
= h m(✓)Fm{f}| m(✓)Fm{f}i
= trace( ˜Fm{f}> ˜ m(✓)
> m(✓)| {z }
I
Fm{f})
= kFm{f}k2
kFm{T✓f}k2 = k m(✓)Fm{f}k2
= h m(✓)Fm{f}| m(✓)Fm{f}i
= trace( ˜Fm{f}> ˜ m(✓)
> m(✓)| {z }
I
Fm{f})
= kFm{f}k2
kFm{T✓f}k2 = k m(✓)Fm{f}k2
= h m(✓)Fm{f}| m(✓)Fm{f}i
= trace( ˜Fm{f}> ˜ m(✓)
> m(✓)| {z }
I
Fm{f})
= kFm{f}k2
Fm{f} :=
Z
Gf(�) m(�) d�
![Page 49: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/49.jpg)
WORK IN PROGRESSOpen questions:• How to jump between streams?
• Transformations such as scalings are non-compact, can we approximate them well on an interval?
• Not all transformations form groups. Harmonics are eigenfunctions of the Laplace-Beltrami operator.
• Could we define equivariance on general manifolds or even graphs? • What about eigenfunctions of other operators? c.f. Sturm-Liouville theory
• There are other equivariant transforms such as the Gaussian convolution (affine equivariant). This has a semi-group structure, how does this fit in?
![Page 50: HARMONIC NETWORKS - cs.ucl.ac. · PDF filescale-space Harmonic Analysis Representation H-Nets theory. THE FOURIER SHIFT THEOREM ON LIE GROUPS Theory is very involved, so this is just](https://reader034.vdocuments.us/reader034/viewer/2022051203/5aa009737f8b9a0d158d9c83/html5/thumbnails/50.jpg)
THANK YOU