tropical presentation short · tropical mathematics (and why you should care) manos theodosis...
TRANSCRIPT
Tropical mathematics(and why you should care)
Manos Theodosis
Harvard UniversitySchool of Engineering and Applied Sciences
OutlineIntroduction Motivation (algebra and geometry) General definitions Eigenvalues and polynomials Dilations and erosions Curves and surfaces Applications Neural networks Curve fitting Spectral analysis Parameter reduction
Harvard UniversitySchool of Engineering and Applied Sciences
Introduction
Harvard UniversitySchool of Engineering and Applied Sciences
Multidimensional manifolds
Geometric algorithms
Neural networks
Motivation (geometry)
Harvard UniversitySchool of Engineering and Applied Sciences
Figure 1Complex manifolds.
(a) (b) (c)
Figure 2DBSCAN algorithm.
(a) Unconnected components. (b) Connected components.
(a) Decision region of a morphological neural network.
(b) Newton polytope of a tropical polynomial.
Figure 3Tropical neural networks and their regions.
Shortest path problems Scheduling problems
Bonus: Matrix representation (vector spaces) Piecewise linear solution spaces
Harvard UniversitySchool of Engineering and Applied Sciences
start
destination
Motivation (algebra)
Figure 4General formulation of a shortest path problem.
Figure 5Scheduling problem of connecting flights.
General definitionsSimilar to linear algebra, but replace the pair with . Matrix-vector multiplication becomes In general two variants; min-plus algebra and max-plus algebra. Example
Harvard UniversitySchool of Engineering and Applied Sciences
( + , × ) (min , + )
(Ax)j= ∑
i
(Aji ⋅ xi) ⇔ (A ⊞′ x)j= min
i(Aji + xi)
A ⊞′ b = [3 41 −26 2 ] ⊞′ [1
3] =min(3 + 1, 4 + 3)
min(1 + 1, − 2 + 3)min(6 + 1, 2 + 3)
= [415]
Ab = [3 41 −26 2 ] [1
3] =(3 ⋅ 1) + (4 ⋅ 3)
(1 ⋅ 1) + (−2 ⋅ 3)(6 ⋅ 1) + (2 ⋅ 3)
=15−512
vs
Eigenvalues and polynomialsEigenvalues Two very important matrices Solutions to the generalized (sub)eigenvalue problems Polynomials Exponents change meaning General form of a tropical polynomial We can allow for non integer (or negative) exponents
Γ(A) = min(A, A2, . . . ) =∞
minn=1
An, Δ(A) = min(I, A, A2, . . . ) =∞
minn=0
An
A ⊞′ x = λ + x, A ⊞′ x ≥ λ + x
Harvard UniversitySchool of Engineering and Applied Sciences
xa = x ⋅ x ⋅ … ⋅ x ⇔ xa = x + x + … + x = a ⋅ x
linear tropical
y = mini
aix + ci ⇒ y = mini
aTi x
y = min(0.3x + 4, − 5x − 5,15
x)
Distributivity Erosions distribute over infima, dilations over suprema Formulation A unique adjunction pair Fundamental property For any dilation, erosion, lattice element Tropical erosions and dilations In our context where and is the max-plus multiplication.
⋀i
ε(Xi) = ε (⋀i
Xi), ⋁i
δ(Xi) = δ (⋁i
Xi)
Dilations and erosions
δ(X) ≤ Y ⇔ X ≤ ε(Y )
Harvard UniversitySchool of Engineering and Applied Sciences
δ(ε(X)) ≤ X ≤ ε(δ(X))
δA(x) = A ⊞ x, εA(x) = A* ⊞′ x
A* = − AT ⊞
Euclidean curves Tropical curves
Harvard UniversitySchool of Engineering and Applied Sciences
Curves: Polynomials
Figure 6Euclidean polynomials.
(a) y = ax + b (b) y = ax2 + bx + c (c) y = ax3 + bx2 + cx + d
Figure 7Tropical polynomials.
(b� a) (c� b)
(2b� a)
c
a
(b) y = min(a + 2x, b + x, c)(b� a)
b
a
(a) y = min(a + x, b)(b� a) (c� b) (d� c)
d
a
2c� b
3b� 2a
(c) y = min(a + 3x, b + 2x, c + x, d )
1D Halfspaces 2D Halfspaces
Harvard UniversitySchool of Engineering and Applied Sciences
Curves: Halfspaces and polytopes
Figure 8Halfspaces and polytopes in 1D. For example, the red line is .y = min(1 + x,2)
�2 0 2 40
1
2
3
4
Rr�
Rr
x
y
(a) Tropical line and its halfspaces.
0 50
1
2
3
4
RP
Rb�
Rg�
Rr
x
y
(b) Tropical polytope; intersection of tropical halfspaces.
Figure 9Tropical polytopes in 2D. Blue is and red is .z2 = min(5 + x,7 + y,9)
z1 = max(0 + x,2 + y,7)
05
100
10
0
10
xy
z
(b) Second view.
0 5 100
10
0
10
x
y
z
(a) First view.
Curves: Tropical Loci
Harvard UniversitySchool of Engineering and Applied Sciences
2x < min(y, c) c < min(2x, y)
y < min(2x, c)
c/2
c
Figure 10Tropical locus of the equation
and the resulting space clustering.y = min(2x, c)
2y + c
2x+ a x+ y + b
y + d
x+ f
e
Figure 11Tropical locus of the general quadratic tropical 2-D polynomial
.z = min(2x + a, x + y + b,2y + c, y + d, x + f, e)
Applications
Harvard UniversitySchool of Engineering and Applied Sciences
Neural networksNeural networks Traditional morphological perceptron ReLUs and maxout units are tropical polynomials Idea: find a bound the number of linear regions/vertices of the solution space.
Harvard UniversitySchool of Engineering and Applied Sciences
τ(x) = maxi
wi + xi = wT ⊞ x
Figure 12Regions of a morphological perceptron for binary classification.
ReLU(x) = max(0,wTx + b), maxout(x) = maxj
(WTj x + bj)
Neural networks: Linear regionsTropical polynomials Remember the definition Newton polytope Defined as the convex hull of the coefficient vectors . How do operations on polynomials affect the Newton polytope?
ai
⋅ Newt(y1 + y2) = P1 ⊕ P2⋅ Newt(max(y1, y2)) = conv(P1, P2)
Harvard UniversitySchool of Engineering and Applied Sciences
y = mini
aTi x
P1 � P2
(0, 3)
(0, 1)
(1, 3)
(3, 2)
(3, 1)P1
P2
conv(P1 [ P2)
(0, 0)
(�1, 1)
(�1, 0)
P1
(3, 1)
(1, 2)
P2
(0, 0)
(0, 1)(�1, 1)
(�1, 0)
P1
(3, 1)(1, 1)
(1, 2)
P2
Figure 13Newton polytopes of the tropical polynomials and , their Minkwoski sum, and their convex hull.
z1 = max(−x, − x + y, y,0)z2 = max(x + y,3x + y, x + 2y)
Minkwoski sum
Modeling Suppose we fit a model For more data Fundamental property Remember that In the tropical case Optimal solutions Optimal solution for Optimal unconstrained solution
w
Curve fitting
Harvard UniversitySchool of Engineering and Applied Sciences
δ(ε(X)) ≤ X ≤ ε(δ(X))
X ⊞ w ≤ b
δX(εX(b)) ≤ b ⇔ X ⊞ (−XT ⊞′ b) ≤ b
y = max(aT1 x + w1, …, aT
k x + wk) hyperplanes
w = − XT ⊞′ b
w∞ = − XT ⊞′ b + μ μ =12
∥X ⊞ w − b∥∞
Tropical models Arbitrary models
Harvard UniversitySchool of Engineering and Applied Sciences
Curve fitting: 1D examples
!10 !5 0 5 10!5
0
5
10
!3x !2x !1x
0x
1x 2x 3x
�2 �1 0 1 2�2
0
2
4
6
8
Input Data
TargetData
Max-plus Tropical Line Fitting
Tropic GLETropic MMAE
0 2 4 6 8 10 120
2
4
6
8
AMTmi .�i�
h�`;
2i.
�i�
J�t@THmb h`QTB+�H GBM2 6BiiBM;
h`QTB+ :G1h`QTB+ JJ�11m+HB/ Ga1
0 2 4 6 8 10 120
2
4
6
8
AMTmi .�i�
h�`;
2i.
�i�
J�t@THmb h`QTB+�H GBM2 6BiiBM;
h`QTB+ :G1h`QTB+ JJ�11m+HB/ Ga1
Figure 14Euclidean vs tropical fitting for the max-plus line under different noises.
y = max(x − 2,3)
Figure 15Toy circle fitting; highlights the various tropical lines.
Figure 16Optimal fitting of the max-afine curve given
by .y = max(−6x − 6,12
x,15
x5 +12
x)
2D surfaces Questions Number of terms? Complexity? How to compute the slopes?
�1 �0.5 0 0.5 1�1
0
1
0
1
2
x
y
z
�1 �0.5 0 0.5 1�1
0
1
0
1
2
x
y
zHarvard UniversitySchool of Engineering and Applied Sciences
Curve fitting: 2D examples
Figure 17Optimal fitting (left) of parabolic data and the corresponding unconstrained solution (right).
Spectral Analysis of WFSTsWeight pushing Can be written in tropical algebra
with . Epsilon removal Can be written in tropical algebra
Do they remind you something?
λ′ = Λ ⊞′ v∞, ρ′ = P ⊞′ (−v∞), A′ = V− ⊞′ A ⊞′ V+
v∞ = Δ(A) ⊞′ ρ
A′ = Δ(E) ⊞′ Aε, ρ′ = Δ(E) ⊞′ ρ
Harvard UniversitySchool of Engineering and Applied Sciences
Figure 18Example showcasing the weight pushing algorithm for WFSTs.
Figure 19Example showcasing the epsilon removal algorithm for WFSTs.
Spectral Analysis of WFSTsEigenproblems
provides the solution to Solution set is Interpretation For weight pushing ; eigenvalue of zero. For epsilon removal ; eigenvalues of zero. Idea: what about other eigenvalues?
Δ(A)
v∞ = Δ(A) ⊞′ ρ
A′ = Δ(E) ⊞′ Aε, ρ′ = Δ(E) ⊞′ ρ
Harvard UniversitySchool of Engineering and Applied Sciences
A ⊞′ x ≥ λ + x
V*(A, λ) = {Δ(−λ + A) ⊞′ u, u ∈ Rn}
Parameter ReductionEuclidean What’s the minimum number of hyperplanes required for a closed region?
: three lines : four planes : (?) hyperplanes
Proof: 1. hyperplanes can bound -dimensional space (axes + a -dimensional hyperplane passing through points) 2. hyperplanes can’t bound the -dimensional space (induction)
How many parameters? parameters per hyperplane; total parameters.
Tropical hyperplanes of parameters each; total parameters.
Common ground For a polytope of vertices Euclidean needs parameters. Tropical needs parameters.
d = 2d = 3d = n n + 1
d + 1 d dd
d d
d d(d + 1)
2 d 2d
2d
⋅ 2d2
⋅ 2d
Harvard UniversitySchool of Engineering and Applied Sciences
Harvard UniversitySchool of Engineering and Applied Sciences
Thanks!