new tensor decompositions in numerical analysis …...new tensor decompositions in numerical...
TRANSCRIPT
![Page 1: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/1.jpg)
NEW TENSOR DECOMPOSITIONS INNUMERICAL ANALYSIS AND DATA
PROCESSING
Eugene Tyrtyshnikov
Institute of Numerical Mathematics of Russian Academy of Sciences
11 October 2012
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 2: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/2.jpg)
COLLABORATION
MOSCOW:
I.Oseledets, D.SavostyanovS.Dolgov, V.Kazeev, O.Lebedeva,A.Setukha, S.Stavtsev, D.ZheltkovS.Goreinov, N.Zamarashkin
LEIPZIG:
W.Hackbusch, B.Khoromskij, R.SchneiderH.-J.Flad, V.Khoromskaia,M.Espig, L.Grasedyck
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 3: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/3.jpg)
TENSORS IN 20TH CENTURY
used chiefly as desriptive tools:
I physics
I differential geometry
I multiplication tables in algebras
I applied data managementI chemometricsI sociometricsI signal/image processingI many others
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 4: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/4.jpg)
WHAT IS TENSOR
Tensor = d -linear form = d -dimensional array:
A = [ai1i2...id ]
Tensor A possesses:
I dimensionality (order) d= number of indices(dimensions, modes, axes, directions, ways)
I size n1 × ...× nd
(number of points at each dimension)
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 5: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/5.jpg)
EXAMPLES OF PROMINENT THEORIES FORTENSORS IN 20th CENTURY
I Kruskal’s theorem (1977) on essential uniquenessof canonical tensor decomposition introduced byHitchcock (1927);
I canonical tensor decompositions as a base forStrassen’s method of matrix multiplication ofcomplexity less than n3 (1969);
I interrelations between tensors (especiallysymmetric) and polynomials as a topic inalgebraic geometry.
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 6: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/6.jpg)
BEGIN WITH 2 × 2 MATRICES
The column-by-row rule for 2× 2 matrices yields 8mults:
[a11 a12
a21 a22
] [b11 b12
b21 b22
]=
[a11b11 + a12b21 a11b12 + a12b22
a21b11 + a22b21 a21b12 + a22b22
]
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 7: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/7.jpg)
DISCOVERY BY STRASSEN
Only 7 mults is enough!IMPORTANT: for block 2× 2 matricesthese are 7 mults of blocks:
α1 = (a11 + a22)(b11 + b22)
α2 = (a21 + a22)b11
α3 = a11(b12 − b22)
α4 = a22(b21 − b11)
α5 = (a11 + a12)b22
α6 = (a21 − a11)(b11 + b12)
α7 = (a12 − a22)(b21 + b22)
c11 = α1 + α4 − α5 + α7
c12 = α3 + α5
c21 = α2 + α4
c22 = α1 + α3 − α2 + α6
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 8: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/8.jpg)
HOW A TENSOR ARISES AND HELPS
[c1 c2
c3 c4
]=
[a1 a2
a3 a4
] [b1 b2
b3 b4
]ck =
n2∑i=1
n2∑j=1
hijk ai bj
hijk =R∑α=1
uiα vjα wkα
⇒ ck =R∑α=1
wkα
n2∑i=1
uiαai
n2∑j=1
vjαbj
Now only R mults of blocks!If n = 2 then R = 7 (Strassen, 1969).Recursion ⇒ O(nlog2 7) scalar mults for any n.
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 9: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/9.jpg)
GENERAL CASE BY RECURSION
Two matrices of order n = 2d can be multiplied with7d = nlog2 7 scalar multiplications and 7nlog2 7 scalaradditions/subtrations.
n = 2d
n/2 n/2 n/2 n/2 n/2 n/2 n/2
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 10: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/10.jpg)
TENSORS IN 21ST CENTURY:NUMERICAL METHODS WITHTENSORIZATION OF DATA
We consider typical problems of numerical analysis(matrix computations, interpolation, optimization)under the assumption that the input, output and allintermediate data are represented bytensors with many dimensions(tens, hundreds, even thousands).
Of course, it assumes a very special structure of data.But we have it in really many problems!
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 11: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/11.jpg)
THE CURSE OF DIMENSIONALITY
The main problem is that using arrays as means tointroduce tensors in many dimensions is infeasible:
I if d = 300 and n = 2, then such an arraycontains 2300 � 1083 entries
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 12: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/12.jpg)
NEW REPRESENTATION FORMATS
Canonical polyadic and Tucker decompositions are oflimited use for our purposes (by different reasons).
New decompositions:I TT (Tensor Train)I HT (Hierarchical Tucker)
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 13: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/13.jpg)
REDUCTION OF DIMENSIONALITY
i1i2i3i4i5i6
i1i2 i3i4i5i6
i1 i2 i3i4 i5i6
i3 i4 i5 i6
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 14: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/14.jpg)
SCHEME FOR TT
i1i2i3i4i5i6
i1i2α i3i4i5i6α
i1β i2αβ i3i4γ i5i6αγ
i3δ i4γδ i5αη i6γη
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 15: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/15.jpg)
SCHEME FOR HTi1i2i3i4i5i6
i1i2α i3i4i5i6α
i1β i2αβ
i2φ αβφ
i3i4γ i5i6αγ
i3δ i4γδ
i4ψ γδψ
i5i6ξ γηξ
i5ζ i6ξζ
i6ν ξζν
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 16: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/16.jpg)
THE BLESSING OF DIMENSIONALITY
TT and HT provide new representation formats ford -tensors + algorithms with complexity linear in d .
Let the amount of data be N. In numerical analysis,complexity O(N) is usually considered as a dream.
With ultimate tensorization we go beyond the dream:
since d ∼ logN, we may obtain complexity O(logN).
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 17: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/17.jpg)
BASIC TT ALGORITHMS
I TT rounding.Like the rounding of machine numbers.COMLEXITY = O(dnr 3).ERROR 6
√d − 1 · BEST ERROR.
I TT interpolation.A tensor train is constructed from sufficiently fewelements of the tensor, the number of them is O(dnr 2).
I TT quantization and wavelets.Low-dimensional → high-dimensional ⇒algebraic wavelet tranbsforms (WTT).
In matrix problems the complexity may dropfrom O(N) down to O(logN).
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 18: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/18.jpg)
SUMMATION AGREEMENT
Omit the symbol of summation. Assume summationif the index in a product of quantities with indices isrepeated at least twice. Equations hold for all valuesof other indices.
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 19: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/19.jpg)
SKELETON DECOMPOSITION
A = UV> =r∑
α=1
u1α
. . .umα
[v1α . . . vnα]
According to the summation agreement,
a(i , j) = u(i , α)v(j , α)
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 20: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/20.jpg)
CANONICAL AND TUCKER
CANONICAL DECOMPOSITION
a(i1 . . . id) = u1(i1α) . . . ud(idα)
TUCKER DECOMPOSITION
a(i1 . . . id) = g(α1 . . . αd)u1(i1α1) . . . ud(idαd)
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 21: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/21.jpg)
TENSOR TRAIN (TT) IN THREE DIMENSIONS
a(i1 ; i2i3) = g1(i1 ; α1)a1(α1 ; i2i3)
a1(α1i2 ; i3) = g2(α1i2 ; α2)g3(α2 ; i3)
TENSOR TRAIN (TT)
a(i1i2i3) = g1(i1α1)g2(α1i2α2)g3(α2i3)
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 22: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/22.jpg)
TENSOR TRAIN (TT) IN d DIMENSIONS
a(i1 . . . id) =
g1(i1α1)g2(α1i2α2) . . .gd−1(αd−2id−1αd−1)gd(αd−1id)
a(i1 . . . id) =d∏
k=1
gk(αk−1ikαk)
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 23: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/23.jpg)
KRONECKER REPRESENTATIONOF TENSOR TRAINS
A = G 1α1⊗ G 2
α1α2⊗ . . .⊗ G d−1
αd−2αd−1⊗ G d
αd−1
A is of size (m1 . . .md)× (n1 . . . nd).
G kαk−1αk
is of size mk × nk .
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 24: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/24.jpg)
ADVANTAGES OF TENSOR-TRAINREPRESENTATION
The tensor is determined throughd tensor carriages gk(αk−1ikαk),each of size rk−1 × nk × rk .
If the maximal size is r × n × r ,then the number of representation parametersdoes not exceed dnr 2 � nd .
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 25: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/25.jpg)
TENSOR TRAIN PROVIDESSTRUCTURED SKELETON DECOMPOSITIONSOF UNFOLDING MATRICES
Ak = a(i1 . . . ik ; ik+1 . . . id) =
uk(i1 . . . ik ; αk) vk(αk ; ik+1 . . . id) = UkV>k
uk(i1 . . . ikαk) = g1(i1α1) . . . gk(αk−1ikαk)
vk(αk ik+1 . . . id) = gk+1(αk ik+1αk+1) . . . gd(αk−1id)
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 26: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/26.jpg)
TT RANKS ARE BOUNDED BY THE RANKSOF UNFOLDING MATRICES
rk > rankAk , Ak = [a(i1 . . . ik ; ik+1 . . . id)]
Equalities are always possible.
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 27: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/27.jpg)
ORTHOGONAL TENSOR CARRIAGES
A tensor carriage g(αiβ) is called row orthogonal ifits first unfolding matrix g(α ; iβ) has orthonormalrows.
A tensor carriage g(αiβ) is called column orthogonalif its second unfolding matrix g(αi ; β) hasorthonormal columns.
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 28: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/28.jpg)
ORTHOGONALIZATION OF TENSORCARRIAGES
∀ tensor carriage g(αiβ) ∃ decomposition
g(αiβ) = h(αα′)q(α′iβ)
with q(α′iβ) being row orthogonal.
∀ tensor carriage g(αiβ) ∃ decomposition
g(αiβ) = q(αiβ′)h(β′β)
with q(αiβ′) being column orthogonal.
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 29: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/29.jpg)
PRODUCTS OF ORTHOGONAL TENSORCARRIAGES
A product of row (column) orthogonal tensorcarriages
p(αs , is . . . it , αt) =t∏
k=s+1
gk(αk−1ikαk)
is also row (column) orthogonal.
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 30: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/30.jpg)
MAKING ALL CARRIAGES ORTHOGONAL
Orthogonalize the columns of g1 = q1h1, thencompute and orthogonalize h1g2 = q2h2. Thus,
g1g2 = q1q2h2
and after k steps
g1 . . . gk = q1 . . . qkhk .
Similarly for the row orhogonalization,
gk+1 . . . gd = hk+1zk+1 . . . zd .
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 31: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/31.jpg)
STRUCTURED ORTHOGONALIZATION
∀ TT decomposition a(i1 . . . id) =d∏
s=1gs(αs−1isαs)
∃ column qk and row zk orthogonal carriages s. t.
a(i1 . . . ik ; ik+1 . . . id) =(k∏
s=1qk(α′s−1isα
′s)
)Hk(α′k , α
′′k)
(d∏
s=k+1zs(α′′s−1isα
′′s )
)
qk and zk can be constructed in dnr 3 operations.
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 32: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/32.jpg)
CONSEQUENCE: STRUCTURED SVDFOR ALL UNFOLDING MATRICESIN O(dnr3) OPERATIONS
It suffices to compute SVD for the matricesHk(α′kα
′′k).
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 33: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/33.jpg)
TENSOR APPROXIMATION VIA MATRIXAPPROXIMATION
We can approximate any fixed unfolding matrix usingits structured SVD:
a(i1 . . . ik ; ik+1 . . . id) = ak + ek
ak = Uk(i1 . . . ik ; α′k)σk(α′k)Vk(α′k ; ik+1 . . . id)
ek = ek(i1 . . . ik ; ik+1 . . . id)
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 34: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/34.jpg)
ERROR ORTHOGONALITY
Uk(i1 . . . ikα′k)ek(i1 . . . ik ; ik+1 . . . id) = 0
ek(i1 . . . ik+1 ; ik+1 . . . id)Vk(α′k ik+1 . . . id) = 0
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 35: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/35.jpg)
COROLLARY OF ERROR ORTHOGONALITY
Let ak be further approximated by a TT but so thatuk or vk are kept. Then the further error, say el , isorthogonal to ek . Hence,
||ek + el ||2F = ||ek ||2F + ||el ||2F
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 36: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/36.jpg)
TENSOR-TRAIN ROUNDING
Approximate successively A1,A2, . . . ,Ad−1
with the error bound ε. Then
FINAL ERROR 6√
d − 1 ε
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 37: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/37.jpg)
TENSOR INTERPOLATION
Interpolate an implicitly given tensor by a TT usingonly small part of its elements, of order dnr 2.
Cross interpolation method for tensors is constructedas a generalization of the cross method for matrices(1995) and relies on the maximal volume principlefrom the matrix theory.
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 38: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/38.jpg)
MAXIMAL VOLUME PRINCIPLE
THEOREM (Goreinov, Tyrtyshnikov) Let
A =
[A11 A12
A21 A22
],
where A11 is a r × r block with maximal determinantin modulus (volume) among all r × r blocks in A.Then the rank-r matrix
Ar =
[A11
A21
]A−1
11
[A11 A12
]approximates A with the Chebyshev-norm error atmost in (r + 1)2 times larger than the error of bestapproximation of rank r .
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 39: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/39.jpg)
BEST IS AN ENEMY OF GOOD
Move a good submatrix M in A to the upper r × rblock. Use right-side multiplications by nonsingularmatrices.
A =
1. . .
1ar+1,1 ... ar+1,r
... ... ...an1 ... anr
NECESSARY FOR MAXIMAL VOLUME:|aij | 6 1, r + 1 6 i 6 n, 1 6 j 6 r
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 40: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/40.jpg)
BEST IS AN ENEMY OF GOOD
COROLLARY OF MAXIMAL VOLUME
σmin(M) > 1/√
r(n − r) + 1
ALGORITHMI If |aij | > 1 + δ, then swap rows i and j .I Make identity matrix in the first r rows byright-side multiplication.
I Quit if |aij | < 1 + δ for all i , j . Otherwise repeat.
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 41: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/41.jpg)
MATRIX CROSS ALGORITHM
I Given initial column indices j1, ..., jr .I Find good row indices i1, ..., ir in these columns.I Find good column indices in the rows i1, ..., ir .I Proceed choosing good columns and rows untilthe skeleton cross approximations stabilize.
E.E.Tyrtyshnikov, Incomplete crossapproximation in the mosaic-skeleton method,Computing 64, no. 4 (2000), 367–380.
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 42: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/42.jpg)
CROSS TENSOR-TRAIN INTERPOLATIONLet a1 = a(i1, i2, i3, i4). Seek crosses in the unfolding matrices.On input: r initial columns in each. Select good rows.
A1 = [a(i1 ; i2, i3, i4)], J1 = {i (β1)2 i (β1)
3 i (β1)4 }
A2 = [a(i1, i2 ; i3, i4)], J2 = {i (β2)3 i (β2)
4 }
A3 = [a(i1, i2, i3 ; i4)], J3 = {i (β3)4 }
rows matrix skeleton decompositionI1 = {i (α1)
1 } a1(i1 ; i2, i3, i4) a1 =∑α1
g1(i1;α1) a2(α1; i2, i3, i4)
I2 = {i (α2)1 i (α2)
2 } a2(α1, i2 ; i3, i4) a2 =∑α2
g2(α1, i2; α2) a3(α2, i3; i4)
I3 = {i (α3)1 i (α3)
2 i (α3)3 } a3(α2, i3 ; i4) a3 =
∑α3
g3(α2, i3; α3) g4(α3; i4)
Finally
a =∑
α1,α2,α3,α4
g1(i1, α1) g2(α1, i2, α2) g3(α2, i3, α3) g4(α3, i4)
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 43: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/43.jpg)
QUANTIZATION OF DIMENSIONS
Increase the number of dimensions.
E.g. 2× . . .× 2.
Extreme case is conversion of a vector of size N = 2d
to a d -tensor of size 2× 2× . . .× 2.
Using TT format with bounded TT ranks may reducethe complexity from O(N) to as little as O(log2 N).
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 44: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/44.jpg)
EXAMPLES OF QUANTIZATION
f (x) is a function on [0, 1]
a(i1, . . . , id) = f (ih), i =i12
+i222 + · · ·+ id
2d
The array of values of f is viewed as a tensor of size2× · · · × 2.
EXAMPLE 1. f (x) = ex + e2x + e3x
ttrank= 2.7 ERROR=1.5e-14
EXAMPLE 2. f (x) = 1 + x + x2 + x3
ttrank= 3.4 ERROR=2.4e-14
EXAMPLE 3. f (x) = 1/(x − 0.1)
ttrank= 10.1 ERROR=5.4e-14
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 45: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/45.jpg)
THEOREMS
If there is an ε-approximation with separated variables
f (x + y) ≈r∑
k=1
uk(x)vk(y), r = r(ε),
then a TT exists with error ε and TT-ranks 6 r .
If f (x) is a sum of r exponents, then an exact TTexists with the ranks r .
For a polynomial of degree m an exact TT existswith the ranks r = m + 1.
If f (x) = 1/(x − δ) then r = log ε−1 + log δ−1.
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 46: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/46.jpg)
ALGEBRAIC WAVELET FILTERS
a(i1 . . . id) = u1(i1α1)a1(α1i2 . . . id) + e1
u1(i1α1)u(i1α′1) = δ(α1, α′1)
a→ a1 = u1a→ a2 = u2a1 → a3 = u3a2 . . .
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 47: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/47.jpg)
TT QUADRATURE
I (d) =
∫[0,1]d
sin(x1 + x2 + . . .+ xd) dx1dx2 . . . dxd =
Im∫
[0,1]de i(x1+x2+...+xd) dx1dx2 . . . dxd = Im
((e i − 1
i
)d)
n nodes in each dimension ⇒ nd values in need!TT interpolation method uses only small part (n = 11)
d I (d) Relative Error Timing500 -7.287664e-10 2.370536e-12 4.641000 -2.637513e-19 3.482065e-11 11.602000 2.628834e-37 8.905594e-12 33.054000 9.400335e-74 2.284085e-10 105.49
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 48: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/48.jpg)
QTT QUADRATURE
∫ ∞0
sinxx
dx =π
2
Truncate the domain and use the rule of rectangles.
Machine accuracy causes to use 277 values.The vector of values is treated as a tensor of size2× 2× . . .× 2.
TT-ranks 6 12 for the machine precision.Less than 1 sec on notebook.
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 49: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/49.jpg)
TT IN QUANTUM CHEMISTRY
Really many dimensions are natural in quantummolecular dynamics:
HΨ = (−12
∆ + V (R1, . . . ,Rf ))Ψ = EΨ
V is a Potential Energy Surface (PES)
Calculation of V requires to solve Schredingerequation for a variety of coordinates of atomsR1, . . . ,Rf . TT interpolation method uses only smallpart of values of V from which it produces a suitableTT approximation of PES.
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 50: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/50.jpg)
TT IN QUANTUM CHEMISTRY
Henon-Heiles PES:
V (q1, . . . , qf ) =12
f∑k=1
q2k + λ
f−1∑k=1
(q2
kqk+1 −13q3
k
)
TT-ranks and timings (Oseledets-Khoromskij)Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 51: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/51.jpg)
SPECTRUM IN THE WHOLE
Use the evolution in time:
∂Ψ
∂t= iHΨ, Ψ(0) = Ψ0.
Physical scheme reads Ψ(t) = e iHtΨ0, then we findthe autocorrelation function a(t) = (Ψ(t),Ψ0) andits Fourier transform.
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 52: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/52.jpg)
SPECTRUM IN THE WHOLE
Henon-Heilse spectra for f = 2 and differentTT-ranks.
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 53: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/53.jpg)
SPECTRUM IN THE WHOLE
Henon-Heiles spectra for f = 4 and f = 10.
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 54: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/54.jpg)
TT FOR EQUATIONS WITH PARAMETERS
Diffusion equation on [0, 1]2. The diffusioncoefficients are constant in each of p × p squaresubdomains, i.e. p2 parameters varing from 0.1 to 1.
256 points in each of parameters, space grid of size256× 256. The solution for all values of parametersis approximated by TT with relative accuracy 10−5:
Number of parameters Storage4 8 Mb16 24 Mb64 78 Mb
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 55: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/55.jpg)
WTT FOR DATA COMPRESSION
f (x) = sin(100x)A signal on uniform grid with the stepsize 1/2d on0 6 x 6 1 converts into a tensor of size2× 2× . . .× 2 with all TT-ranks = 2.The Dobechis transform gives much more nonzeros:
ε storage(WTT) storage forfilters
storage(D4) storage(D8)
10−4 2 152 3338 88010−6 2 152 19696 201010−8 2 152 117575 657010−10 2 152 845869 1570310−12 2 152 1046647 49761
sin(100x), n = 2d , d = 20
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 56: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/56.jpg)
WTT FOR COMPRESSION OF MATRICESWTT for vectorized matrices applies after reshaping:
a(i1 . . . id ; j1 . . . jd) → a(i1j1 ; . . . ; id jd).
WTT compression with accuracy ε = 10−8 for theCauchy-Hilbert matrix
aij = 1/(i − j) for i 6= j , aii = 0.
n = 2d storage(WTT) storage(D4) storage(D8) storage(D20)25 388 992 992 99226 752 4032 3792 334827 1220 15750 13246 866228 1776 59470 41508 2097029 2260 213392 102078 45638210 2744 780590 215738 95754211 3156 1538944 306880 176130
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 57: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/57.jpg)
TT IN DISCRETE OPTIMIZATION
Among all elements of a tensor given by TT findminimum or maximum. Discrete optimizationproblem is solved a an eigenvalue problem fordiagonal matrices. Block minimization of Raleighquotient in TT format, blocks of size 5, TT-ranks6 5 (O.S.Lebedeva).
Function Domain Size Iter. (Ax , x) (Aei , ei )ei ≈ x
Exactmax
3Qi=1
(1+0.1 xi +sin xi ) [1, 50]3 215 30 428.2342 429.2342 429.2342
same [1, 50]3 230 50 430.7838 430.78453Q
i=1(x + sin xi ) [1, 20]3 215 30 8181.2 8181.2 8181.2
same [1, 20]3 230 50 8181.2 8181.2
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 58: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/58.jpg)
CONCLUSIONS AND PERSPECTIVES
I TT algorithms (http://pub.inm.ras.ru) areefficient new instruments for compression ofvectors and matrices. Storage and complexitydepend on matrix size logarithmically.
I Free access to a current version of TT-library:http://spring.inm.ras.ru/osel.
I There are some theorems with TT-rankestimates. Sharper and more general estimatesare to be derived. Difficulty is in nonlinearity ofTT decompositions.
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 59: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/59.jpg)
CONCLUSIONS AND PERSPECTIVES
I TT interpolation methods provide new efficientmethods for tabulation of functions of manyvariables, also those that are hard to evaluate.
I There are examples of application of TT methodsfor fast and accurate computation ofmultidimensional integrals.
I TT methods are successfully applied to imageand signal processing and may compete withother known methods.
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA
![Page 60: NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS …...NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING EugeneTyrtyshnikov Institute of Numerical Mathematics of Russian](https://reader030.vdocuments.us/reader030/viewer/2022041118/5f2f2990e10fc35953013da8/html5/thumbnails/60.jpg)
CONCLUSIONS AND PERSPECTIVES
I TT methods are a good base for numericalsolution of multidimensional problems ofquantum chemistry, quantum moleculardynamics, optimization in parameters, modelreduction, multiparametric and stochasticdifferential equations.
Eugene Tyrtyshnikov NUMERICAL METHODS WITH TENSOR DATA