federico tessmann, ferienakademie tum efficiant polynomial interpolation algorithms
TRANSCRIPT
Federico Tessmann, Ferienakademie TUM
Efficiant polynomial interpolation algorithms
Federico Tessmann, Ferienakademie TUM
Overview
Introduction to Vandermonde Matrices and its utilities
Univariate Interpolation
Multivariate Interpolation
Federico Tessmann, Ferienakademie TUM
Properties of Vandermonde Matrices
Easy to ensure that they are non-singular
Systems of linear equations whose coefficients form Vandermonde matrices are easy to solve exactly
Federico Tessmann, Ferienakademie TUM
The Vandermonde Matrix
1
122
111
1
1
1
nnn
n
n
n
kk
kk
kk
V
Federico Tessmann, Ferienakademie TUM
Generalized Vandermonde
n
n
n
en
en
en
eee
eee
n
kkk
kkk
kkk
V
21
21
21
222
111
neee 21where
Federico Tessmann, Ferienakademie TUM
Determinant of a Vandermonde
121
1
12
21
212
1
122
111
1
1
001
1
1
1
det
kkkkk
kkkkk
kk
kk
kk
V
nn
nnn
nn
nnn
n
n
n
Federico Tessmann, Ferienakademie TUM
Determinant of a Vandermonde
njjn
nn
nnn
nn
kkV
kkkkk
kkkkk
111
121
1
12
21
212 )(det
1
1
001
nj
jnj
jnnj
jnn kkkkVkkVV1
12
221
11 )()(det)(detdet
nji
ijn kkV1
)(det
Federico Tessmann, Ferienakademie TUM
Determinant of a Vandermonde
nji
ijn kkV1
)(det
The Vandermonde matrix
is non-singular the ki are distinct
Federico Tessmann, Ferienakademie TUM
The previous result can not be applyed for generalized Vandermonde matrices
Example
wich is 0 also when
ee
e
e
kkk
kV 12
2
12
1
1det
11
2
e
k
k
Federico Tessmann, Ferienakademie TUM
Non-singularity of generalized Vandermonde matrices
Proposition 1:
If the ki are distinct positiv real numbers
=> the matrix is non-zero
Federico Tessmann, Ferienakademie TUM
The inverse of a Vandermonde matrix
)()()(
)()()(
)()()(
*
1
1
1
21
22221
11211
21
22221
11211
1
122
111
nnnn
n
n
nnnn
n
n
nnn
n
n
kPkPkP
kPkPkP
kPkPkP
ppp
ppp
ppp
kk
kk
kk
12321)( n
injijijjij kpkpkppkP
Federico Tessmann, Ferienakademie TUM
The inverse of a Vandermonde matrix
)()()(
)()()(
)()()(
21
22221
11211
nnnn
n
n
n
kPkPkP
kPkPkP
kPkPkP
E
ni
ji ij
ij kk
kZZP
1
)(
Federico Tessmann, Ferienakademie TUM
Solving a Vandermonde system of equations
nnn
nnn
nn
nn
wxkxkxkx
wxkxkxkx
wxkxkxkx
13
221
21
232
2221
11
132
1211
Federico Tessmann, Ferienakademie TUM
Solving a Vandermonde system of equations
Tnxxx ),,( 1
T
nwww ),,( 1
wxVn
wVxVV nnn
11
wVx n
1
Federico Tessmann, Ferienakademie TUM
Solving a Vandermonde system of equations
nnnnn
n
n
n w
w
w
ppp
ppp
ppp
x
x
x
1
1
21
22221
11211
2
1
niniii wpwpwpx 2211
ni
ni
i wZZPcoefwZZPcoefx )),(()),(( 11
11
11)( n
njjj ZppZP
Federico Tessmann, Ferienakademie TUM
The algorithm to solve the system
ni
ikZZQ1
)()(
ii kZ
ZQZQ
)()(
)(
)()(
ii
ii kQ
ZQZP
Federico Tessmann, Ferienakademie TUM
The algorithm to solve the systemThe computation of the xi is arranged as follows:
),(
),(
),(
),(
),(
),(
1
1
0
111
111
011
2
1
nnn
nn
nn
nn ZPcoefw
ZPcoefw
ZPcoefw
ZPcoefw
ZPcoefw
ZPcoefw
x
x
x
Calculate each vector and add it to the accumulating X
Federico Tessmann, Ferienakademie TUM
Analysis of the algorithm
1. By calculating the vectors one after the other we only need to compute one Pi(Z) at the time
2. Each Pi(Z) only needs O(n) time and since we have n polinoms to compute, the complexity is O(n2) and the space needed is O(n)
3. Because the inverse of the transposed matrix is the transpose of the inverse of the matrix, the algorithm only need a little adjustment to solve a transposed Vandermonde system of equations
On the Appendix there is an example of this alorithm taken from Zippel
Federico Tessmann, Ferienakademie TUM
Univariate Interpolation
Lagrange Interpolation
Newton Interpolation
Abstract Interpolation
Federico Tessmann, Ferienakademie TUM
Lagrange Interpolation
nnZpZppZP 10)(
nzzz ,,, 10
Giving are a set of distinct evaluation points with its correspondating functional values
ii wzP )(
The goal is to find the polinome
Federico Tessmann, Ferienakademie TUM
Lagrange Interpolation
nn
nnnn
nn
nn
wzpzpzpp
wzpzpzpp
wzpzpzpp
2210
112
12110
002
02010
ni
ni
i wZZPcoefwZZPcoefp )),(()),(( 00
This is a Vandermonde system where
Federico Tessmann, Ferienakademie TUM
Lagrange Interpolation
n
nn
nn
nn
nn
ZwZZPcoefwZZPcoef
ZwZZPcoefwZZPcoef
wZZPcoefwZZPcoefZP
))),(()),(((
))),(()),(((
)),(()),(()(
00
10
10
00
00
nn wZPwZPZP )()()( 00
Federico Tessmann, Ferienakademie TUM
Lagrange Interpolation
nn wZPwZPZP )()()( 00
nj
j
ji
ni ij
i wzz
zZZP
0 0
)(
Federico Tessmann, Ferienakademie TUM
Newton Interpolation
f(a)=f(x)(mod (x-a))
Federico Tessmann, Ferienakademie TUM
The Chinese remainder algorithm over ZZ
)(mod
)(mod
22
11
akx
akx
)(mod)mod()( 21121
1121 aaaaakkkx
1),gcd(mod 2121
1 aaaa
Federico Tessmann, Ferienakademie TUM
Chinese remainder with polinoms
When given and nzz ,,1 ii wzf Then we change it to the following situation:Given
Compute
)(mod)( ii pwXf ii zxp
))(mod()( 10 nn
n ppaxaxf
Federico Tessmann, Ferienakademie TUM
Newton Interpolation algorithm
1. Let f(x)=0, q(x)=1
2. Loop for n times doing following:1. f(x)=f(x)+q(ki)-1q(x)(wi-f(ki))
2. q(x)=(x-ki)q(x)
Federico Tessmann, Ferienakademie TUM
Newton´s interpolation formula
Let
Newton´s interpolation formula claims that there exist constants such that
In fact, and is the solution of
)( ii kff
))(()()( 212110 kxkxkxxf
110 f
i
)( 12112 kkff
Federico Tessmann, Ferienakademie TUM
Newton´s interpolation formula
)()(
)()())(()()(
11
111
1)(
lll
llll
ll
kkkk
kxkxkffxfxf
)()( 112
121
)2( kXkk
fffxf
)()( 111 llllll kkkkff
Then
And more generally
Solving the givesi
Federico Tessmann, Ferienakademie TUM
Multivariate Interpolation
Dense Interpolation
Probabilistic Sparse Interpolation
Deterministic Sparse Interpolation without degree bounds
Federico Tessmann, Ferienakademie TUM
Multivariate dense InterpolationWe are given a black box with a degree bound „d“ for
the polinom P(xi,..,xn)
So we can assume that P has the form
),,(),,(),,( 21201 ndd
nn XXPXXXPXXP
P
),,,(
),,,(
),,,(
0201
0201
02010
n
ndP
nP
xxXP
xxx
xxx
Federico Tessmann, Ferienakademie TUM
Multivariate dense Interpolation
So we get the values of
which are the coeficients found by interpolating P on X1
By doing this procedure we compute recursively P(X1,...,Xk,x(k+1)0,...,xn0)
),,( 020 ni XxP
Federico Tessmann, Ferienakademie TUM
Multivariate dense Interpolation
),,,,(
),,,(
),,,(
),,,(
),,,(
),,,(
),,,(
03021
021
021
0210
0201
0201
02010
n
nd
nddP
ndP
n
ndP
nP
xxXXP
xxXP
xxx
xxx
xxXP
xxx
xxx
Federico Tessmann, Ferienakademie TUM
The complexity of the dense interpolationLet I(d) be the complexity of interpolating d+1 values to produce a
univariate plynomial of degree „d“ and Nk the complexity for the first k variables
)()1(
)()1()1(
)()1()1(
)(
1
11
12
1
dIdkN
dIdNdN
dIdNdN
dIN
kk
kkk
Federico Tessmann, Ferienakademie TUM
Probabilistic Sparse Interpolation
Formal Presentation
Example
Analysis
Federico Tessmann, Ferienakademie TUM
Probabilistic Sparse InterpolationAssume we want to dermine P(X1,..., Xn) which is an element of L[X]
where L is a field of cardinal q and the degree of each Xi is bounded by „d“ and there are no more than T non-zero monomials
TeT
en XcXcXXP
1
11 ),,(
),,(),,,,,(
,,,,,,
),,(
:
111
11
1
nknkk
kkk
T
XXPskelxxXXskelP
eeeskelPeeePskel
eeskelP
Def
Federico Tessmann, Ferienakademie TUM
Probabilistic Sparse Interpolation
),,(),,,,,(,
),,(
111
1
nknkk
n
XXPskelxxXXskelPk
xx
Def:
is a precise evaluation point if:
Federico Tessmann, Ferienakademie TUM
Probabilistic Sparse Interpolation
Tkk enkTk
enkkn XXXcXXXcXXP
),,(),,(),,( 1111
1
q
dTnn
q
dT
q
dTn
2
)1()1(
q
dTkn )(
The probability by wich is an imprecise evaluation point:
For each k we can write
It is an imprecise evaluation point if one of the cik = 0And the probability that this happends is no more than
),,( 1 nxx
Federico Tessmann, Ferienakademie TUM
Probabilistic Sparse Interpolation
Given is a k-1 tuple The probability that
is 0 if we are we are working on a field of characteristic 0 or at least
When working on a field of q elements the
probability is bounded by
),,( 11 kyyy
jiee eeyy ji
,
dkp )32( 1
q
dTT
2
)1(
Federico Tessmann, Ferienakademie TUM
Probabilistic Sparse Interpolation
So the following probability is then one that underlines the Probabilistic Sparse Interpolation
q
ntdT
q
dTnn
q
dTT
2
)2(
2
)1(
2
)1( 2
Federico Tessmann, Ferienakademie TUM
Probabilistic Sparse InterpolationAssume we want to dermine P(X1,..., Xn) which is an element of L[X] where L is a field of cardinal q
and the degree of each Xi is bounded by „d“ and there are no more than T non-zero monomials
As in the dense interpolation we Interpolate TeT
en XcXcXXP
1
11 ),,(
),,(),,(),,( 21201 ndd
nn XXPXXXPXXP
),,,(
),,,(
),,,(
0201
0201
02010
n
ndP
nP
xxXP
xxx
xxx
Federico Tessmann, Ferienakademie TUM
Probabilistic Sparse Interpolation
q
dTknkn
2
))(1(1
TeT
enkk XpXpxxXXP
010,00,11
1),,,,(
kk eePskel
,,1
At the kth stage the first computation gives us:
We then assume that
The probability of that being the right skeleton is
We then pick a (k-1) tuple
And we set up the following transposed Vandermonde system of linearecuations
),,( 11 kyyy
Federico Tessmann, Ferienakademie TUM
Probabilistic Sparse Interpolation
T
T
T
eTTj
eTjkj
Tk
T
eTj
ejkjk
eTj
ejkjk
Tjjkj
ypypxyyP
ypypxyyP
ypypxyyP
ppxP
1
1
1
111
221
21
21
111
1
),,,(
),,,(
),,,(
),1,,1(
So each of the can be computed using O(n2) and we can avoid computing the other interpolations
ijp
Federico Tessmann, Ferienakademie TUM
Probabilistic Sparse Interpolation
The probability that the Vandermonde system of equation is non-singular is bounded by
q
dTT
2
)1(1
Federico Tessmann, Ferienakademie TUM
Probabilistic Sparse Interpolation
So we get
for each k
Then we solve
trough the dense interpolation
We then expand it and we get
And we are ready to compute the (k+1)th stage
TeTj
ejkjk XpXpxXXP
11,11 ),,(
)( kii XPp
TeT
enkk XpXpxxXXP
01000)1(,1
0),,,,(
Federico Tessmann, Ferienakademie TUM
Probabilistic Sparse InterpolationExample
Lets assume we are given a Black Box representing the following polinom
ZYXYZXYZXZXZYXP 554525),,(
532
41
50
17165
15
052
050
545
3
012
010
215
0
005
000
)(
),,(
),,(
),,(
),,(),,(
),,(
YdXYdYdXd
cXcXc
zyx
zyx
cXcXc
zyx
zyx
cXcXc
zyx
zyx
P
P
P
P
P
P
Federico Tessmann, Ferienakademie TUM
Deterministic Sparse Interpolation without degree bounds
Given are a bound on the number of non-zero terms „T“ and the number of variables „n“
We want to compute
By choosing a distinct prime for each Xi then the quantities will all be distinct.
Let
Then we get:
TeT
e XcXcXP
11)(
1ei Xm
jj
TTj vmcmc 11
),,( 1j
njj xxp
)( j
j pPv
Federico Tessmann, Ferienakademie TUM
Deterministic Sparse Interpolation without degree bounds
111
11
111
11
01
tt
TTt
TT
T
vmcmc
vmcmc
vcc
The rank of the system of equations is exactly the number of non-zero monomials in P
This could be easily done by taking the first T equations and computing their rank which requires O(T3)
Federico Tessmann, Ferienakademie TUM
Deterministic Sparse Interpolation without degree bounds
Let
and consider
so
consider also
01 )()()( qZqmZmZZQ ttt
iti
itt
ij
tjj qmcmcmQc )()(
111
1
00110 qvqvv ttt
iti
kitt
kij
kj
tjj qmcmcmQmc )()(
111
1
Federico Tessmann, Ferienakademie TUM
Deterministic Sparse Interpolation without degree bounds
Then we get the following Toeplitz system of linear equations
1212201
1101
1100
tttt
ttt
ttt
vqvqv
vqvqv
vqvqv
Federico Tessmann, Ferienakademie TUM
Deterministic Sparse Interpolation without degree bounds
1
122
111
2
1
112
11
21
221
21
110
1
1
1
00
00
00111
ttt
t
t
ttt
tt
t
ttt
t
t
mm
mm
mm
c
c
c
mmm
mmm
vvv
vvv
vvv
So the system is non-singular if the mi are distinct
Federico Tessmann, Ferienakademie TUM
Deterministic Sparse Interpolation without degree bounds
1212201
1101
1100
tttt
ttt
ttt
vqvqv
vqvqv
vqvqv
•So the system can be solved by Gaussian elimination O(t3)
•So then we get Q(Z)
•In order to find the mi we just need to find the zeroes of Q which are in fact positive integers making this procedure much easier
•Knowing the mi, the ei can easily be determined by factoring each of the mi, which is in fact very easy because the possible divisors are the first n primes allready known
•By knowing the mi it is allso easy to compute the ci just by solving the Vandermonde system, formed by the first t equations
Federico Tessmann, Ferienakademie TUM
The End
Questions?
Bibliography: Richard Zippel
Federico Tessmann, Ferienakademie TUM
Appendix
pseudo-codes for the interpolation algorithms
Federico Tessmann, Ferienakademie TUM
Code for Vandermonde Matrices
);(
}
}
);),((][][:][
{1:
]);[(/)(:)(
]);[),((:)(
{1:
];0;...;0[][:[]
]);[(])1[(:)(
]){[],[(
1
xreturn
ZZPcoefiwjxjx
dontojfor
ikQZQZP
ikZZQntPolyQuotieZQ
dontoifor
nnewArrayx
nkZkZZQ
nwnkrmondeSolveVande
ji
iii
i
Federico Tessmann, Ferienakademie TUM
Code for Lagrange Interpolation
);(
}
);(][:
]);[(/)(:)(
]);[),((:)(
{1:
;0:
]);[(])1[(:)(
]){[],[(
freturn
ZPiwff
ikQZQZP
ikZZQntPolyQuotieZQ
dontoifor
ialnewPolinomf
nkZkZZQ
nwnknterpolatioLagrangeIn
i
iii
i
Federico Tessmann, Ferienakademie TUM
Code for Newton Interpolation
);(
}
;])[(:
]));[(][()(])[(:
{1:
;0:
;1:
]){[],[(
1
freturn
qikXq
ikfiwXqikqff
dontoifor
ialnewPolinomf
ialnewPolinomq
nwnkrpolationNewtonInte