chapter 1 divide and conquer - uni-freiburg.deac.informatik.uni-freiburg.de/teaching/ws14_15/... ·...

Chapter 1Divide and ConquerPolynomial Multiplication

Algorithm TheoryWS 2014/15

Fabian Kuhn

Algorithm Theory, WS 2014/15 Fabian Kuhn 2

Representation of PolynomialsCoefficient representation:

• Polynomial ∈ of degree is given by its1 coefficients , … , :

⋯

• Example:3 15 18

• The most typical (and probably most natural) representation of polynomials


Representation of PolynomialsPoint‐value representation:

• Polynomial ∈ of degree is given by1 point‐value pairs:

, , , , … , ,

where for .

• Example: The polynomial

3 2 3is uniquely defined by the four point‐value pairs 0,0 , 1,6 , 2,0 , 3,0 .


Operations: Coefficient RepresentationDeg.‐ polynomials ⋯ , ⋯

Addition:⋯

• Time:

Multiplication:

⋅ ⋯ , where

• Naive solution: Need to compute product for all 0 ,

• Time:


Operations Point‐Value RepresentationDegree‐ polynomials

, , … , , , , , … , ,

• Note: we use the same points , … , for both polynomials

Addition:

, , … , ,

• Time:

Multiplication:

⋅ , ⋅ , … , , ⋅

• Time:


Faster Polynomial Multiplication?Multiplication is fast when using the point‐value representation

Idea to compute ⋅ (for polynomials of degree ):

, of degree 1, coefficients

2 2 point‐value pairs , and ,

2 point‐value pairs ,

of degree 2 2, 2 1 coefficients

Evaluation at points , , … ,

Point‐wise multiplication

Interpolation


Point‐Value Representation of , • Select points , , … , to evaluate and in a clever way

Consider the powers of the principle th root of unity:

Principle root of unity:

1, 1

Powers of (roots of unity):

1 , , … ,

Note: cos ⋅ sin


Discrete Fourier Transform• The values for 0,… , 1 uniquely define a

polynomial of degree .

Discrete Fourier Transform (DFT):• Assume , … , is the coefficient vector of poly.

⋯

DFT ≔ , ,… ,


Faster Polynomial Multiplication?






Evaluation at points , , … ,


Interpolation


Properties of the Roots of Unity• Cancellation Lemma:

For all integers 0, 0, and 0, we have:

• Proof:


Divide‐and‐Conquer Approach• Divide of degree 1 ( is even) into 2 polynomials of

degree ⁄ 1 differently than in Karatsuba’s algorithm

• ⋯ ⁄ (even coeff.)⋯ ⁄ (odd coeff.)


Discrete Fourier TransformEvaluation for 0,… , 1:

⋅

⁄ ⋅ ⁄ if 2

⁄⁄ ⋅ ⁄

⁄ if 2

For the coefficient vector of :

DFT ⁄ ,… , ⁄⁄ , ⁄ , … , ⁄

⁄

⁄ , … , ⁄⁄⁄ , ⁄

⁄ , … , ⁄⁄


ExampleFor the coefficient vector of :

DFT ⁄ ,… , ⁄⁄ , ⁄ , … , ⁄

⁄

⁄ , … , ⁄⁄⁄ , ⁄

⁄ , … , ⁄⁄

4:

Need: , and ,

(DFTs of coefficient vectors of and )


Recursive StructureFor simplicity, we abuse notation in the following:

• Poly. ⋯ with coefficient vector Let DFT ≔ DFT

Recursive structure: • For 4:

DFT DFT ⋅ DFT

• General (assume is even): DFT

DFT /

⋅ DFT /


Computation of • Divide‐and‐conquer algorithm for DFT :

1. Divide

1: DFT

1: Divide into (even coeff.) and (odd coeff).

2. Conquer

Solve DFT ⁄ and DFT ⁄ recursively

3. Combine

Compute DFT based on DFT ⁄ and DFT ⁄


Analysis• : time to compute DFT :

2 2 , 1 1

• As for mergesort, comparing orders, closest pair of points:

⋅ log


Small ImprovementPolynomial of degree 1:

⁄ ⋅ ⁄ if 2

⁄⁄ ⋅ ⁄

⁄ if 2

⁄ ⋅ ⁄ if 2

⁄⁄ ⁄ ⋅ ⁄

⁄ if 2

Need to compute ⁄ and ⋅ ⁄ for 0 ⁄ .


Example

⋅

⋅

⋅

⋅


Fast Fourier Transform (FFT) AlgorithmAlgorithm FFT(a)• Input: Array of length , where is a power of 2• Output: DFT

if 1 then return ; // ≔ FFT , , … , ;≔ FFT , , … , ;≔ ; ≔ 1;

for 0 to ⁄ 1 do //

≔ ⋅ ;≔ ; ⁄ ≔ ;≔ ⋅

end;return , , … , ;


Example• 3 15 18 0, 0,18, 15,3








Evaluation at , , … , using FFT


Interpolation


InterpolationConvert point‐value representation into coefficient representation

Input: , , … , , with for

Output: Degree‐ 1 polynomial with coefficients , … , such that

⋯ ⋯

⋮⋮⋯

linear system of equations for , … ,


InterpolationMatrix Notation:

1 ⋯1 ⋯⋮ ⋮ ⋱ ⋮1 ⋯

⋅ ⋮ ⋮

• System of equations solvable iff for all

Special Case :

1 1 1 ⋯ 11 ⋯1 ⋯⋮ ⋮ ⋮ ⋱ ⋮1 ⋯

⋅⋮ ⋮


Interpolation• Linear system:

⋅ ⟹ ⋅

, , ⋮ , ⋮

Claim:

Proof: Need to show that


DFT Matrix Inverse

⋯ 1

⋯ ⋮ ⋯

⋅

⋯ 1 ⋯⋯ ⋯⋯ ⋯ ⋮ ⋯ ⋯


DFT Matrix Inverse

,

ℓ

ℓ

Need to show ,1if0if

Case :


DFT Matrix Inverse

,

ℓ

ℓCase :


Inverse DFT

•

⋯

⋯ ⋮ ⋯

• We get ⋅ and therefore

1⋯ ⋅ ⋮

1⋅ ⋅


DFT and Inverse DFTInverse DFT:

1⋅ ⋅

• Define polynomial ⋯ :

1⋅

DFT:• Polynomial ⋯ :


DFT and Inverse DFT

⋯ , ⋅ :

• Therefore:, , … ,

1⋅ , , , … ,

1⋅ , , , … ,

• Recall:DFT , , , … ,

⋅ , , , … , ,


DFT and Inverse DFT• We have DFT ⋅ , , , … , , :

1⋅ DFT if 0

1⋅ DFT if 0

• DFT and inverse DFT can both be computed using FFT algorithm in log time.

• 2 polynomials of degr. can be multiplied in time log .








Evaluation at , , … , using FFT


Interpolation using FFT


Convolution• More generally, the polynomial multiplication algorithm

computes the convolution of two vectors:

, , … ,, , … ,

∗ , , … , ,

where, :

,

• is exactly the coefficient of in the product polynomial of the polynomials defined by the coefficient vectors and


More Applications of ConvolutionsSignal Processing Example:• Assume , … , represents a sequence of

measurements over time• Measurements might be noise and have to be smoothed out• Replace by weighted average of nearby last and next

measurements (e.g., Gaussian smoothing):

1⋅

• New vector ′ is the convolution of and the weight vector1⋅ , , … , , 1, , … , ,

• Might need to take care of boundary points…


More Applications of ConvolutionsCombining Histograms:• Vectors and represent two histograms• E.g., annual income of all men & annual income of all women

• Goal: Get new histogram representing combined income of all possible pairs of men and women:

∗

Also, the DFT (and thus the FFT alg.) has many other applications!

chapter 1 divide and conquer - uni-freiburg.deac.informatik.uni-freiburg.de/teaching/ws14_15/... ·...

Documents