lecture02 - factorization methods

8/9/2019 Lecture02 - Factorization Methods

1/23

Solution of Linear System of Equations

Lecture 2:

Factorization Methods

MTH2212 Computational Methods and Statistics


2/23

Dr. M. HrairiDr. M. Hrairi MTH2212MTH2212 -- Computational Methods and StatisticsComputational Methods and Statistics 22

Objectives

Introduction

LU Decomposition

Computational complexity

The Matrix Inverse

Extending the Gaussian Elimination Process


3/23


Introduction

Provides an efficient way to compute matrix inverse by

separating the time consuming elimination of the Matrix [A]

from manipulations of the right-hand side {B}.

Gauss elimination, in which the forward elimination

comprises the bulk of the computational effort, can be

implemented as an LU decomposition.


4/23


LU Decomposition

The matrix [A] for the linear system [A]{X}={B} is factorizedinto the product of two matrices [L] and [U] (L- lower triangularmatrix and U- upper triangular matrix)

[L][U]=[A]

[L][U]{X}={B}

Similar to first phase ofGauss elimination, consider

[U]{X}={D}

[L]{D}={B}

The solution can be obtained by1. First solve [L]{D}={B} to generate an intermediate vector{D}by

forward substitution

2. Then, solve [U]{X}={D} to get {X}by back substitution.


5/23


LU Decomposition


6/23


LU Decomposition

In matrix form, this is written as

How to obtain the triangular factorization?

Use Gauss elimination and store the multipliers mij as the

subdiagonal entries in [L]

!

-

3

2

1

3

2

1

333231

232221

131211

b

b

b

x

x

x

aaa

aaa

aaa

-

-

!

333231

23

2221

131211

100

010

001

aaa

aaa

aaa

A


7/23


LU Decomposition

The multipliers are

The triangular factorization of matrix [A]

A = [L] [U]

-

-

!33

"

3''

3

33 0000

00

a

aa

aaa

mm

mA

11

21

21

a

am !

11

1

1

a

am !

22'

32'

32a

a!


8/23


Example 1

Use LU decomposition to solve:

3x1 0.1x2 0.2x3 = 7.85

0.1x1 + 7x2 0.3x3 = -19.3

0.3x1 0.2x2 + 10x3 = 71.4

use 6 significant figures in your computation.


9/23

Dr. M. HrairiDr. M. Hrairi MTHMTH22122212 -- Computational Methods and StatisticsComputational Methods and Statistics 99

Example 1 - Solution

In matrix form

The multipliers are

-

4.71

3.19

85.7

10.03.0

3.071.0

.01.03

3

1

x

x

x

0333333.03

1.021 !!m

100000.03

3.0

31!!

m

0271300.000333.7

19.0

32

m


10/23



The LU decomposition is

The solution can be obtained by

1. First solve [L]{D}={B} for{D}by forward substitution

-

-

!0120.1000

293333.000333

.70

2.01.03

10271 00.0100000.0

01

0333333.0

001

A

!

-

4.71

3.19

85.7

10271300.0100000.0

010333333.0

001

3

2

1

d

d

d


11/23



Then, solve [U]{X}={D} to get {X}by back substitution.

84.56.9.85..4.

56.985...9

85.

!!

!!

!

d

d

d

!

-

0843.70

5617.19

85.7

01 0.1000

93333.000333.70

.01.03

3

1

x

x

x

33/))00003.7(.0)5.(1.085.7(

5.00333.7/))00003.7(93333.05617.19(

00003.701 0.10/0843.70

1

3

!!

!!

!!

x

x

x


12/23


The triangular factorization portion of[A]=[L][U] requires

(N3-N)/3 multiplications and divisions

(2N3-3N2+N)/6subtractions

Finding the solution to [L][U]{X}={B} requires

N2

multiplications and divisions N2-Nsubtractions

The bulk of the calculation lies in the triangularization portion.

LU decomposition is usually chosen over Gauss elimination when thelinear system is to be solved many times, with the same [A] but withdifferent {B}.

Saves computing time by separating time-consuming elimination step fromthe manipulations of the right hand side.

Provides efficient means to compute the matrix inverse which provides ameans to test whether systems are ill-conditioned

ComputationalComplexity


13/23


Find matrix [A]-1, the inverse of[A], for which

[A][A]-1 = [A]-1 [A]=[I]

The inverse can be computed in a column-by-column

fashion by generating solutions with unit vectors {B}constants.

The solution of[L][U]{X}={B} with will be the firstcolumn of[A]-1

The solution of[L][U]{X}={B} with will be the second

column of[A]-1

The solution of[L][U]{X}={B} with will be the thirdcolumn of[A]-1

The MatrixInverse

_ a

!

0

0

1

B

_ a

!0

1

0

B

_ a

!

1

0

0

B


14/23


Example 2

Use LU decomposition to determine the matrix inverse for

the following system and use it to find the solution:

3x1 0.1x2 0.2x3 = 7.85

0.1x1 + 7x2 0.3x3 = -19.3

0.3x1 0.2x2 + 10x3 = 71.4

use 6 significant figures in your computation.


15/23


Example 2- Solution

In matrix form

The triangular factorization of[A]

? A

-

!

102.03.0

3.071.0

2.01.03

A

-

!

10271300.0100000.0010333333.0

001

L

-

01 0.100093333.000333.70

.01.03

U


16/23


Example 2- Solution

The first column of[A]-1

_ a

!

!

-

1009.0

03333.0

1

0

0

1

10271300.0100000.0

010333333.0

001

3

2

1

d

d

d

_ a

-

01008.0

00518.0

33249.0

1009.0

03333.0

1

0120.1000

293333.000333.70

2.01.03

3

2

1

X


17/23


Example 2- Solution

The second column of[A]-1

_ a

!

!

-

02713.0

1

0

0

1

0

10271300.0100000.0

010333333.0

001

3

2

1

d

d

d

_ a

!

!

-

00271.0

142903.0

004944.0

02713.0

1

0

0120.1000

293333.000333.70

2.01.03

3

2

1

X


18/23


Example 2- Solution

The third column of[A]-1

_ a

!

!

-

1

0

0

1

0

0

10271300.0100000.0

010333333.0

001

3

2

1

d

d

d

_ a

!

!

-

09988.0

004183.0

006798.0

1

0

0

0120.1000

293333.000333.70

2.01.03

3

2

1

X

x

x

x


19/23


Example 2- Solution

The matrix inverse [A]-1 is:

Check your result by verifying that [A][A]-1 =[I]

The final solution is

-

!

09988.000271.001008.0

004183.0142903.000518.0

00 798.0004944.033249.01

A

_ a ? A _ a

!

-

!!

7

50002.23

4.71

3.1985.7

09988.000271.001008.0

004183.0142903.000518.0006

798

.0004944

.033249

.01

BAX


20/23


Extending the Gaussian Elimination Process

If pivoting is required to solve [A]{X}={B}, then there exists

a permutation matrix [P] so that:

[P][A ]=[L][U]

The solution {X} is found in four steps:

1. Construct the matrices [L], [U] and [P].

2. Compute the column vector[P]{B}.

. Solve [L]{D}=[P]{B} for{D} using forward substitution.

. Solve [U]{X}={D} for{X} using back substitution.


21/23


Example 3

Use LU decomposition with permutation to solve the

following system of equations

0.0003 x1 + 3.0000 x2 = 2.0001

1.0000 x1 + 1.0000 x2 = 1.0000


22/23



In matrix form [A ]{X}={B}

We saw previously that pivoting is required to solve this

system of equations, hence [P][A ]=[L][U]

The solution {X} is found in four steps:

1. Construct the matrices [L], [U] and [P].

!

-

1

0001.2

11

30003.0

2

1

x

x

? A

-

!

01

10P ? A

-

10003.0

01L ? A

-

!

9997.20

11U


23/23



2. Compute the column vector[P]{B}.

. Solve [L]{D}=[P]{B} for{D} using forward substitution.

. Solve [U]{X}={D} for{X} using back substitution.

!

-

0001.2

1

1

0001.2

01

10

_ a

!

!

-

8.1

1

0001.2

1

1000.0

01

2

1

_ a

66667.0

33333.0

9998.1

1

9997.20

11

2

1X

x

x

lecture02 - factorization methods

Documents