Advanced Computer Graphics Spring 2009

K. H. Ko

Department of MechatronicsGwangju Institute of Science and Technology

2

Today’s Topics

Linear AlgebraSystems of Linear EquationsMatricesVector Spaces

3

Systems of Linear Equations

Linear Equation

System of Linear Equations (n equations, m unknowns)

bxaxa mn 11

nmn

nmn

mn

bxdxd

bxcxc

bxaxa

11

111

111

4

Systems of Linear Equations

Solve a system of n linear equations in m unknown variables

A common problem in applications In most cases m = n. The system has three cases

No solutions, one solution or infinitely many solutions

How to solve the system? Forward elimination followed by back

substitution

5

Systems of Linear Equations

A closer look at two equations in two unknowns

When the solution method needs to be implemented for a computer, a practical concern is the amount of time required to compute a solution.

2222121

1212111

bxaxa

bxaxa

6

Systems of Linear Equations

Division is more expensive than multiplication and addition.

• 3 additions

• 3 multiplications

• 3 divisions

• 3 additions

• 5 multiplications

• 2 divisions

7

Gaussian Elimination

Forward elimination + back substitution = Gaussian elimination

8

Gaussian Elimination

Basic Operations for Forward Elimination

9

Gaussian Elimination

Basic Operations for Forward Elimination

10

Gaussian Elimination

Basic Operations for Forward Elimination

11

Gaussian Elimination

Basic Operations for Back Substitution

12

Gaussian Elimination

Example

13

Geometry of Linear Systems

Consider 2222121

1212111

bxaxa

bxaxa

021122211 aaaa0

0

121211

21122211

baba

aaaa 0

0

121211

21122211

baba

aaaa

14

Geometry of Linear Systems

Consider 3 equations and 3 unknowns

15

Numerical Issues

If the pivot is nearly zero, the division can be a source of numerical errors.

Use of floating point arithmetic with limited precision is the main cause.

/11

/1

/120

/11

1

1

21

1

16

Numerical Issues

A better algorithm involves searching the entries with the pivot that is largest in absolute magnitude.

1

1

210

21

1

1

1

21

No division by ε. -> Numerically robust and stable.

17

Numerical Issues

However, even the previous approach can be a problem.

Swap columns to avoid such problem.

Blackboard!!!

0,12

1

21212

211

xx

xx

18

Numerical Issues

Generally, for a system of n equations in n unknowns…

Full Pivoting: Search the entire matrix of coefficients looking for the entry of largest absolute magnitude to be used as the pivot.

If that entry occurs in row r and column c, then rows r and 1 are swapped followed by a swap of column c and column 1.

After both swaps, the entry in row 1 and column 1 is the largest absolute magnitude entry in the matrix.

19

Numerical Issues

Generally, for a system of n equations in n unknowns…

If that entry is nearly zero, the linear system is ill-conditioned and notify the user.

If you choose to continue, the division is performed and forward elimination begins.

20

Iterative Methods for Solving Linear Systems Look for a good numerical

approximation instead of the exact mathematical solution.

Useful in sparse linear systems Approaches

Splitting Method Minimization problem

21

Iterative Methods for Solving Linear Systems Splitting

Method

2222121

1212111

bxaxa

bxaxa

22

12122

11

21211

a

xabx

a

xabx

22

)(1212)1(

2

11

)(2121)1(

1

a

xabx

a

xabx

ii

ii

Issues

• Convergence

• Numerical Stability

22

Iterative Methods for Solving Linear Systems Formulate the linear system Ax=b

as a minimization problem

0)()(),( 22222121

2121211121 bxaxabxaxaxxf

23

Matrices

Square matrices Identity matrix Transpose of a matrix Symmetric matrix: A = AT

Skew-symmetric: A = -AT

24

Matrices

Upper echelon matrix U = [uij](nxm) if uij = 0 for i > j If m=n, upper triangular matrix

Lower echelon matrix L = [lij](nxm) if lij = 0 for i < j If m=n, lower triangular matrix

25

Matrices

Elementary Row Matrices

26

Matrices

Elementary Row Matrices

27

Matrices

Elementary Row Matrices The final result of forward elimination can be state

d in terms of elementary row matrices Ek, … E1 applied to the augmented matrix [A|b].

[U|v] = Ek … E1[A|b]

28

Matrices

Inverse Matrix PA = I: P is a left inverse A-1A = I, AA-1 = I. Inverses are unique If A and B are invertible, so is AB. Its inverse

is (AB)-1 = B-1A-1

29

Matrices

LU Decomposition of the matrix A The forward elimination of a matrix A produces an

upper echelon matrix U. The corresponding elementary row matrices are Ek…E1

U = Ek…E1A., L = (Ek…E1)-1. L is lower triangular. A = LU: L is lower triangular and U is upper echelo

n.

30

Matrices

LDU Decomposition of the matrix A L is lower triangular, D is a diagonal matrix,

and U is upper echelon with diagonal entries either 1 or 0.

31

Matrices

LDU Decomposition of the matrix A

32

Matrices

In general the factorization can be written as PA = LDU.

33

Matrices

If A is invertible, its LDU decomposition is unique

If A is symmetric, U in the LDU decomposition must be U = LT.

A = LDLT. If the diagonal entries of D are

nonnegative, A = (LD1/2) (LD1/2)T

34

Vector Spaces

The central theme of linear algebra is the study of vectors and the sets in which they live, called vector spaces.

What is the vector???

35

Vector Spaces

Definition of a Vector Space (the triple (V,+,ᆞ ) )

36

Q & A?