lecture 14: solving systems of linear equations by

Lecture 14: Solving Systems of Linear Equationsby Gaussian Elimination, Part I

Winfried JustDepartment of Mathematics, Ohio University

MATH3200: Applied Linear Algebra

Winfried Just, Ohio University MATH3200, Lecture 14: Gaussian Elimination I

Review: The extended matrix

Consider a system of linear equations:

a11x1 + a12x2 + · · ·+ a1nxn = b1

a21x1 + a22x2 + · · ·+ a2nxn = b2...

am1x1 + am2x2 + · · ·+ amnxn = bm

The extended aka augmented matrix [A, ~b] represents this system:

[A, ~b] =

a11 a12 . . . a1n b1a21 a22 . . . a2n b2

......

......

am1 am2 . . . amn bm


Equivalent systems and their augmented matrices

We will say that two systems of m linear equations in n variablesare equivalent if they have the same sets of solutions.

Similarly, we will say that two matrices [A1, ~b1], [A2, ~b2] of orderm × (n + 1) are equivalent if they represent equivalent systems oflinear equations.

Here a matrix [A, ~b] is said to represent a system of linearequations if it is the augmented matrix of the system.

Note that any matrix with at least two columns represents somesystem of linear equations.

Solving a system boils down to transforming it successively intosimpler equivalent systems.

This can be accomplished by successively transforming extendedmatrices into equivalent ones.


Review: A strategy for doing these transformations

When the extended matrix

[A, ~b] =

a11 a12 . . . a1n b1a21 a22 . . . a2n b2

. . .am1 am2 . . . amn bm

of the system is in row echelon form, we can accomplish this byback-substitution.

If not, then we can transform the system step-by-step into anequivalent system, that is, a system with the same solution set,whose extended matrix is in row echelon form.

Each step involves an elementary operation on systems of linearequations that does not change the solution set.


Review: Elementary operations on systems of linearequations

Consider a system of m linear equations in n variables.

Each of the following operations preserves the set of solutions andthus transforms the system into an equivalent one:

(i) Interchanging the positions of any two equations.

(ii) Multiplying an equation by a nonzero scalar.

(iii) Adding to one equation a scalar multiple of another equation.

In practice, it is more convenient to work with elementary rowoperations on extended matrices.


Elementary row operations on matrices

Consider a matrix [A, ~b] of order m × (n + 1).

Each of the following elementary row operations transforms [A, ~b]into an equivalent matrix:

(E1) Interchanging any two rows.

(E2) Multiplying any row by a nonzero scalar.

(E3) Adding to one row of the matrix a scalar times another row ofthe matrix.


The method of Gaussian elimination

Gaussian elimination is a method for solving systems of linearequations. It is named after Carl Friedrich Gauss (1777–1855).

The method relies on transforming the augmented matrix of agiven system into an equivalent matrix in row echelon form bysuccessive elementary row operations.

Gaussian elimination usually works well for numerical solutions onthe computer.

We will see later that it also gives us some additional insights.


Where do we want to go?Review of the row echelon form

A matrix is in row echelon form, or simply a echelon form if:

(R1) All zero rows appear below all nonzero rows when both typesare present.

(R2) The first nonzero entry in any nonzero row is 1.

(R3) All elements in the same column below the first nonzeroelement of a nonzero row are 0.

(R4) The first nonzero element in a nonzero row appears in acolumn further to the right of the first nonzero element in anypreceding row.


The method of Gaussian elimination: Example 1

Consider the following system of linear equations:

0.5x1 − x2 + x3 = 2

2x1 − 7x2 − 3x3 = 0

x1 + x2 + x3 = 5

The augmented matrix is

[A, ~b] =

0.5 −1 1 22 −7 −3 01 1 1 5

Ideally, we’d like to get an equivalent matrix that looks like this:1 ? ? ?

0 1 ? ?0 0 1 ?


Gaussian elimination for Example 1

We want to transform our matrix into:

1 ? ? ?0 1 ? ?0 0 1 ?

In order to keep track of our work, we always indicate in shorthandnotation which elementary row operations are used at each step.0.5 −1 1 2

2 −7 −3 01 1 1 5

R17→2R1−→

1 −2 2 42 −7 −3 01 1 1 5

Here we multiplied Row 1 by 2.1 −2 2 4

2 −7 −3 01 1 1 5

R27→R2−2R1−→

1 −2 2 40 −3 −7 −81 1 1 5

Here we subtracted two times Row 1 from Row 2.


Gaussian elimination for Example 1, continued


1 ? ? ?0 1 ? ?0 0 1 ?

In the next step we want to achieve:1 −2 2 4

0 −3 −7 −81 1 1 5

−→1 ? ? ?

0 ? ? ?0 ? ? ?

Question L14.1: Which elementary row operation should we perform?

1 −2 2 40 −3 −7 −81 1 1 5

R37→R3−R1−→

1 −2 2 40 −3 −7 −80 3 −1 1

We need to subtract Row 1 from Row 3.




1 ? ? ?0 1 ? ?0 0 1 ?


0 −3 −7 −80 3 −1 1

−→1 ? ? ?

0 ? ? ?0 0 ? ?


1 −2 2 40 −3 −7 −80 3 −1 1

R37→R3+R2−→

1 −2 2 40 −3 −7 −80 0 −8 −7

We need to add Row 2 to Row 3.




1 ? ? ?0 1 ? ?0 0 1 ?


0 −3 −7 −80 0 −8 −7

−→1 ? ? ?

0 1 ? ?0 0 ? ?


1 −2 2 40 −3 −7 −80 0 −8 −7

R27→R2/(−3)−→

1 −2 2 40 1 7/3 8/30 0 −8 −7

We need to divide Row 2 by -3.


Gaussian elimination for Example 1, completed


1 ? ? ?0 1 ? ?0 0 1 ?


0 1 7/3 8/30 0 −8 −7

−→1 ? ? ?

0 1 ? ?0 0 1 ?


1 −2 2 40 1 7/3 8/30 0 −8 −7

R37→R3/(−8)−→

1 −2 2 40 1 7/3 8/30 0 1 7/8

We need to divide Row 3 by -8.


Example 1: Back-substitution

We have transformed the augmented matrix of the original systeminto an equivalent matrix in row echelon form:1 −2 2 4

0 1 7/3 8/30 0 1 7/8

It represents the following equivalent system:

x1 − 2x2 + 2x3 = 4

x2 +7

3x3 =

8

3

x3 =7

8

Back-substitution gives the solution: x3 = 78 , x2 = 5

8 , x1 = 72 .


The ideal order of operations

Although there may be many different ways to successfully performGaussian elimination on a given matrix, there is one ideal order ofsteps that often works best, as long as it it feasible. Here is anillustration of it for a 3× 4 matrix. For each step, elements of thematrix that may change are shown in color. You will see that thisorder of operations preserves zeros and ones that were created atearlier steps of the process.

Start with dividing the first row by a11:a11 a12 a13 a14a21 a22 a23 a24a31 a32 a33 a34

R1 7→R1/a11−→

1 b12 b13 b14a21 a22 a23 a24a31 a32 a33 a34

Question L14.5: Is this step always feasible?

No. This step is feasible only if a11 6= 0; otherwise we may need toswitch some rows first.


The ideal order of operations, continued

Now use the 1 in the upper left corner as a so-called pivot tocancel the elements of the matrix below it:

1 b12 b13 b14a21 a22 a23 a24a31 a32 a33 a34

R27→R2−a21R1−→

1 b12 b13 b140 b22 b23 b24a31 a32 a33 a34

1 b12 b13 b14

0 b22 b23 b24a31 a32 a33 a34

R37→R3−a31R1−→

1 b12 b13 b140 b22 b23 b240 b32 b33 b34

Question L14.6: Are the two steps shown on this slide alwaysfeasible?

Yes.


The ideal order of operations, completed

If feasible, divide the second row by b22 and use the resulting 1 asa pivot to cancel the element(s) below it. For larger matrices,continue analogously until you get the row echelon form.1 b12 b13 b14

0 b22 b23 b240 b32 b33 b34

R27→R2/b22−→

1 b12 b13 b140 1 c23 c240 b32 b33 b34

1 b12 b13 b14

0 1 c23 c240 b32 b33 b34

R3 7→R3−b32R2−→

1 a12 a13 a140 1 c23 c240 0 c33 c34

1 b12 b13 b14

0 1 c23 c240 0 c33 c34

R3 7→R3/c33−→

1 b12 b13 b140 1 c23 c240 0 1 d34


Example 2 of a Gaussian elimination

Recall that in Conversation 13, Bob transformed the extendedmatrix of the system

xa + xb + xc = 11xa − xc = −1xa − 2xb = 0

into row echelon form. He did in fact perform a Gaussianelimination. To see this, let us trace his steps, in the ideal order.

Here the extended matrix of the system is:1 1 1 111 0 −1 −11 −2 0 0

It has already a 1 in the upper left corner.


Example 2 of a Gaussian elimination, continued

Now use the 1 in the upper left corner as a pivot to cancel theelements of the matrix below it:1 1 1 11

1 0 −1 −11 −2 0 0

R27→R2−R1−→

1 1 1 110 −1 −2 −121 −2 0 0

1 1 1 11

0 −1 −2 −121 −2 0 0

R3 7→R3−R1−→

1 1 1 110 −1 −2 −120 −3 −1 −11


Example 2 of Gaussian elimination, completed

Next divide the second row by −1 and use the resulting 1 as apivot to cancel the element below it. In the last step, divide thethird row by 5 to obtain 1 as the first nonzero element of this row.1 1 1 11

0 −1 −2 −120 −3 −1 −11

R27→R2/(−1)−→

1 1 1 110 1 2 120 −3 −1 −11

1 1 1 11

0 1 2 120 −3 −1 −11

R37→R3+3R2−→

1 1 1 110 1 2 120 0 5 25

1 1 1 11

0 1 2 120 0 5 25

R37→R3/5−→

1 1 1 110 1 2 120 0 1 5


Take-home message

Gaussian elimination is a method for transforming matricesinto row echelon form by successively applying elementary rowoperations.

If we start with the augmented matrix of a linear system, themethod will transform it into the extended matrix of anequivalent system that has the same solution set.

The ideal order of operations creates ones and zeros insuccessive columns, starting from the leftmost column. Ineach column, it is usually most convenient to first create a 1in the appropriate place and then use it as a pivot to cancelthe elements below it.

After you have obtained an equivalent matrix in row echelonform, translate it back into a system of linear equations andsolve that equivalent system with back-substitution.


lecture 14: solving systems of linear equations by

Documents