Álgebra Linear e Aplicações

RECTANGULAR SYSTEMS AND ECHELON FORMS

Rectangular systems

• General linear system

• What happens to elimination when m ≠ n?

a11x1 + a12x2 + · · · a1nxn = b1a21x1 + a22x2 + · · · a2nxn = b2

...am1x1 + am2x2 + · · · amnxn = bm

More on matrix notation

• Matrix of size m × n

A =

0

BBB@

a11 a12 · · · a1na21 a22 · · · a2n...

.... . .

...am1 am2 · · · amn

1

CCCA= [aij ]m⇥n

m = n: square matrixm �n: rectangular matrix

Ai⇤ =�ai1 ai2 · · · ain

�row vector

A⇤j =

0

BBB@

a1ja2j...

amj

1

CCCAcolumn vector

0

BB@

1 2 1 3 30 0 �2 �2 �20 0 2 2 20 0 �2 �2 1

1

CCA

0

BB@

1 2 1 3 30 0 �2 �2 �20 0 2 2 20 0 �2 �2 1

1

CCA

Gaussian Elimination example

• Consider the system

• Gaussian Elimination

x1 + 2x2 + x3 + 3x4 + 3x5 = 52x1 + 4x2 + 4x4 + 4x5 = 6x1 + 2x2 + 3x3 + 5x4 + 5x5 = 92x1 + 4x2 + 4x4 + 7x5 = 9

0

BB@

1 2 1 3 32 4 0 4 41 2 3 5 52 4 0 4 7

1

CCA

0

BB@

1 2 1 3 30 0 �2 �2 �20 0 0 0 00 0 0 0 3

1

CCA

0

BB@

1 2 1 3 30 0 �2 �2 �20 0 0 0 30 0 0 0 0

1

CCA

Modified Gaussian Elimination

• Look for a column that has a viable pivot

• If needed, exchange rows to move pivot up

• Stop if all rows below last pivot are zero

0

BB@

⇤ ⇤ ⇤ ⇤ ⇤0 s s s s0 s s s s0 s s s s

1

CCA

0

BB@

⇤ ⇤ ⇤ ⇤ ⇤0 0 s s s0 0 s s s0 0 s s s

1

CCA

0

BB@

⇤ ⇤ ⇤ ⇤ ⇤0 0 0 s s0 0 ⇤ s s0 0 s s s

1

CCA

0

BB@

⇤ ⇤ ⇤ ⇤ ⇤0 0 ⇤ s s0 0 0 s s0 0 s s s

1

CCA

0

BB@

⇤ ⇤ ⇤ ⇤ ⇤0 0 ⇤ ⇤ ⇤0 0 0 0 ⇤0 0 0 0 0

1

CCA

Row Echelon Form

• Two conditions• All rows below a zero-row are also zero• All elements below or left of pivots are zero

• Elements not unique, but form is unique!

0

BBBBBB@

⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤0 0 ⇤ ⇤ ⇤ ⇤ ⇤ ⇤0 0 0 ⇤ ⇤ ⇤ ⇤ ⇤0 0 0 0 0 0 ⇤ ⇤0 0 0 0 0 0 0 00 0 0 0 0 0 0 0

1

CCCCCCA

Rank of a Matrix

• Let A have row echelon form E• rank(A) = number of pivots• rank(A) = number of non-zero rows in E• rank(A) = number of basic columns in A

• Basic column is a column at a pivotal position

A =

0

BB@

1 2 1 3 32 4 0 4 41 2 3 5 52 4 0 4 7

1

CCA E =

0

BB@

1 2 1 3 30 0 �2 �2 �20 0 0 0 30 0 0 0 0

1

CCA

A⇤1 =

0

BB@

1212

1

CCA

A⇤3 =

0

BB@

1030

1

CCA A⇤5 =

0

BB@

3457

1

CCA

Modified Gauss-Jordan example

• Back to our system0

BB@

1 2 1 3 30 0 �2 �2 �20 0 0 0 30 0 0 0 0

1

CCA

0

BB@

1 2 1 3 30 0 1 1 10 0 0 0 30 0 0 0 0

1

CCA

0

BB@

1 2 0 2 20 0 1 1 10 0 0 0 30 0 0 0 0

1

CCA

0

BB@

1 2 0 2 20 0 1 1 10 0 0 0 10 0 0 0 0

1

CCA

0

BB@

1 2 0 2 00 0 1 1 00 0 0 0 10 0 0 0 0

1

CCA

Reduced Row Echelon Form

• Echelon form is result of Gaussian Elimination• Gauss-Jordan leads to reduced row echelon form

• Notation: EA is reduced row echelon form of A• Reduced form is unique (proof later on)• Useful for theoretical results

0

BBBBBB@

1 ⇤ 0 0 ⇤ ⇤ 0 ⇤0 0 1 0 ⇤ ⇤ 0 ⇤0 0 0 1 ⇤ ⇤ 0 ⇤0 0 0 0 0 0 1 ⇤0 0 0 0 0 0 0 00 0 0 0 0 0 0 0

1

CCCCCCA

Why basic columns are basic

• Non-basic columns are combinations of basic columns, as made obvious by reduced form

• Row operations preserve column relationships

EA =

0

BB@

1 2 0 2 00 0 1 1 00 0 0 0 10 0 0 0 0

1

CCA

A =

0

BB@

1 2 1 3 32 4 0 4 41 2 3 5 52 4 0 4 7

1

CCA

E⇤2 = 2E⇤1

E⇤4 = 2E⇤1 +E⇤3

A⇤2 = 2A⇤1

A⇤4 = 2A⇤1 +A⇤3

Column relationships in A and EA

• are the basic columns to the left of E*k

• Same relationships hold for and A*k

• Why?

E⇤k =

0

BBBBBBBB@

µ1

µ2...µj...0

1

CCCCCCCCA

= µ1

0

BBBBBBBB@

10...0...0

1

CCCCCCCCA

+ µ2

0

BBBBBBBB@

01...0...0

1

CCCCCCCCA

+ · · ·+ µj

0

BBBBBBBB@

00...1...0

1

CCCCCCCCA

= µ1E⇤b1 + µ2E⇤b2 + · · ·+ µjE⇤bj

A⇤bi

E⇤bi

A⇤k = µ1A⇤b1 + µ2A⇤b2 + · · ·+ µjA⇤bj

SOLVING RECTANGULAR SYSTEMS

Consistency

• Rectangular system is consistent if it has at least one solution

• It is inconsistent if it has no solution

Consistent system Inconsistent system

Using Gaussian elimination instead

• Reduce [A|b] to [E|c] in row echelon form

• If , we have a problem

• Otherwise, back-substitution works!

0

BBBBBBB@

⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤0 0 ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤0 0 0 ⇤ ⇤ ⇤ ⇤ ⇤ ⇤0 0 0 0 0 0 ⇤ ⇤ ⇤0 0 0 0 0 0 0 0 ↵...

......

......

......

......

1

CCCCCCCA

↵ 6= 0

0x1 + 0x2 + · · ·+ 0xn = �

Equivalent consistency criteria

• When row reducing [A|b], a row of the following form never appears

• b is a combination of the basic columns in A• rank[A|b] = rank(A)

�0 0 · · · 0 ↵

�↵ 6= 0

Homogeneous systems

• An homogeneous system is of the form

• Can it be inconsistent?• Values of ? • Trivial solution!

a11x1 + a12x2 + · · · a1nxn = 0a21x1 + a22x2 + · · · a2nxn = 0

...am1x1 + am2x2 + · · · amnxn = 0

↵ 6= 0

x1 = x2 = · · · = xm = 0

Homogeneous system example

• Consider the system

• In row echelon form

• Equivalent to

• 4 unknowns and only 2 equations!

• Pick basic variables• pivotal positions

• as functions of free variables

x1 + 2x2 + 2x3 + 3x4 = 02x1 + 4x2 + x3 + 3x4 = 03x1 + 6x2 + x3 + 4x4 = 0

0

@1 2 2 32 4 1 33 6 1 4

1

A

0

@1 2 2 30 0 �3 �30 0 �5 �5

1

A

0

@1 2 2 30 0 �3 �30 0 0 0

1

A

x1 + 2x2 + 2x3 + 3x4 = 0� 3x3 � 3x4 = 0

x3 = �x4

x1 = �2x2 � 2x3 � 3x4

= �2x2 � x4

Representating the general solution

• Pick any values for x2 and x4, and you create a new solution to the system

• Every solution is a combination of two particular solutions

x1 = �2x2 � x4

x2 = x2

x3 = �x4

x4 = x4

h1 =

0

BB@

�2100

1

CCA h2 =

0

BB@

�10

�11

1

CCA

0

BB@

x1

x2

x3

x4

1

CCA =

0

BB@

�2x2 � x4

x2

�x4

x4

1

CCA = x2

0

BB@

�2100

1

CCA+ x4

0

BB@

�10

�11

1

CCA

General homogeneous system

• Consider an m×n system [A|0]• How many basic variables?• r = rank(A)

• How many free variables?• n – r

• General form of solution isx = xf1h1 + xf2h2 + · · ·+ xfn�rhn�r

Unicity of homogeneous solution

• Consistency has already been shown• Trivial solution

• Look at general solution

• When is it unique?• When there are no free variables• When n-r = 0• i.e., when rank(A) = n

x1 = x2 = · · · = xm = 0

x = xf1h1 + xf2h2 + · · ·+ xfn�rhn�r

Nonhomogeneous systems

• A system is non-homogeneous

• whenever for at least one i• Solution follows the same procedure as the

homogeneous case

a11x1 + a12x2 + · · · a1nxn = b1a21x1 + a22x2 + · · · a2nxn = b2

...am1x1 + am2x2 + · · · amnxn = bm

bi 6= 0

Solving nonhomogeneous systems

• Start with [A|b] and reduce to [E|c]• Identify the basic and free variables• Apply back-substitution to solve for basic

variables as function of free variables• Write the result in the form

• The only difference is the term p that was not needed in the homogeneous case

x = p+ xf1h1 + xf2h2 + · · ·+ xfn�rhn�r

Example showing the p term

• The system

• Augmented matrix

• Gauss-Jordan result

• Reduced system

• Back substitution

• General form

x1 + 2x2 + 2x3 + 3x4 = 42x1 + 4x2 + x3 + 3x4 = 53x1 + 6x2 + x3 + 4x4 = 7

E[A|b] =

0

@1 2 0 1 20 0 1 1 10 0 0 0 0

1

A

[A|b] =

0

@1 2 2 3 42 4 1 3 53 6 1 4 7

1

A

x1 + 2x2 + x4 = 2x3 + x4 = 1

x1 = 2� 2x2 � x4

x2 = x2

x3 = 1� x4

x4 = x4

0

BB@

x1

x2

x3

x4

1

CCA =

0

BB@

2010

1

CCA+ x2

0

BB@

�2100

1

CCA+ x4

0

BB@

�10

�11

1

CCA

General solution of nonhomogeneous system

• Particular solution + general solution of the associated homogeneous system!

E[A|b] = [EA|c] =

0

BBBBBBBBB@

1 � 0 0 � � 0 � c10 0 1 0 � � 0 � c2...

......

......

......

......

0 0 0 1 � � 0 � cr�1

0 0 0 0 0 0 1 � cr0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0

1

CCCCCCCCCA

xbi = �ifixfi + �ifi+1xfi+1 + · · ·�ifn�rxfn�r

xbi = ci + �ifixfi + �ifi+1xfi+1 + · · ·�ifn�rxfn�r

E[A|0] = [EA|0] =

0

BBBBBBBBB@

1 � 0 0 � � 0 � 00 0 1 0 � � 0 � 0...

......

......

......

......

0 0 0 1 � � 0 � 00 0 0 0 0 0 1 � 00 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0

1

CCCCCCCCCA

Summary

• General solution of linear system is• Column p is a particular solution to

nonhomogeneous system and

is the general solution of the associated homogeneous system

• Unique solution if and only if any of• rank(A) = n = number of unknowns• There are no free variables• Associated homogeneous system has only trivial solution

xh = xf1h1 + xf2h2 + · · ·+ xfn�rhn�r

x = p+ xh

Application to electric circuits

Ohm’s law and Kirchhoff’s rules

• Reduce problem to linear system• Ohm’s Law• For a current I, the voltage drop across a

resistor R is V = R I• Kirchhoff’s rules• Nodes: algebraic sum of currents entering a

node is zero• Loop: algebraic sum of voltage drops along each

loop is zero

Formulating the linear system

• Nodes

• LoopsA : I1R1 � I3R3 + I5R5 = E1 � E3

B : I2R2 � I5R5 + I6R6 = E2

C : I3R3 + I4R4 � I6R6 = E3 + E4

1 : I1 � I2 � I5 = 02 : �I1 � I3 + I4 = 03 : I3 + I5 + I6 = 04 : I2 � I4 � I6 = 0