March 27 Math 3260 sec. 55 Spring 2018

Section 4.6: Rank

Definition: The row space, denoted Row A, of an m × n matrix A isthe subspace of Rn spanned by the rows of A.

We now have three vector spaces associated with an m × n matrix A,its column space, null space, and row space.

March 26, 2018 1 / 52

Theorem

If two matrices A and B are row equivalent, then their row spaces arethe same.

In particular, if B is an echelon form of the matrix A, then the nonzerorows of B form a basis for Row B—and also for Row A since these arethe same space.

March 26, 2018 2 / 52

ExampleA matrix A along with its rref is shown.

A =

−2 −5 8 0 −171 3 −5 1 53 11 −19 7 11 7 −13 5 −3

∼

1 0 1 0 10 1 −2 0 30 0 0 1 −50 0 0 0 0

(a) Find a basis for Row A and state the dimension dim Row A.

March 26, 2018 3 / 52

Example continued ...(b) Find a basis for Col A and state its dimension.

March 26, 2018 4 / 52

Example continued ...(c) Find a basis for Nul A and state its dimension.

March 26, 2018 5 / 52

March 26, 2018 6 / 52

Remarks

I We can naturally associate three vector spaces with an m × nmatrix A. Row A and Nul A are subspaces of Rn and Col A is asubspace of Rm.

I Careful! The rows of the rref do span Row A. But we go back tothe columns in the original matrix to get vectors that span Col A.(Get a basis for Col A from A itself!)

I Careful Again! Just because the first three rows of the rref spanRow A does not mean the first three rows of A span Row A. (Geta basis for Row A from the rref!)

March 26, 2018 7 / 52

Remarks

I Row operations preserve row space, but change lineardependence relations of rows. Row operations change columnspace, but preserve linear dependence relations of columns.

I Another way to obtain a basis for Row A is to take the transposeAT and do row operations. We have the following relationships:

Col A = Row AT and Row A = Col AT .

I The dimension of the null space is called the nullity.

March 26, 2018 8 / 52

Rank

Definition: The rank of a matrix A (denoted rank A) is the dimensionof the column space of A.

Theorem: For m × n matrix A, dim Col A = dim Row A = rank A.Moreover

rank A + dim Nul A = n.

Note: This theorem states the rather obvious fact that{number of

pivot columns

}+

{number of

non-pivot columns

}=

{total numberof columns

}.

March 26, 2018 9 / 52

Examples(1) A is a 5× 4 matrix with rank A = 4. What is dim Nul A?

(2) If A is 7× 5 and dim Col A = 2. Determine the nullity1 of A and rankAT .

1Nullity is another name for dim Nul A.

March 26, 2018 10 / 52

Addendum to Invertible Matrix Theorem

Let A be an n × n matrix. The following are equivalent to the statementthat A is invertible.

(m) The columns of A form a basis for Rn

(n) Col A = Rn

(o) dim Col A = n(p) rank A = n(q) Nul A = {0}(r) dim Nul A = 0

March 26, 2018 11 / 52

Section 6.1: Inner Product, Length, and OrthogonalityRecall: A vector u in Rn can be considered an n × 1 matrix. It followsthat uT is a 1× n matrix

uT = [u1 u2 · · · un].

Definition: For vectors u and v in Rn we define the inner product of uand v (also called the dot product) by the matrix product

uT v = [u1 u2 · · · un]

v1v2...

vn

= u1v1 + u2v2 + · · ·+ unvn.

Note that this product produces a scalar. It is sometimes called ascalar product.

March 26, 2018 12 / 52

Theorem (Properties of the Inner Product)

We’ll use the notation u · v = uT v.

Theorem: For u, v and w in Rn and real scalar c(a) u · v = v · u

(b) (u + v) ·w = u ·w + v ·w

(c) c(u · v) = (cu) · v = u · (cv)

(d) u · u ≥ 0, with u · u = 0 if and only if u = 0.

March 26, 2018 13 / 52

The Norm

The property u · u ≥ 0 means that√

u · u always exists as a realnumber.

Definition: The norm of the vector v in Rn is the nonnegative numberdenoted and defined by

‖v‖ =√

v · v =√

v21 + v2

2 + · · ·+ v2n

where v1, v2, . . . , vn are the components of v.

As a directed line segment, the norm is the same as the length.

March 26, 2018 14 / 52

Norm and Length

Figure: In R2 or R3, the norm corresponds to the classic geometric propertyof length.

March 26, 2018 15 / 52

Unit Vectors and Normalizing

Theorem: For vector v in Rn and scalar c

‖cv‖ = |c|‖v‖.

Definition: A vector u in Rn for which ‖u‖ = 1 is called a unit vector.

Remark: Given any nonzero vector v in Rn, we can obtain a unitvector u in the same direction as v

u =v‖v‖

.

This process, of dividing out the norm, is called normalizing the vectorv.

March 26, 2018 16 / 52

Example

Show that v/‖v‖ is a unit vector.

March 26, 2018 17 / 52

Example

Find a unit vector in the direction of v = (1,3,2).

March 26, 2018 18 / 52

Distance in Rn

Definition: For vectors u and v in Rn, the distance between u and vis denoted and defined by

dist(u,v) = ‖u− v‖.

Example: Find the distance between u = (4,0,−1,1) andv = (0,0,2,7).

March 26, 2018 19 / 52

OrthogonalityDefinition: Two vectors are u and v orthogonal if u · v = 0.

Figure: Note that two vectors are perpendicular if ‖u− v‖ = ‖u + v‖

March 26, 2018 20 / 52

Orthogonal and PerpendicularShow that ‖u− v‖ = ‖u + v‖ if and only if u · v = 0.

March 26, 2018 21 / 52

March 26, 2018 22 / 52

March 26, 2018 23 / 52

The Pythagorean Theorem

Theorem: Two vectors u and v are orthogonal if and only if

‖u + v‖2 = ‖u‖2 + ‖v‖2.

March 26, 2018 24 / 52

Orthogonal ComplementDefinition: Let W be a subspace of Rn. A vector z in Rn is said to beorthogonal to W if z is orthogonal to every vector in W .

Definition: Given a subspace W of Rn, the set of all vectorsorthogonal to W is called the orthogonal complement of W and isdenoted by

W⊥.

March 26, 2018 25 / 52

Theorem:W⊥ is a subspace of Rn.

March 26, 2018 26 / 52

March 26, 2018 27 / 52

Example

Let W =Span

1

00

, 0

01

. Show that W⊥ =Span

0

10

.

Give a geometric interpretation of W and W⊥ as subspaces of R3.

March 26, 2018 28 / 52

March 26, 2018 29 / 52

March 26, 2018 30 / 52

Example

Let A =

[1 3 2−2 0 4

]. Show that if x is in Nul(A), then x is in

[Row(A)]⊥.

March 26, 2018 31 / 52

March 26, 2018 32 / 52

March 26, 2018 33 / 52

Theorem

Theorem: Let A be an m × n matrix. The orthogonal complement ofthe row space of A is the null space of A. That is

[Row(A)]⊥ = Nul(A).

The orthongal complement of the column space of A is the null spaceof AT —i.e.

[Col(A)]⊥ = Nul(AT ).

March 26, 2018 34 / 52

Example: Find the orthogonal complement of Col(A)

A =

5 2 1−3 3 02 4 12 −2 90 1 −1

March 26, 2018 35 / 52

March 26, 2018 36 / 52

Section 6.2: Orthogonal Sets

Remark: We know that if B = {b1, . . . ,bp} is a basis for a subspaceW of Rn, then each vector x in W can be realized (uniquely) as a sum

x = c1b2 + · · ·+ cpbp.

If n is very large, the computations needed to determine thecoefficients c1, . . . , cp may require a lot of time (and machine memory).

Question: Can we seek a basis whose nature simplifies this task?And what properties should such a basis possess?

March 26, 2018 37 / 52

Orthogonal SetsDefinition: An indexed set {u1, . . . ,up} in Rn is said to be anorthogonal set provided each pair of distinct vectors in the set isorthogonal. That is, provided

ui · uj = 0 whenever i 6= j .

Example: Show that the set

3

11

, −1

21

, −1−47

is an

orthogonal set.

March 26, 2018 38 / 52

3

11

,

−121

,

−1−47

March 26, 2018 39 / 52

Orthongal Basis

Definition: An orthogonal basis for a subspace W of Rn is a basisthat is also an orthogonal set.

Theorem: Let {u1, . . . ,up} be an orthogonal basis for a subspace Wof Rn. Then each vector y in W can be written as the linearcombination

y = c1u1 + c2u2 + · · ·+ cpup, where the weights

cj =y · uj

uj · uj.

March 26, 2018 40 / 52

Example 3

11

, −1

21

, −1−47

is an orthogonal basis of R3. Express

the vector y =

−230

as a linear combination of the basis vectors.

March 26, 2018 41 / 52

March 26, 2018 42 / 52

ProjectionGiven a nonzero vector u, suppose we wish to decompose anothernonzero vector y into a sum of the form

y = y + z

in such a way that y is parallel to u and z is perpendicular to u.

March 26, 2018 43 / 52

ProjectionSince y is parallel to u, there is a scalar α such that

y = αu.

March 26, 2018 44 / 52

Projection onto the subspace L =Span{u}

Notation: y = projL =(y · u

u · u

)u

Example: Let y =

[76

]and u =

[42

]. Write y = y + z where y is in

Span{u} and z is orthogonal to u.

March 26, 2018 45 / 52

Example Continued...Determine the distance between the point (7,6) and the line Span{u}.

March 26, 2018 46 / 52

Orthonormal SetsDefinition: A set {u1, . . . ,up} is called an orthonormal set if it is anorthogonal set of unit vectors.

Definition: An orthonormal basis of a subspace W of Rn is a basisthat is also an orthonormal set.

Example: Show that

3

5

45

, −4

5

35

is an orthonormal basis for

R2.

March 26, 2018 47 / 52

March 26, 2018 48 / 52

Orthogonal Matrix

Consider the matrix U =

[ 35 −4

545

35

]whose columns are the vectors in

the last example. Compute the product

UT U

What does this say about U−1?

March 26, 2018 49 / 52

Orthogonal Matrix

Definition: A square matrix U is called an orthogonal matrix ifUT = U−1.

Theorem: An n × n matrix U is orthogonal if and only if it’s columnsform an orthonormal basis of Rn.

The linear transformation associated to an orthogonal matrixpreserves lenghts and angles in the following sense:

March 26, 2018 50 / 52

Theorem: Orthogonal Matrices

Let U be an n × n orthogonal matrix and x and y vectors in Rn. Then

(a) ‖Ux‖ = ‖x‖

(b) (Ux) · (Uy) = x · y, in particular

(c) (Ux) · (Uy) = 0 if and only if x · y = 0.

Proof (of (a)):

March 26, 2018 51 / 52

March 26, 2018 52 / 52