chapter 5 orthogonality. 1 the scalar product in r n the product x t y is called the scalar product...
TRANSCRIPT
![Page 1: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/1.jpg)
Chapter 5
Orthogonality
![Page 2: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/2.jpg)
1 The scalar product in Rn
The product xTy is called the scalar product of x and y.
In particular, if x=(x1, …, xn)T and y=(y1, …,yn)T, then
xTy=x1y1+x2y2+ +‥‥ xnyn
The Scalar Product in R2 and R3
Definition
Let x and y be vectors in either R2 or R3. The distance bet
ween x and y is defined to be the number ‖x-y‖.
![Page 3: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/3.jpg)
Example If x=(3, 4)T and y=(-1, 7)T, then the distance
between x and y is given by
‖y-x‖= 5
![Page 4: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/4.jpg)
Theorem 5.1.1 If x and y are two nonzero vectors in either
R2 or R3 and θ is the angle between them, then
(1) xTy=‖x‖‖y‖cosθ
Corollary 5.1.2 ( Cauchy-Schwarz Inequality)
If x and y are vectors in either R2 or R3 , then
(2) ︱ xTy︱≤‖x‖‖y‖
with equality holding if and only if one of the vectors is 0 or one
vector is a multiple of the other.
![Page 5: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/5.jpg)
Definition
The vector x and y in R2 (or R3) are said to be orthogonal if
xTy=0.
![Page 6: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/6.jpg)
Example
(a) The vector 0 is orthogonal to every vector in R2.
(b) The vectors and are orthogonal in R2.
(c) The vectors and are orthogonal in R3.
2
3
6
4
1
3
2
1
1
1
![Page 7: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/7.jpg)
Scalar and Vector Projections
x z=x-py
u
p=αuθ
y
yx
y
cosyxcosx
T
The scalar is called the scalar projection of x and y, and
the vector p is called the vector projection of x and y.
![Page 8: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/8.jpg)
Scalar projection of x onto y:
y
yxT
Vector projection of x onto y:
yyy
yxy
y
1up
T
T
![Page 9: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/9.jpg)
Example The point Q is the point on the line that is
closet to the point (1, 4). Determine the coordinates of Q.
xy3
1
xy3
1
(1, 4)
v
Qw
![Page 10: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/10.jpg)
Orthogonality in Rn
The vectors x and y are said to be orthogonal if xTy=0.
![Page 11: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/11.jpg)
2 Orthogonal Subspaces
Definition
Two subspaces X and Y of Rn are said to be orthogonal if
xTy=0 for every x∈X and every y∈Y. If X and Y are orthogon
al, we write X⊥Y.
Example Let X be the subspace of R3 spanned by e1, and
let Y be the subspace spanned by e2.
Example Let X be the subspace of R3 spanned by e1 and e2,
and let Y be the subspace spanned by e3.
![Page 12: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/12.jpg)
Definition
Let Y be a subspace of Rn . The set of all vectors in Rn that a
re orthogonal to every vector in Y will be denoted Y⊥. Thus
Y⊥={ x∈Rn︱ xTy=0 for every y∈Y }
The set Y⊥ is called the orthogonal complement of Y.
Remarks
1. If X and Y are orthogonal subspaces of Rn, then X∩Y={0}.
2. If Y is a subspace of Rn, then Y⊥ is also a subspace of Rn.
![Page 13: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/13.jpg)
Fundamental Subspaces
Theorem 5.2.1 ( Fundamental Subspaces Theorem)
If A is an m×n matrix, then N(A)=R(AT) ⊥ and N(AT)=R(A) ⊥.
Theorem 5.2.2 If S is a subspace of Rn, then
dim S+dim S⊥=n. Furthermore, if {x1, …, xr} is a basis for S and
{xr+1, …, xn} is a basis for S⊥, then {x1, …, xr, xr+1, …, xn}
is a basis for Rn.
![Page 14: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/14.jpg)
Definition
If U and V are subspaces of a vector space W and each w∈W can be written uniquely as a sum u+v, where u∈U and v
∈V, then we say that W is a direct sum of U and V, and we w
rite W=U V.
Theorem 5.2.3 If S is a subspace of Rn, then Rn=S S⊥.
Theorem 5.2.4 If S is a subspace of Rn, then (S⊥) ⊥=S.
![Page 15: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/15.jpg)
Theorem 5.2.5 If A is an m×n matrix and b∈Rm, then
either there is a vector x∈Rn such that Ax=b or there is a
vector y∈Rm such that ATy=0 and yTb≠0.
Example Let
431
110
211
A
Find the bases for N(A), R(AT), N(AT), and R(A).
![Page 16: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/16.jpg)
4 Inner Product Spaces
Definition
An inner product on a vector space V is an operation on V th
at assigns to each pair of vectors x and y in V a real number
<x, y> satisfying the following conditions:
Ⅰ. <x, x>≥0 with equality if and only if x=0.
Ⅱ. <x, y>=<y, x> for all x and y in V.
Ⅲ. <αx+βy, z>=α<x, z>+β<y, z> for all x, y, z in V and all sc
alars α and β.
![Page 17: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/17.jpg)
The Vector Space Rm×n
Given A and B in Rm×n, we can define an inner product by
m
i
n
jijijbaBA
1 1
,
![Page 18: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/18.jpg)
Basic Properties of Inner product Spaces
Theorem 5.4.1 ( The Pythagorean Law )
If u and v are orthogonal vectors in an inner product space V,
then
222vuvu
If v is a vector in an inner product space V, the length or norm
of v is given byvv,v
![Page 19: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/19.jpg)
Example If
33
21
11
A and
43
03
11
B
then 6, BA
5A
6B
![Page 20: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/20.jpg)
Definition
If u and v are vectors in an inner product space V and v≠0, th
en the scalar projection of u onto v is given by
v
vu,
and the vector projection of u onto v is given by
vvv,
vu,v
v
1p
![Page 21: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/21.jpg)
Theorem 5.4.2 ( The Cauchy- Schwarz Inequality)
If u and v are any two vectors in an inner product space V, then
vuvu,
Equality holds if and only if u and v are linearly dependent.
![Page 22: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/22.jpg)
5 Orthonormal Sets
Definition
Let v1, v2, …, vn be nonzero vectors in an inner product space
V. If <vi, vj>=0 whenever i≠j, then { v1, v2, …, vn} is said to be
an orthogonal set of vectors.
Example The set {(1, 1, 1)T, (2, 1, -3)T, (4, -5, 1)T} is an
orthogonal set in R3.
Theorem 5.5.1 If { v1, v2, …, vn} is an orthogonal set of
nonzero vectors in an inner product space V, then v1, v2, …,vn
are linearly independent.
![Page 23: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/23.jpg)
Definition
An orthonormal set of vectors is an orthogonal set of unit vect
ors.
i
n
iic uv
1
Theorem 5.5.2 Let { u1, u2, …, un} be an orthonoemal basis
for an inner product space V. If , then ci=<v, ui>.
The set {u1, u2, …, un} will be orthonormal if and only if
ijji u,u
where
jiif
jiifij 0
1
![Page 24: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/24.jpg)
Corollary 5.5.3 Let { u1, u2, …, un} be an orthonoemal basis
for an inner product space V. If and , theni
n
iia uu
1
i
n
iib uv
1
i
n
iiba
1
vu,
Corollary 5.5.4 If { u1, u2, …, un} is an orthonoemal basis
for an inner product space V and , theni
n
iic uv
1
n
iic
1
22v
![Page 25: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/25.jpg)
Orthogonal MatricesDefinition
An n×n matrix Q is said to be an orthogonal matrix if the colu
mn vectors of Q form an orthonormal set in Rn.
Theorem 5.5.5 An n×n matrix Q is orthogonal if and only if
QTQ=I.
Example For any fixed , the matrix
cossin
sincosQ
is orthogonal.
![Page 26: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/26.jpg)
Properties of Orthogonal Matrices
If Q is an n×n orthogonal matrix, then
(a) The column vectors of Q form an orthonormal basis for Rn.
(b) QTQ=I
(c) QT=Q-1
(d) det(Q)=1 or -1
(e) The thanspose of an orthogonal matrix is an orthogonal
matrix.
(f) The product of two orthogonal matrices is also an orthogonal
matrix.
![Page 27: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/27.jpg)
6 The Gram-Schmidt Orthogonalization Process
Theorem 5.6.1 ( The Gram-Schmidt Process)
Let {x1, x2, …, xn} be a basis for the inner product space V. Let
1
11 x
x
1u
and define u2, …, un recursively by
)px(px
1u 1
11 kk
kkk
for k=1, …, n-1
![Page 28: Chapter 5 Orthogonality. 1 The scalar product in R n The product x T y is called the scalar product of x and y. In particular, if x=(x 1, …, x n ) T and](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649e3c5503460f94b2e466/html5/thumbnails/28.jpg)
where
pk=<xk+1, u1>u1+<xk+1, u2>+ <‥‥ xk+1, uk>uk
is the projection of xk+1 onto Span(u1, u2, …, uk). The set
{u1, u2, …, un}
is an orthonormal basis for V.
Example Let
011
241
241
411
A
Find an orthonormal basis for the column space of A.