Download - LINEAR ALGEBRA - Shandong University
LINEAR ALGEBRA
Lecture 8: Inner-Product Spaces
April 21, 2014
Jianfei Shen
School of Economics, Shandong University
Motivation
» We study vector spaces in which it makes sense to speak of˚
the length of a vector, and
the angle between two vectors.
,Jianfei Shen (Shandong University) 2/31
1 Euclidean Structure
2 Inner-Product Spaces
3 Orthonormal Bases
,Jianfei Shen (Shandong University) 3/31
Norms
» The length of a vector x in R2,
denoted as kxk, is defined as its
distance to the origin.
» In Rn, the norm of x D .x1; : : : ; xn/is defined by
kxk Dqx21 C � � � C x2n:
» The norm is not linear on Rn.
0x1
x2
kxkDq
x21C x22
x
xy
,Jianfei Shen (Shandong University) 4/31
Inner Product on Rn
» For x;y 2 Rn, the inner product of x and y , denoted hx;yi, is
defined by
hx;yi D x1y1 C � � � C xnyn:
EXAMPLE On R, the inner prod-
uct of x; y 2 R is defined by
hx; yi D xy:
» Fix Nx 2 R. Then h Nx; yi D Nxy is
linear in y.
» Fix Ny 2 R. Then hx; Nyi D x Ny is
linear in x. xy
,Jianfei Shen (Shandong University) 5/31
Properties of Inner Product on Rn
» hx;xi D x21 C � � � C x2n D kxk2.» hx;xi > 0 for all x 2 Rn, with equality iff x D 0.
» If y 2 Rn is fixed, then the map x 7! hx;yi is linear.
» hx;yi D hy;xi for all x;y 2 Rn.
» Bilinearity:
hx C u;yi D hx;yi C hu;yi ;hx;y C vi D hx;yi C hx; vi :
,Jianfei Shen (Shandong University) 6/31
Interpretation of Inner Product on Rn
» On Rn we have
ku � vk2 D hu � v;u � viD hu;ui � 2 hu; vi C hv; viD kuk2 � 2 hu; vi C kvk2:
(1)
» kuk, kvk and ku � vk have the same value in any Cartesian
coordinate system.
» It follows from (1) that hu; vi has the same value in any Cartesian
coordinate system.
,Jianfei Shen (Shandong University) 7/31
» Chose a coordinate axes, the first one
through u, the second so that v is
contained in the plane spanned by the
first two axes.
» The coordinates of u and v in this new
coordinate system are
u D �kuk; 0; : : : ; 0�v D �kvk cos �; : : :
�:
x
y
z
u
v
kuk
v
x0
y0
kvk cos��
kvk
» Therefore,
hu; vi D kuk � kvk cos �; (2)
� the angle between u and v.
,Jianfei Shen (Shandong University) 8/31
Law of Cosine
» The vectors x and y form a triangle.
» Relations (1) and (2) can be written as
kx�yk2 D kxk2Ckyk2�2kxkkyk cos �:0
x
y
x�
y
�
» If � D �2
, we get the Pythagorean Theorem.
,Jianfei Shen (Shandong University) 9/31
1 Euclidean Structure
2 Inner-Product Spaces
3 Orthonormal Bases
,Jianfei Shen (Shandong University) 10/31
Inner Product on Vector Spaces
DEFINITION An inner product on V is a function that takes each
ordered pair .u; v/ of elements of V to a number hu; vi 2 R and has
the following properties:
» hv; vi > 0 for all v 2 V ;
» hv; vi D 0 iff v D 0;
» huC v;wi D hu;wi C hv;wi for all u; v;w 2 V ;
» hav;wi D a hv;wi for all a 2 R and v;w 2 V ;
» hv;wi D hw; vi.
DEFINITION An inner-product space is a vector space V along
with an inner product on V .,
Jianfei Shen (Shandong University) 11/31
Norms
» For v 2 V , we define the norm of v, denoted kvk, by
kvk Dphv; vi:
» Then for every u; v 2 V ,
kuC vk2 D huC v;uC vi D kuk2 C 2 hu; vi C kvk2: (3)
» Eq. (3) is called the polar identity.
,Jianfei Shen (Shandong University) 12/31
Orthogonal Vectors
» The greatest advantage of an inner product space is its
underlying concept of orthogonality.
DEFINITION Let V be an inner product space. We call u; v 2 Vorthogonal and write u ? v if hu; vi D 0.
» 0 ? v for every v 2 V .
» 0 is the only vector that is orthogonal to itself:
v ? v ” hv; vi D 0 ” v D 0:
,Jianfei Shen (Shandong University) 13/31
Pythagorean Theorem
THEOREM If u and v are orthogonal vectors in V , then
kuC vk2 D kuk2 C kvk2 :
Proof. If u ? v, then
kuC vk2 D huC v;uC viD kuk2 C kvk2 C 2 hu; viD kuk2 C kvk2 : ut
,Jianfei Shen (Shandong University) 14/31
Orthogonal Decomposition
» u; v 2 V with v ¤ 0.
» We seek w 2 V and a 2 R so that˚
u D avCw
w ? v
» Let u D avC .u � av/˜w
. So w ? v implies
u
U
v
av
w
hu � av; vi D 0 H) a D hu; viıkvk2:» Therefore,
u D hu; vikvk2 vC
u � hu; vikvk2 v
!: (6.5)
,Jianfei Shen (Shandong University) 15/31
Example
» Let V D R2 and u D .1; 1/.» If v D .x; y/ 2 R2, then a D xCy
x2Cy2.
�20
2 �2
0
2�2
0
2
xy
a
�20
2 �2
0
2�2
0
2
xy
a
,Jianfei Shen (Shandong University) 16/31
Cauchy-Schwarz Inequality
THEOREM If u; v 2 V , thenˇ̌hu; viˇ̌ 6 kuk kvk. This inequality is an
equality if and only if one of u, v is a scalar multiple of the other.
Proof. We suppose that v ¤ 0.
» The orthogonal decomposition: u D hu;vi
kvk2 vCw, where hv;wi D 0.
» By the Pythagorean theorem:
kuk2 D hu; vikvk2 v
2
C kwk2 D hu; vi2
kvk2 C kwk2 >
hu; vi2kvk2 :
» The Cauchy-Schwarz inequality holds with equality iff w D 0 iff u
is a multiple of v. ut
,Jianfei Shen (Shandong University) 17/31
u
U
v
av
w
ˇ̌hu; viˇ̌ < kuk � kvk
v
ˇ̌hu; viˇ̌ D kuk � kvk
,Jianfei Shen (Shandong University) 18/31
Example
» Let V D R2 and u D .1; 2/.» Then
ˇ̌hu; viˇ̌ Djx C 2yj and
kukkyk Dq5.x2 C y2/. �2
02 �2
0
20
5
xy
,Jianfei Shen (Shandong University) 19/31
Triangle Inequality
THEOREM If u; v 2 V , then kuC vk 6 kuk C kvk. This inequality
is an equality if and only if one of u, v is a nonnegative multiple of
the other.
Proof. We have
kuC vk2 D huC v;uC viD kuk2 C 2 hu; vi C kvk26 kuk2 C 2 kuk kvk C kvk2
D �kuk C kvk�2 : u
vuC
v
» The triangle inequality holds with equality iff hu; vi D kuk kvk iff
one of u, v is a nonnegative multiple of the other.
,Jianfei Shen (Shandong University) 20/31
Example
» Let V D R2 and u D .1; 1/.» Then
kuC vk Dq.1C x/2 C .1C y/2
and
kuk C kyk Dp2C
qx2 C y2.
�20
2 �2
0
20
2
4
xy
,Jianfei Shen (Shandong University) 21/31
Parallelogram Equality
THEOREM If u; v 2 V , then
kuC vk2 C ku � vk2 D 2�kuk2 C kvk2
�:
Proof. The polar identity (3) implies that
kuC vk2 D kuk2 C 2 hu; vi C kvk2ku � vk2 D kuk2 � 2 hu; vi C kvk2:
Now add. ut
u
v
uC vu �v
,Jianfei Shen (Shandong University) 22/31
1 Euclidean Structure
2 Inner-Product Spaces
3 Orthonormal Bases
,Jianfei Shen (Shandong University) 23/31
Orthonormal Bases
DEFINITION A list of vectors is called orthonormal if the vectors
in it are pairwise orthogonal and each vector has norm 1.
» A list .e1; : : : ; em/ of vectors in V
is orthonormal if
˝ej ; ek
˛ D˚0 if j ¤ k1 if j D k:
» The standard basis in Rn is
orthonormal.
x
y
z
e1
e2
e3
,Jianfei Shen (Shandong University) 24/31
Properties of Orthonormal Vectors
PROPOSITION If .e1; : : : ; em/ is an orthonormal list of vectors in
V , then for all a1; : : : ; am 2 R,
ka1e1 C � � � C amemk2 D a21 C � � � C a2m:
Proof. We apply the Pythagorean Theorem repeatedly:
» ha1e1; a2e2 C � � � C amemi D a1a2 he1; e2i C � � � C a1am he1; emi D 0» Hence, .a1e1/ ? .a2e2 C � � � C amem/
» We now have
ka1e1 C .a2e2 C � � � C amem/k2 D a21 C ka2e2 C � � � C amemk2D a21 C a22 C ka3e3 C � � � C amemk2D a21 C � � � C a2m: ut
,Jianfei Shen (Shandong University) 25/31
Orthonormal Vectors and Linear Independence
PROPOSITION Every orthonormal list of vectors is linearly inde-
pendent.
Proof. Suppose .e1; : : : ; em/ is orthonormal and a1; : : : ; am 2 R
are such that
a1e1 C � � � C amem D 0:
Then a21 C � � � C a2m D 0 H) a1 D � � � D am D 0. ut
,Jianfei Shen (Shandong University) 26/31
Orthonormal Bases
DEFINITION An orthonormal basis (ONB) of V is an orthonor-
mal list of vectors in V that is also a basis of V .
» The standard basis .e1; : : : ; en/ is an ONB of Rn.
» Every orthonormal list of vectors in V with length dim.V / is an
ONB of V .
» For instance, the following list �1
2;1
2;1
2;1
2
�;
�1
2;1
2;�12;�12
�;
�1
2;�12;�12;1
2
�;
��12;1
2;�12;1
2
�!
is an ONB of R4.
,Jianfei Shen (Shandong University) 27/31
The Coordinate Matrix w.r.t. ONB
THEOREM Suppose O D .e1; : : : ; en/ is an ONB of V . Then
Œv�O D
2664hv; e1i:::
hv; eni
3775 ; (6.18)
and
kvk2 D hv; e1i2 C � � � C hv; eni2 ; (6.19)
for every v 2 V .
,Jianfei Shen (Shandong University) 28/31
Proof
» O is a basis H) v D a1e1 C � � � C anen
»˝v; ej
˛ D aj» kvk2 D a21 C � � � C a2n D hv; e1i2 C � � � C hv; eni2
,Jianfei Shen (Shandong University) 29/31
The Gram-Schmidt Orthogonalization Process
THEOREM If .v1; : : : ; vm/ is a linearly independent list of vectors
in V , then there exists an orthonormal list .e1; : : : ; em/ of vectors in
V such that
span.v1; : : : ; vj / D span.e1; : : : ; ej /
for j D 1; : : : ; m.
Proof. I postpone the proof until the next lecture. ut
,Jianfei Shen (Shandong University) 30/31
Corollaries
COROLLARY Every finite-dimensional inner-product space has an
ONB.
COROLLARY Every orthonormal list of vectors in V can be ex-
tended to an ONB of V .
,Jianfei Shen (Shandong University) 31/31