lecture 10: vector algebra: orthogonal...
TRANSCRIPT
Lecture 10: Vector Algebra: Orthogonal Basis
• Orthogonal Basis of a subspace
• Computing an orthogonal basis for a subspace using Gram-Schmidt Orthogonalization Process
1
Orthogonal Set
• Any set of vectors that are mutually orthogonal, is a an orthogonal set.
Orthonormal Set
• Any set of unit vectors that are mutually orthogonal, is a an orthonormal set.
• In other words, any orthogonal set is an orthonormal set if all the vectors in the set are unit vectors.
• Example: ෞ𝒖1, ෞ𝒖2, ෞ𝒖3 is an orthonormal set, where,
ෞ𝒖1 =
3
111
111
11
, ෞ𝒖2 =
−1
62
61
6
, ෞ𝒖3 =
−1
66
−4
667
66
An orthogonal set is Linearly Independent
Projection of vector 𝒃 on vector 𝒂
• 𝒂 ∙ 𝒃 = 𝒂 𝒃 cos 𝜃
• Vector 𝒄 is the image/perpendicular projection of 𝒃 on 𝒂
• Direction of 𝒄 is the same as 𝒂
• Magnitude of 𝒄 is 𝒄 = 𝒃 cos 𝜃 =𝒂∙𝒃
𝒂
𝒄 = ෝ𝒂 ∙ 𝒃
• If ෝ𝒂 is the unit vector of 𝒂, then
• vector 𝒄 =𝒂∙𝒃
𝒂ෝ𝒂 =
𝒂∙𝒃
𝒂
𝒂
𝒂=
𝒂∙𝒃
𝒂∙𝒂𝒂 𝒃 cos 𝜃
𝒃 sin 𝜽𝒃
Orthogonal Basis
• An orthogonal basis for a subspace 𝑊 of 𝑅𝑛 is a basis for 𝑊 that is also an orthogonal set.
• Example: 100
,010
,001
is basically the 𝑥, 𝑦, and 𝑧 axis. It is an orthogonal basis in ℝ3,
and it spans the whole ℝ3 space. It is also an orthogonal set.
Orthogonal Basis
• We know that given a basis of a subspace, any vector in that subspace will be a linear combination of the basis vectors.
• For example, if 𝒖 𝑎𝑛𝑑 𝒗 are linearly independent and form the basis for a subspace S, then any vector 𝒚 in S can be expressed as:
𝒚 = 𝑐1𝒖 + 𝑐2𝒗
• But computing 𝑐1 and 𝑐2 is not straight forward.
• On the other hand, if 𝒖 𝑎𝑛𝑑 𝒗 form an orthogonal basis, then
𝑐1 =𝒚 ∙ 𝒖
𝒖=𝒚 ∙ 𝒖
𝒖 ∙ 𝒖and 𝑐2 =
𝒚 ∙ 𝒗
𝒗=𝒚 ∙ 𝒗
𝒗 ∙ 𝒗
𝒗
𝒖
𝒚
𝑐2
𝑐1
Does not work if it is not Orthogonal basis
• 𝒚 = 𝑐1𝒖 + 𝑐2𝒗
• But computing 𝑐1 and 𝑐2 is not straight forward (yet).
What is computed is,
• 𝑑1 =𝒚∙𝒖
𝒖=
𝒚∙𝒖
𝒖∙𝒖
• 𝑑2 =𝒚∙𝒗
𝒗=
𝒚∙𝒗
𝒗∙𝒗
8
𝒗
𝒖
𝒚
𝑐2
𝑐1𝑑1
𝑑2
Orthogonal Decomposition Theorem
Orthogonal Decomposition Theorem
Orthogonal Decomposition Theorem
Orthogonal Decomposition Theorem
Example
Projection of a vector on a Subspace
• 𝒖 and 𝒗 are orthogonal 3D vectors.
• They span a plane (green plane) in 3D
• 𝒚 is an arbitrary 3D vector out of the plane.
• 𝒚′ is the projection of 𝒚 onto the plane.
• 𝒚′ =𝒚∙𝒖
𝒖∙𝒖𝒖 +
𝒚∙𝒗
𝒗∙𝒗𝒗
• The “point” 𝒚′ is also
the closest point to 𝒚 on the plane.
𝒚 − 𝒚′is perpendicular to 𝒚′, Span{𝒖, 𝒗}, and hence 𝒖 𝑎𝑛𝑑 𝒗
𝒗
𝒖
𝒚
Perpendicular to 𝒚′, 𝒖, 𝒗
𝒚′
Closest point of a vector to a span does not depend on the basis of that span𝒚 is an arbitrary 3D vector out of the plane.• 𝒚′ is the projection of 𝒚 onto the plane.
• 𝒚′ = 𝑥1𝒖𝟏 + 𝑥2𝒖𝟐
• 𝒚′ = 𝑥3𝒖𝟑 + 𝑥4𝒖𝟒
• 𝒚′ = 𝑥5𝒖𝟏 + 𝑥6𝒖𝟑
• …• Coordinates of 𝒚′ change if the basis changes. • But the vector 𝒚′ itself does not change. • Hence the the closest point to 𝒚 on the plane does not change.Even here, 𝒚 − 𝒚′is perpendicular to 𝒚′ and Span{., . }
𝒖𝟏
𝒚
Perpendicular to 𝒚′and the
span
𝒚′𝒖𝟐
𝒖𝟑
𝒖𝟒
Closest point of a vector to a span does not depend on the basis of that span
• FINDING THE CLOSEST POINT OF A VECTOR TO A SPAN means :
“Finding the coordinates of the projection of the vector”
• So, if you want to compute the closest point of a vector to a span, then find an appropriate basis with which you can compute the coordinates of the projection easily.
What would be that basis? Answer: An orthogonal basis!
𝒖𝟏
𝒚
Perpendicular to 𝒚′and the
span
𝒚′𝒖𝟐
𝒖𝟑
𝒖𝟒
Projection on a span of non-orthogonal vectors• How to find projection of any arbitrary 3D vector onto
the span of two non-orthogonal, linearly independent vectors?
• 𝒖1and 𝒖2 are not orthogonal, but linearly independent vectors in 3D.
• 𝒚 is an arbitrary 3D vector.• Find the projection of 𝒚 in the space spanned by 𝒖1and 𝒖2.• a) First, find the orthogonal set of vectors 𝒗1and 𝒗2that span
the same subspace as 𝒖1and 𝒖2 . In other words, find an orthogonal basis.
• b) Project 𝒚 onto the space spanned by orthogonal 𝒗1and 𝒗2vectors, as we earlier.
𝒖𝟏
𝒚
𝒚′
𝒗𝟏
𝒖𝟐
𝒗𝟐
How to find an orthogonal basis?
How to find an orthogonal basis?
• Assume that the first vector 𝒖1 is in the orthogonal basis. Other vector(s) of the basis are computed that are perpendicular to 𝒖1
• Let 𝒗1 = 𝒖1• Let 𝒗2 = 𝒖2 −
𝒖2𝒗1
𝒗1𝒗1𝒗1
• We know that 𝒗2is perpendicular to 𝒗1.
• 𝒗2 is in the Span{𝒖1, 𝒖2} (Why?)
• So Span{𝒖1, 𝒖2} = Span{𝒗1, 𝒗2}
• And {𝒗1, 𝒗2} is an orthogonal basis
• Projection of 𝒚 on to the Span{𝒖1, 𝒖2}
𝒚′ =𝒚∙𝒗1
𝒗1∙𝒗1𝒗1 +
𝒚∙𝒗2
𝒗2∙𝒗2𝒗2
𝒖1
𝒚
𝒚′
𝒖2𝒗2
𝒗1
Gram-Schmidt Orthogonalization Process
Gram-Schmidt Orthogonalization Process
Example:
Solving Inconsistent Systems
Solving Inconsistent Systems
Solving Inconsistent Systems
Example 1:
A trader buys and/or sells tomatoes and potatoes. (Negative number means buys, positive number means sells.) In the process, he either makes profit (positive number) or loss (negative number). A week’s transaction is shown; find the approximate cost of tomatoes and potatoes.
Tomatoes(tons)
Potatoes(tons)
Profit/Loss (in thousands)
1 -6 -1
1 -2 2
1 1 1
1 7 6
Solving Inconsistent Systems
• 𝟏𝒕 - 6𝒑 = −𝟏
• 𝟏𝒕 - 2𝒑 = 𝟐
• 𝟏𝒕 + 1𝒑 = 𝟏
• 𝟏𝒕 + 7𝒑 = 𝟔
•
1111
𝑡 +
−6−217
p =
−1216
The above equation might not have a solution (values of t and p that would satisfy that equation). So the best we can do is to find the values of t and p that would result in a vector on the right hand side that is as close as possible to the desired right hand side vector.
Solving Inconsistent Systems
Example 2: