Scientific Computing

QR FactorizationPart 1 – Orthogonal Matrices,

Reflections and Rotations

Orthogonal Matrices

Matrix Factoring: So far, we have seen that we can factor a non-singular matrix A as A = LU, where L is lower triangular and U is upper triangular.

QR Factoring: Similar idea to factor A = QR, where R is upper triangular and Q is orthogonal.

Definition: A square matrix Q is orthogonal ifQt Q = I

Orthogonal Matrix Properties

• Multiplication of a vector by an orthogonal matrix preserves Euclidean norm

• In general, multiplication by an orthogonal matrix Q preserves inner products, angles, and distances.

Qv 2

2 (Qv)tQv v tQtQv v tv v 2

2

Rotations

• Rotation matrices are one example of orthogonal matrices. In R2 the matrix

applied to a vector x rotates x counter-clockwise about the origin.

Qcos( ) sin( )sin( ) cos( )

θ

v

Qv

Rotations

• In general, the matrix

acts as a rotation in the (xi,xj) plane

Q

1

O

1

cos( ) sin( )1

O

1

sin( ) cos( )1

O

1

i

j

Reflections

• Let w be a unit vector (column vector). The matrix acts as a reflection across the line (plane) perpendicular to w.

QI 2ww t

w

v

Qv

O

(I 2ww t )w w 2ww tw w 2w w

(I 2ww t )2 I 4ww t 4ww tww t

I 4ww t 4ww t I

Householder Transformations

• A Householder transformation (also known as Householder reflection or elementary reflector) is a linear transformation that describes a reflection about a plane (or hyperplane ) containing the origin. The Householder transformation was introduced in 1958 by Alston Scott Householder.

(Wikipedia)

Effect of Householder Transformations

• Householder transformations can be used to zero out entries of a matrix below the diagonal.

• Let A = [A1 A2 … An] be a square matrix, with columns Ai.

• Let Let v1 = A1 – α1 e1 where e1 =(1,0,….,0)

Then,

2

2121211,11 )sgn( AThenAa

(**)22

2)()(

1,1121

211,11

2

211111112

21

a

aAeAeAv

Effect of Householder Transformations

• Now, let Then, w1 is a unit vector.

Thus, H1 = I – 2 w1 w1t is a reflection (Householder transf)

The first column of H1 A will be H1 A1 .

21

11 v

vw

1111111111

2

211,1121

1,112

21111111

1121

11111111

)(,

)(**)(22

22)(22,

22

eeAAvAAHThus

slidepreviousfrombyva

aAAeAAvNow

Avv

vAAwwAAH

t

tt

Effect of Householder Transformations

• Thus,

1,

12,

1,2

12,2

1,1

12,11

1

0

0

nnn

n

n

aa

aa

aa

AH

QR Factorization

Theorem: If A is any square matrix of size n, then, there is an orthogonal matrix Q, and an upper triangular matrix R, such that A = QR.

Proof: The construction above shows that there is a Householder matrix H1 such that the first column of

H1 A has zeros below the first entry. Proceed by induction. Suppose, after step k-1 we have found a series of matrices H1, H2, …, Hk-1 such that

Hk-1 …H2 H1 A has all zeroes below the diagonal to position (k-1,k-1).

QR Factorization

1,

1,

1,1

1,11

1,1

11,1

12,11

121

000

00

0

knn

kkn

knk

kkkk

kn

kk

k

k

aa

aa

aaa

AHHH

QR FactorizationProof:(continued) . We will zero out the column below the (k,k)

entry as follows: Let

Then, Let

Note:

Thus, where Ik is a kxk identity matrix.

),,,,0,,0()sgn(

),,,,0,,0(

,,1,,

,,1,

kkn

kkk

kkk

kkkk

kkn

kkkk

kkkk

aaaa

aaav

kkkkkk av ,

22

222 t

kkkk

kk wwIHand

v

vw 2

2

**00

**00

0000

**0

*

*

0

tkkww

B

IwwIH tkkkk 0

0)2(

QR FactorizationProof:(continued) We see that Hk will only affect the lower (n-k)x(n-k) portion of the

current matrix. Thus, as before, Hk( Hk-1 …H2 H1 A ) will zero out the column below the (k,k) entry.

Thus, we have by induction, that we can find a sequence fo Householder transformations such that ( Hn-1 …H2 H1 A) = R,

where R is upper triangular. Then, A = H1-1

H2-1

… Hn-1-1 R. Since

Hi-1 =Hi , we have A = H1 H2 … Hn-1 R. Let Q = H1 H2 … Hn-1 . Since Q is

a product of orthogonal matrics, it is also orthogonal. QED.

B

IwwIH tkkkk 0

0)2(

QR ExampleTo derive H1 for the given matrix A :

We have :

Thus,

So,

074

212

034

4

2

4

1A

4

2

10

0

0

1

6

4

2

4

6

4

2

4

1 11 v

ttt

w

30

2,

30

1,

30

5

120

4,

120

2,

120

10

4210

4,2,10

2221

15

11

15

2

3

215

2

15

14

3

13

2

3

1

3

2

15

2

15

1

3

115

1

30

1

6

13

1

6

1

6

5

230

2,

30

1,

30

5

30

230

130

5

221 IIwwIH T

Example con’tThen,

k=2:

15

430

15

2810

3

276

074

212

034

15

11

15

2

3

215

2

15

14

3

13

2

3

1

3

2

1AH

3

1

7)2(

2A

3

101

0

0

1

0

10

3

1

0

10

3

1

0

1 22 v

1623.4101

1307.5

3,

1307.5

1623.4,

1307.5

0

9

3,,0

22

b

b

bw

tt

3162.09487.00

9487.03162.00

001

94868.120

4868.123246.170

000

3246.26

23,,0

3

0

9

22

2222 Ibbb

IwwIH t

Example con’tThen,

So, A = H1 H2 R = Q R where

RAH

6865.100

8433.01623.303

276

15

430

15

2810

3

276

3162.09487.00

9487.03162.00

001

2

1054.07379.06667.0

8433.04216.03333.0

5270.05271.06667.0

3162.09487.00

9487.03162.00

001

15

11

15

2

3

215

2

15

14

3

13

2

3

1

3

2

21HHQ

Example con’tFrom Matlab:>> A=[4 3 0; 2 1 2; 4 7 0];>> [q r] = qr(A)q = 0.6667 -0.5270 -0.5270 0.3333 -0.4216 0.8433 0.6667 0.7379 0.1054r = 6.0000 7.0000 0.6667 0 3.1623 -0.8433 0 0 1.6865 Note the slight difference between matlab’s result and our algorithm result. Matlab always uses αk values so that the diagonal of R has non-negative values.

QR Factorization

Note: The factorization of A=QR is not unique!