matrix methods and pattern recognition - wordpress.com · 1.2 computing problem 1.2.1 write a code...

Matrix Methods andPattern Recognition

Michael Tagare De Guzman

January 31, 2012

Set One

1.1 Theory

Let the basis B1 be the standard basis, i.e., e(1) = (0,1)T, e(2) = (1,0)T, and the basis B2 be givenby the two vectors v(1) = (1,1)T, v(2) = (−1,1)T. Given uB1 = (1,1)T, find uB2

Before we can answer this question, let us review the theory of change of basis. Let {v(i)}ni=1 and{w(i)}ni=1 both be bases for Rn, call them B1 and B2, respectively. We let u be an arbitrary element ofRn. We can u in terms of the basis, B1 and B2, respectively, as a linear combination:

u = x1v1 + x2v2 + ...+ xnvn

u = y1w1 + y2w2 + ...+ ynwn

which give the representation, or the coordinates,

uB1 = (x1....xn)T

with respect to B1 and,

uB2 = (y1....yn)T

with respect to B2. This can be also written as u = VuB1 = WuB2 . Now, we can solve our originalproblem given uB1 = (1,1)T, find uB2

using a change of basis.

Solution. To find uB2, use the change of basis

B1 =

{[10

],

[01

]}B2 =

{[11

],

[−11

]}[1 00 1

]uB1 =

[1 −11 1

]uB2[

1 00 1

] [11

]=

[1 −11 1

]uB2[

11

]=

[1 −11 1

]uB2

Now, taking the inverse of both sides

uB2 =

[1 −11 1

]−1 [11

]=

1

2

[1 1−1 1

] [11

]=

[10

]

1

1.2 Computing

Problem 1.2.1

Write a code to generate 1000 random numbers contained on the unit circle. Apply several randommatrices to this data and describe your results in the terminology of bases and change of bases. Howdo your results differ if the multiplying matrix is constrained to be orthogonal?

Solution. Also Refer to Matlab codes in the Appendix below in Section 1.2.1 . For the unit circle,0 ≤ α ≤ 2π is a random number which will generate one row and one thousand columns, which will beused in a matrix A, to construct the unit circle. When we apply the transformations by multiplying, weare multiplying the coordinates of the new base for which it will be sent to. Notice in the figures thechanges as from the original unit circle and the following three transformations. The results differ usingan orthogonal matrix, since we just get our unit circle again when we multiply by an orthogonal matrix

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

−1500 −1000 −500 0 500 1000 1500−200

−150

−100

−50

0

50

100

150

200

Figure 1: Unit Circle and its first transformation

2

−1000 −800 −600 −400 −200 0 200 400 600 800 1000−800

−600

−400

−200

0

200

400

600

800

−200 −150 −100 −50 0 50 100 150 200−1000

−800

−600

−400

−200

0

200

400

600

800

1000

Figure 2: Transformations on Unit Circle

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1orthogonal left multiplied

Figure 3: Unit Circle and its Orthogonal transformation, respectively

Problem 1.2.2

Given an algorithm for computing small principal angles between two subspaces given by the realmatrices X and Y , where X is in Rn× p and Y is in Rn× q (Principal angles are defined to be between0 and π

2 and listed in ascending order).

(a) Implement this algorithm in MATLAB under the function name prinAngles. Your function shouldhave the input and output arguments:

3

[theta] = prinAngles(X1,X2)

where θ is the principal angles listed from the smallest to the largest, X1 is the first set of images listedby the columns, and X2 is the second set of images listed the columns as well.

Solution. From the pseudocode, the only issue here was what function to use to generate theorthonormal bases for the two matrices, respectively. Before it was brought up during lecture, aclassmate and I discussed the possible ways to generate these matrices using either the commands inMatlab ’orth()’ or ’qr()’. Applying either command generated orthonormal bases for the inputedmatrices, X1,X2. For a further look, we can refer back to the first Matlab code in the Appendix belowfor Section 1.2.2 . For part (b), we test the images and list θ from the output.

(b) To verify that your implementation is correct, download face1.mat and face2.mat from the coursewebsitewhere face1.mat contains 21 distinct images of person 1 in its columns and face2.mat contains21 distinct images of person 2 in its columns and test your implementation with this data. Note: incase you want to see what the images look like, the images are of resolution 160 x 138. The followingMATLAB commands will display the first image of person 1:

load face1 imagesc ( reshape(face1(:,1),160,138)), colormap(gray),axis off

Solution. With the script file loadimages, which can also be found in Appendix in Section 1.2.2, wearrived at θ in ascending order as listed: 0.3026, 0.4689, 0.7318, 1.0081, 1.1159, 1.2027, 1.2276, 1.2915,1.3709, 1.3860, 1.4227, 1.4294, 1.4682, 1.4865, 1.5117, 1.5185, 1.5286, 1.5433, 1.5480, 1.5573, 1.5690Below these are one of the images in each set of the figures we used to get the solution. At first I noticethere was an issue here since the first three principle angles were 0, which would mean the anglebetween the two images taken from the two sets, respectively, are the same. So I had to look at mycode one more time and noticed I needed to sort the diagonals after the commands for svd. The anglesare given in increasing order by the Matlab command ’sort()’. Also, initially I had trouble loading theimages and this was a syntax issue with the commands figure and hold on before the for loop is calledin the main script file for problem 2 and before the main function.

Figure 4: Face1 and Face2

Problem 1.2.3

Write a function in MATLAB using the homogeneous coordinates that scales (enlarge and shrink) a 2Dimage about a point P = [tx, ty, 1]’. Specifically, the first line of your function will be (other than thecomments)

function [newImg] = scale(Img, alpha, P)

where α = [sx, sy]’ is the scale parameter that controls how much scaling is applied in x-direction andin y-direction, respectively. Make sure your routine works by applying it to an image of your choice.The MATLAB commands that are useful here:

meshgrid, interp2, imread, imagesc, reshape

4

For example, to find out how to use meshgrid, type

Help meshgrid

in the MATLAB command prompt.

Solution. Notice in figure 5, the left image of Figure 5 is the original color picture before anyreshaping, grayscaling, and scaling applied. The image after loading as a matrix needed to be reshapedand grayscaled causing this difference in size in order to apply calculation without using all the datafrom the color and size of the original image. Otherwise, an issue with memory and computationalspeed would arise. In figure 6, using the point [100, 100, 100]’, which is a point about Supergirl’s hair,scaling was applied using a factor of 5 that stretched out that portion of the image. Refer to Matlabcodes in the Appendix below in Section 1.2.3 .

grayscaled unstretched original

Figure 5: Original Color Supergirl and Grayscaled

5


scale(’supergirl.jpg’, [5, 5]’, [100, 100, 100]’)

50 100 150 200 250 300 350

50

100

150

200

250

300

350

400

450

500

Figure 6: Grayscaled and Scaled or Stretched images

Problem 1.2.4

Write a routine in MATLAB using the homogeneous coordinates that translates a 2D imagehorizontally and a routine that translates the image vertically. Specifically, one of your functionsshould have the input and output arguments

[newImg] = translateH(Img, tx)

where tx is the amount of horizontal translation applied (make sure it works for both positive andnegative values). Test your routine by applying it to an image of your choice.

Solution. The figures below are the horizontal and vertical translations, respectively with a x or ytranslation as a factor of 400 as the input. Refer to Matlab codes in the Appendix below in Section1.2.4 .

6

50 100 150 200 250 300 350

50

100

150

200

250

300

350

400

450

50050 100 150 200 250 300 350

50

100

150

200

250

300

350

400

450

500

Figure 7: Horizontal and Vertical Translations

Problem 1.2.5

Write a routine in MATLAB using the homogeneous coordinates that rotates a 2D image about a pointP = [tx, ty, 1]. Specifically, your function should have the input and output arguments

[newImg] = rotate[Img, theta, P]

where θ is the amount of rotation applied in counterclockwise orientation. Make sure your routineworks by applying it to an image of your choice.

Solution. Again, with the similar point as in the scaling of Supergirl’s hair at P = [100, 100, 100]’, werotate counterclockwise by an angle of π

4 . Refer to Matlab codes in the Appendix below in Section1.2.5 .

7


rotate(’supergirl.jpg’, pi.\4 , [100, 100, 100]’)

50 100 150 200 250 300 350

50

100

150

200

250

300

350

400

450

500

Figure 8: Gray and Pi Fourths Rotation about a Point (by Supergirl’s hair)

APPENDIX: MATLAB

1.2.1

% Michael Tagare De Guzman

% Problem 1.2.1

p = 1000;

ucircle(p)

% this will be the 1,000 points in the input

% that will create the boundary of the unit circle

% alpha is the random number btwn (0,1)x2pi , so

% we will have 1,000 random points marked as

%’+’ on the unit circle

transformations_on_ucircle(p)

% this will plot all the transformations for ucircle.

Deguzman M HW1 Problem1.m


% Homework One Problem 1 Part 1

function [A] = ucircle(p)

alpha = 2* pi* rand(1, p);

% p will be the 1,000 points in the input

% that will create the boundary of the unit circle

8

% alpha is the random number btwn (0,1)x2pi , so

% we will have 1,000 random points on the unit circle

x = cos(alpha); y = sin(alpha);

A = [x ; y];

% A will output a matrrix of

% 2 rows of x and y with p columns

figure; plot(x, y, ’+’);

end

ucircle.m


% Homework One Problem 1 Part 2

function [P] = transformations_on_ucircle(p)

A = ucircle(p);

for P = 1:3

P = randi (1000 , 2)* A;

% randi gives a random square matrix of 1000 values ,

% n = 2 is the size. we left multiply it with our ucircle matrix

figure; hold on; plot(P(1,:), P(2, :), ’o’);

% we want to plot the matrix of P. Recall x was the first row ,

% and y was the second row

end

% using a 2x2 orthogonal matrix such as the identity or reflection

% about the x-axis , we get geometrically ’a rigid rotation in space ’

B = [1 0; 0 -1]; %B = [1 0 ; 0 1]

T = B*A;

figure; hold on; title(’orthogonal left multiplied ’); plot(T(1,:), T(2, :), ’o’

)

end

% discussed the idea of the randi and for loop with Hector Valdez ,

% since we needed many random matrices , on Piazza

transformations on ucircle.m

1.2.2


% Problem 1.2.2 (b) Main code to test principle angles.

% simply run this script file pushing the green button

% and the angles will appear as outputs in command line

% using PrinAngles (), we load our images

% then , each file face1 and face2

% contains 21 distinct images of person 1, and

% person 2, respectively.

% we use these , then , reshape the matrices and apply principle angles

% it run within the for loop to get all the principle angles btwn each img

% btwn two sets of images.

figure; hold on; load(’face1.mat’);

figure; hold on; load(’face2.mat’);

9

% for i = 1:2 for shorter time

for i = 1:2

X1 = face1; X2 = face2;

A = imagesc(reshape( X1(:,i) ,160 ,138))’; colormap(gray); axis off;

B = imagesc(reshape( X2(:,i) ,160 ,138))’; colormap(gray); axis off;

end

[theta] = prinAngles(face1 ,face2);

% face1 and face2 are the subspaces of the image ,

% while the for loop shows us the image so A,B was not used



% Problem 1.2.2 (a)

% outputs principle angles between two matrices , as inputs

function [theta] = prinAngles(X1, X2)

%load face1

%imagesc(reshape( face1 (:,1) ,160 ,138)); colormap(gray); axis off;

% collaborated with Hector Valdez in generating orthonormal bases.

Qx = orth(X1); Qy = orth(X2);

%(b) take the SVD of cosine , Qx ’Qy , S will be my diagonal matrix ,

% we input a zero after the matrix in svd to save memory space

[H, S, Z] = svd(Qx ’*Qy , 0);

G = sort(S); % need to sort the diagonal terms

% (c) computing the matrix of Y

if rank(Qx) >= rank(Qy)

Y = Qy - Qx*(Qx ’*Qy);

else

Y = Qx - Qy*(Qy ’*Qx);

end

% (d) take SVD of sine , with Mu my diagonal

[h, Mu , z] = svd(Y, 0);

U = sort(Mu); % need to sort the diagonal terms

% (e) we want to compute the principal angles from 1 to the

% size of the range space of the diagonals so we need to define

% the size then use a for loop to get the values of theta and

% if statements for the conditions.

[i, j] = size(S);

for k = 1:i

if (G(k, k))^2 < 1/2

theta(k) = acos(S(k,k));

end

if(U(k, k))^2 <= 1/2

theta(k) = asin(Mu(k,k));

end

end

theta = sort(theta); % sorts least to greatest theta

theta ’

end

10

prinAngles.m

1.2.3


% Problem 1.2.3

% just run this file to test scale function

scale(’supergirl.jpg’, [5, 5]’, [100, 100, 100] ’)

% takes in the supergirl color image and will scale

% it by a factor of 5 at the point [100 ,100 ,100]



% Problem 1.2.3

% type in the command line ,

% scale(’supergirl.jpg ’, [5, 5]’, [100, 100, 100] ’)

% make sure the jpg of supergirl is loaded in the

% same directory as the matlab file

% this function will scale the picture and it will

% turn it will also turn the picture gray in order

% to deal with the regular matrix instead of the array

% in color.

function [newImg] = scale(Img , alpha , P)

% ’imread ’ loads the picture data into a Matrix Img ,

% and ’imagesc ’ displays the image

A = imread(Img);

% gray scale the colored picture

% this will return the size of the image in

% two variables

% discussed grayscaling with Hector

B = rgb2gray(A);

[m, n] = size(B);

% take meshgrid

[x, y] = meshgrid (1:n, 1:m);

% create a matrix from meshgrid and from

% a row of ones

row_ones = ones(1, n*m);

Mxy = [ x(:)’ ; y(:)’ ; row_ones ];

% tx and ty are the first values

% of the point we are rotating about

% sx and sy we need to construct Msc , the

% transformation to scale and stretch

tx = P(1); ty = P(2);

sx = alpha (1); sy = alpha (2);

Msc = [sx , 0, -tx*(sx -1); 0, sy , -ty*(sy -1); 0, 0, 1];

11

% to gix the problem of scaling in

% to a larger or smaller image we take ,

M = inv(Msc);

% take the transformation matrix and

% multiply it to the matrix created with

% the data and rows of ones

Mnew = M* Mxy;

% use interp2 to fill in the missing pixels

% making it visually smooth

newImg = interp2( x, y, double(B), Mnew (1,:), Mnew (2,:), ’linear ’);

% file is too big so we reshape the image in

% in order for matlab to display

R = reshape(newImg , m, n);

% displays old and new images

figure (1); imshow(B); colormap(gray);

% do not use hold on since the image will not scale properly

figure (2); imagesc(R); colormap(gray);

end

scale.m

1.2.4


% Problem 1.2.4

% just run this file to test translateH function

translateH(’supergirl.jpg’, 400)

% calls in the translateH file which translates the image

% in the horizontal and vertical directions



% Problem 1.2.4

function [newImg] = translateH(Img ,tx)

% type in the command line ,

% translateH(’supergirl.jpg ’, 400)



A = imread(Img);



% two variables

B = rgb2gray(A);

[m, n] = size(B);

12

% take meshgrid



% a row of ones

row_ones = ones(1, n* m);

Mxy = [ x(:)’ ; y(:)’ ; row_ones ];

% Msch will be the horizontal translation

% Mscv will be the vertical translation

Msch = [1, 0, tx; 0, 1, 0; 0, 0, 1];

Mscv = [1, 0, 0; 0, 1, tx; 0, 0, 1];



Mh = inv(Msch); Mv = inv(Mscv);

% take the transformation matrix and



Mhnew = Mh* Mxy;

Mvnew = Mv* Mxy;

% use interp2 to filll in the missing pixels


newImgh = interp2( x, y, double(B), Mhnew (1,:), Mhnew (2,:), ’linear ’);

newImgv = interp2( x, y, double(B), Mvnew (1,:), Mvnew (2,:), ’linear ’);



H = reshape(newImgh , m, n);

V = reshape(newImgv , m, n);



figure (2); imagesc(H); colormap(gray);

figure (3); imagesc(V); colormap(gray);

end

translateH.m

1.2.5


% Problem 1.2.5

% just run this file to test rotate function

rotate(’supergirl.jpg’, pi.\4 , [100, 100, 100]’)

% calls in the rotate file which rotates the image

% counterclockwise by an angle of pi/4 at the point

% [100 ,100 ,100]



13

% Problem 1.2.5

function [newImg] = rotate(Img , theta , P)



A = imread(Img);



% two variables

% discussed grayscaling with Hector

B = rgb2gray(A); colormap(gray);

[m, n] = size(B);

% take meshgrid



% a row of ones

row_ones = ones(1, n* m);

Mxy = [ x(:)’ ; y(:)’ ; row_ones ];

% tx and ty are the first values

% of the point we are rotating about

% sx and sy we need to construct Msc , the

% transformation to scale and stretch

tx = P(1); ty = P(2);

Msc = [cos(theta), -sin(theta), tx;

sin(theta), cos(theta), ty;

0, 0, 1];



M = inv(Msc);

% take the rotation transformation matrix and



Mnew = M* Mxy;

% use interp2 to filll in the missing pixels


newImg = interp2( x, y, double(B), Mnew (1,:), Mnew (2,:), ’linear ’);



R = reshape(newImg , m, n);



figure (2); imagesc(R); colormap(gray);

end

rotate.m

14

References

[1] Jen-Mei Chang. Matrix Methods For Geometric Data Analysis and Pattern Recognition.

15

matrix methods and pattern recognition - wordpress.com · 1.2 computing problem 1.2.1 write a code...

Documents