linear algebra with matlab

8/8/2019 Linear Algebra With Matlab

1/5

Basic Linear Algebra in MATLAB

Russ Tedrake

9.29 Optional Lecture 2: February 13, 2002

In the last optional lecture we learned the the basic type in MATLAB is a matrix

of double precision floating point numbers. You learned a number of different tools

for initializing matrices and some basic functions that used them. This time, well

make sure that we understand the basic algebraic operations that can be performed on

matrices, and how we can use them to solve a set of linear equations.

A Note on Notation

The convention used in this lecture and in most linear algebra books is that an italics

lower case letter (k) denotes a scalar, a bold lower case letter (x) denotes a vector, and

a capital letter (A) denotes a matrix. Typically we name our MATLAB variables with

a capital letter if they will be used as matrices, and lower case for scalars and vectors.

1 Vector Algebra

Remember that in MATLAB, a vector is simply a matrix with the size of one dimension

equal to 1. We should distinguish between a row vector (a 1xn matrix) and a column

vector (an nx1 matrix). Recall that we change a row vector x into a column vector

using the transpose operator (x in MATLAB). The same trick works for changing a

column vector into a row vector.

We can add two vectors, x and y, together if they have the same dimensions. The

resulting vector z = x+y is simply an element by element addition of the componentsofx and y: zi = xi + yi. From this is follows that vector addition is both commutativeand associative, just like regular addition. MATLAB also allows you to add a scalar k

(a 1x1 matrix) to a vector. The result ofz = x + k is the element by element additionzi = k + xi.

Vector multiplication can take a few different forms. First of all, if we multiply a

scalar k times a vector x, the result is a vector with the same dimension as x: z = kximplies zi = kxi. There are two standard ways to multiply two vectors together: theinner product and the outer product.

The inner product, sometimes called the dot product, is the result of multiplying a

row vector times a column vector. The result is a scalar z = xy =i

xiyi. To take

the inner product of two column vectors, use z = x

y. As well see, the orientation ofthe vectors matters because MATLAB treats vectors as matrices.

1


2/5

Unlike the inner product, the result of the outer product of two vectors is a matrix.

In MATLAB, you get the outer product my multiplying a column vector times a row

vector: Z = xy. The components ofZ are Zij = xiyj . To take the outer product oftwo column vectors, use Z = xy.

Occassionally, what we really want to do is to multiply two vectors together ele-

ment by element: zi = xiyi. MATLAB provides the .* command for this operation:

z = x. y.To test our understanding, lets try some basic matlab commands:

x = 1:5

y = 6:10

x+y

x+5

5*x

x*y

x*y

x.*y

How would you initialize the following matrix in MATLAB using outer products?

1 2 3 4 51 2 3 4 51 2 3 4 51 2 3 4 51 2 3 4 5

2 Matrix Algebra

The matrix operations are simply generalizations of the vector operations when the

matrix has multiple rows and columns. You can think of a matrix as a set of row

vectors or as a set of column vectors.

Matrix addition works element by element, just like vector addition. It is defined

for any two matrices of the same size. C = A + B implies that Cij

= Aij

+ Bij

. Once

again, it is both commutative and associative. Scalar multiplication of matrices is also

defined as it was with vectors. The result is a matrix: C = kA implies Cij = kAij .If you multiply a matrix times a column vector, then the result is another column

vector - the column of inner products: b = Ax implies bi =

j Aijxj . Similarly,

you can multiply a row vector times a matrix to get a row of inner products: b = xAimplies bi =

j Ajixj . Notice that in both cases, the definitions require that the first

variable must have the same number of columns as the the second variable has rows.

This idea generalizes to multiplying two matrices together. For the multiplication

C = AB, the matrix C is simply a collection of inner products: Cik =

j AijBjk .

In this case, A must have the same number of columns as B has rows. Like ordinary

multiplication, matrix multiplication is associative and distributive, but unlike ordinary

multiplication, it is not commutative. In general, AB = BA.

Now we are in a position to better understand the matrix transpose. IfB = A

, thenBij = Aji. Think of this as flipping the matrix along the diagonal. This explains why

2


3/5

the transpose operator changes a row vector into a column vector and vice versa. The

following identity holds for the definitions of multiplication and transpose: (AB) =BA. This help us to understand the difference between xA and Ax. Notice that for

column vector x, (Ax) = xA.There are a few more matrix terms we should know. A square matrix is an nxn ma-

trix (it has the same number of rows and columns). A diagonal matrix A has non-zero

elements only along the diagonal (Aii), and zeros everywhere else. You can initializea diagonal matrix in MATLAB by passing a vector to the diag command. The iden-

tity matrix is a special diagonal matrix with all diagonal elements set to 1. You can

initialize an idenitity matrix using the eye command.

Try the following matlab commands:

diag(1:5)

3 Solving Linear Equations

Lets take a step back for a moment, and try to solve the following set of linear equa-

tions:

x1 + 3x2 = 4

2x1 + 2x2 = 9

With a little manipulation, we find that x1 = 4.75 and x2 = 0.25. We could solvethis set of equations because we had 2 equations and 2 unknowns. How should we

solve a set of equations with 50 equations and 50 unknowns?

Lets rewrite the previous expression in matrix form:

1 32 2

x1x2

=

49

Notice that we could use the same form, Ax = b, for our set of 50 equations with 50unknowns. As expected, MATLAB provides all of the tools that we need to solve this

matrix formula, and it uses the idea of a matrix inverse.The inverse of a square matrix A, which is A1 in the textbooks and inv(A) in

MATLAB, has the property that A1A = AA1 = I. Using this, lets manipulate ourprevious equation:

Ax = b

A1Ax = A1b

x = A1b

Now solve the original equations in MATLAB using inv(A) b. You should get thevector containing 4.75 and -0.25.

There are a few things to remember about matrix inverses. First of all, they are only

defined for square matrices. It works with the transpose and multiplication operationswith the following identities: (A)1 = (A1) and (AB)1 = B1A1. You should

3


4/5

be able to verify these properties on your own using the ideas weve developed. But the

most important thing to know about matrix inverses is that they dont always exist, even

for square matrices. Using MATLAB, try taking the inverse of the following matrix:

A =

1 22 4

Now try inserting this A into the system of equations at the beginning of this section,

and solving it using good old fashioned algebra. Why doesnt the inverse exist?

4 Quadratic Optimization

We would like to solve the equation Ax = b even ifA is not square. Lets seperate theproblem into a few cases where the matrix A is an mxn matrix:

If m < n, then we have more unknowns than equations. In general, this system

will have infinitely many solutions.

If m > n, then we have more equations than unknowns. In general, this system

doesnt have any solution. What if we dont want to have matlab always return no

solution, but we actually want the closest solution in the least squares sense? This is

equivalent to minimizing the following quantity:

E =1

2

i

(j

Aijxj bi)2

MATLAB provides the backslash operator to accomplish the least squares fit for a

matrix equation: x = A \b. Type help slash to appreciate the power of this command.Youll see that we could have used this command to solve the square matrix equations,

too.

5 Eigenmannia

How does this relate to the fish data that we used in problem set 1? Recall that we weregiven a set of points (xi, yi), and we were asked to find the coefficients a and b to fitthe following linear model:

yi a + bxi

You can think of each point as an equation, and write the entire data set in matrix form:

1 x11 x2...

...

1 xm

a

b

=

y1y2...

ym

If you call the left matrix A and the right side b, then calling A \b will ask MATLAB

to solve for the values ofa and b that minimize the least-squared error of the model. Itwill return exactly the same values for a and b that the polyfitcommand returns.

4


5/5

For this simple example, the polyfitcommand and the backslash command accom-

plished the same task. But what if you were given a set of points (xi, yi, zi) and youwere asked to fit the following linear model:

zi a + bxi + cyi

The matrix notation easily scales to this problem, but the polyfitfunction does not.

6 Where to find more information

A good book on this material is Introduction to Linear Algebra by Gilbert Strang. All

of the material weve covered here can be found in the first two chapters of that book.

Sebastian also has a number matrix handouts from different courses which we can post

to the web if there is a demand for them. Make sure that you post to the class news

group athena.classes.9.29 if you have any questions.

5

linear algebra with matlab

Documents