linear ization

LinearizationFrom Wikipedia, the free encyclopedia

Contents

1 Dierentiable function 11.1 Dierentiability and continuity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Dierentiability classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Dierentiability in higher dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.4 Dierentiability in complex analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.5 Dierentiable functions on manifolds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.6 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Linearization 62.1 Linearization of a function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.2 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.3 Linearization of a multivariable function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.4 Uses of linearization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.4.1 Stability analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.4.2 Microeconomics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.5 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.7 External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.7.1 Linearization tutorials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3 System of linear equations 93.1 Elementary example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.2 General form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

3.2.1 Vector equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.2.2 Matrix equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.3 Solution set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.3.1 Geometric interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.3.2 General behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.4 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.4.1 Independence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.4.2 Consistency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.4.3 Equivalence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

i

ii CONTENTS

3.5 Solving a linear system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.5.1 Describing the solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.5.2 Elimination of variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.5.3 Row reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.5.4 Cramers rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.5.5 Matrix solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.5.6 Other methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.6 Homogeneous systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203.6.1 Solution set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203.6.2 Relation to nonhomogeneous systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.7 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.8 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.9 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3.9.1 Textbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.10 Text and image sources, contributors, and licenses . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3.10.1 Text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.10.2 Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.10.3 Content license . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Chapter 1

Dierentiable function

A dierentiable function

In calculus (a branch of mathematics), a dierentiable function of one real variable is a function whose derivativeexists at each point in its domain. As a result, the graph of a dierentiable function must have a (non-vertical) tangentline at each point in its domain, be relatively smooth, and cannot contain any breaks, bends, or cusps.

1

2 CHAPTER 1. DIFFERENTIABLE FUNCTION

More generally, if x0 is a point in the domain of a function f, then f is said to be dierentiable at x0 if the derivativef (x0) exists. This means that the graph of f has a non-vertical tangent line at the point (x0, f(x0)). The function fmay also be called locally linear at x0, as it can be well approximated by a linear function near this point.

1.1 Dierentiability and continuitySee also: Continuous functionIf f is dierentiable at a point x0, then f must also be continuous at x0. In particular, any dierentiable function

1

2

3

4

3 2 1 1 2 30

y = |x|

The absolute value function is continuous (i.e. it has no gaps). It is dierentiable everywhere except at the point x = 0, where itmakes a sharp turn as it crosses the y-axis.

must be continuous at every point in its domain. The converse does not hold: a continuous function need not bedierentiable. For example, a function with a bend, cusp, or vertical tangent may be continuous, but fails to bedierentiable at the location of the anomaly.Most functions that occur in practice have derivatives at all points or at almost every point. However, a result ofStefan Banach states that the set of functions that have a derivative at some point is a meager set in the space ofall continuous functions.[1] Informally, this means that dierentiable functions are very atypical among continuousfunctions. The rst known example of a function that is continuous everywhere but dierentiable nowhere is theWeierstrass function.

1.2 Dierentiability classesA function f is said to be continuously dierentiable if the derivative f '(x) exists and is itself a continuous function.Though the derivative of a dierentiable function never has a jump discontinuity, it is possible for the derivative tohave an essential discontinuity. For example, the function

f(x) =

(x2 sin(1/x) ifx 6= 00 ifx = 0

1.2. DIFFERENTIABILITY CLASSES 3

1 1 2

xx

2

1

1

2

yy

An ordinary cusp on the cubic curve (semicubical parabola) x3 y2 = 0, which is equivalent to the multivalued function f(x) = x3/2. This relation is continuous, but is not dierentiable at the cusp.

is dierentiable at 0, since

f 0(0) = lim!0

2 sin(1/) 0

= 0;

exists. However, for x0,

4 CHAPTER 1. DIFFERENTIABLE FUNCTION

Dierentiable functions can be locally approximated by linear functions.

f 0(x) = 2x sin(1/x) cos(1/x)

which has no limit as x 0. Nevertheless, Darbouxs theorem implies that the derivative of any function satises theconclusion of the intermediate value theorem.Sometimes continuously dierentiable functions are said to be of class C1. A function is of class C2 if the rst andsecond derivative of the function both exist and are continuous. More generally, a function is said to be of class Ckif the rst k derivatives f(x), f(x), ..., f(k)(x) all exist and are continuous. If derivatives f(n) exist for all positiveintegers n, the function is smooth or equivalently, of class C.

1.3 Dierentiability in higher dimensionsSee also: Multivariable calculus

If all the partial derivatives of a function all exist and are continuous in a neighborhood of a point, then the functionmust be dierentiable at that point, and it is of class C1.Formally, a function of several real variables f: Rm Rn is said to be dierentiable at a point x0 if there exists alinear map J: Rm Rn such that

1.4. DIFFERENTIABILITY IN COMPLEX ANALYSIS 5

limh!0

kf(x0 + h) f(x0) J(h)kRnkhkRm = 0:

If a function is dierentiable at x0, then all of the partial derivatives must exist at x0, in which case the linear mapJ is given by the Jacobian matrix. A similar formulation of the higher-dimensional derivative is provided by thefundamental increment lemma found in single-variable calculus.Note that existence of the partial derivatives (or even all of the directional derivatives) does not in general guaranteethat a function is dierentiable at a point. For example, the function f: R2 R dened by

f(x; y) =

(x ify 6= x20 ify = x2

is not dierentiable at (0, 0), but all of the partial derivatives and directional derivatives exist at this point. For acontinuous example, the function

f(x; y) =

(y3/(x2 + y2) if(x; y) 6= (0; 0)0 if(x; y) = (0; 0)

is not dierentiable at (0, 0), but again all of the partial derivatives and directional derivatives exist.

1.4 Dierentiability in complex analysisMain article: Holomorphic function

In complex analysis, any function that is complex-dierentiable in a neighborhood of a point is called holomorphic.Such a function is necessarily innitely dierentiable, and in fact analytic.

1.5 Dierentiable functions on manifoldsSee also: Dierentiable manifold Dierentiable functions

If M is a dierentiable manifold, a real or complex-valued function f on M is said to be dierentiable at a point p ifit is dierentiable with respect to some (or any) coordinate chart dened around p. More generally, if M and N aredierentiable manifolds, a function f: M N is said to be dierentiable at a point p if it is dierentiable with respectto some (or any) coordinate charts dened around p and f(p).

1.6 See also Generalizations of the derivative Semi-dierentiability

1.7 References[1] Banach, S. (1931). "ber die Bairesche Kategorie gewisser Funktionenmengen. Studia. Math. 3 (1): 174179.. Cited

by Hewitt, E and Stromberg, K (1963). Real and abstract analysis. Springer-Verlag. Theorem 17.8.

Chapter 2

Linearization

For the linearization in concurrent computing, see Linearizability.

In mathematics linearization refers to nding the linear approximation to a function at a given point. In the study ofdynamical systems, linearization is a method for assessing the local stability of an equilibrium point of a system ofnonlinear dierential equations or discrete dynamical systems.[1] This method is used in elds such as engineering,physics, economics, and ecology.

2.1 Linearization of a functionLinearizations of a function are linesusually lines that can be used for purposes of calculation. Linearization is aneective method for approximating the output of a function y = f(x) at any x = a based on the value and slopeof the function at x = b , given that f(x) is dierentiable on [a; b] (or [b; a] ) and that a is close to b . In short,linearization approximates the output of a function near x = a .For example,

p4 = 2 . However, what would be a good approximation of

p4:001 =

p4 + :001 ?

For any given function y = f(x) , f(x) can be approximated if it is near a known dierentiable point. The mostbasic requisite is that La(a) = f(a) , where La(x) is the linearization of f(x) at x = a . The point-slope form ofan equation forms an equation of a line, given a point (H;K) and slope M . The general form of this equation is:y K = M(xH) .Using the point (a; f(a)) , La(x) becomes y = f(a) + M(x a) . Because dierentiable functions are locallylinear, the best slope to substitute in would be the slope of the line tangent to f(x) at x = a .While the concept of local linearity applies the most to points arbitrarily close to x = a , those relatively close workrelatively well for linear approximations. The slope M should be, most accurately, the slope of the tangent line atx = a .Visually, the accompanying diagram shows the tangent line of f(x) at x . At f(x+h) , where h is any small positiveor negative value, f(x+ h) is very nearly the value of the tangent line at the point (x+ h;L(x+ h)) .The nal equation for the linearization of a function at x = a is:y = f(a) + f 0(a)(x a)For x = a , f(a) = f(x) . The derivative of f(x) is f 0(x) , and the slope of f(x) at a is f 0(a) .

2.2 ExampleTo nd

p4:001 , we can use the fact that

p4 = 2 . The linearization of f(x) = px at x = a is y = pa+ 1

2pa(xa)

, because the function f 0(x) = 12pxdenes the slope of the function f(x) = px at x . Substituting in a = 4 , the

linearization at 4 is y = 2 + x44 . In this case x = 4:001 , sop4:001 is approximately 2 + 4:00144 = 2:00025 .

The true value is close to 2.00024998, so the linearization approximation has a relative error of less than 1 millionth

6

2.3. LINEARIZATION OF A MULTIVARIABLE FUNCTION 7

x

tangent line

slope= f'(x)

An approximation of f(x)=x^2 at (x, f(x))

of a percent.

2.3 Linearization of a multivariable functionThe equation for the linearization of a function f(x; y) at a point p(a; b) is:

f(x; y) f(a; b) + @f(x;y)@xa;b

(x a) + @f(x;y)@ya;b

(y b)

The general equation for the linearization of a multivariable function f(x) at a point p is:f(x) f(p) + rf jp (x p)where x is the vector of variables, and p is the linearization point of interest .[2]

2.4 Uses of linearizationLinearization makes it possible to use tools for studying nonlinear systems to analyze the behavior of a nonlinearfunction near a given point. The linearization of a function is the rst order term of its Taylor expansion around thepoint of interest. For a system dened by the equation

dxdt

= F(x; t)

the linearized system can be written as

dxdt

F(x0; t) +DF(x0; t) (x x0)

8 CHAPTER 2. LINEARIZATION

where x0 is the point of interest and DF(x0) is the Jacobian of F(x) evaluated at x0 .

2.4.1 Stability analysisIn stability analysis of autonomous systems, one can use the eigenvalues of the Jacobian matrix evaluated at ahyperbolic equilibrium point to determine the nature of that equilibrium. This is the content of linearization the-orem. For time-varying systems, the linearization requires additional justication.[3]

2.4.2 MicroeconomicsIn microeconomics, decision rules may be approximated under the state-space approach to linearization.[4] Under thisapproach, the Euler equations of the utility maximization problem are linearized around the stationary steady state.[4]A unique solution to the resulting system of dynamic equations then is found.[4]

2.5 See also Linear stability Tangent stiness matrix Stability derivatives Linearization theorem Taylor approximation Functional equation (L-function)

2.6 References[1] The linearization problem in complex dimension one dynamical systems at Scholarpedia

[2] Linearization. The Johns Hopkins University. Department of Electrical and Computer Engineering

[3] G.A. Leonov, N.V. Kuznetsov, Time-Varying Linearization and the Perron eects, International Journal of Bifurcationand Chaos, Vol. 17, No. 4, 2007, pp. 1079-1107

[4] Moatt, Mike. (2008) About.com State-Space Approach Economics Glossary; Terms Beginning with S. Accessed June 19,2008.

2.7 External links

2.7.1 Linearization tutorials Linearization for Model Analysis and Control Design

Chapter 3

System of linear equations

A linear system in three variables determines a collection of planes. The intersection point is the solution.

In mathematics, a system of linear equations (or linear system) is a collection of linear equations involving thesame set of variables.[1] For example,

9

10 CHAPTER 3. SYSTEM OF LINEAR EQUATIONS

3x + 2y z = 12x 2y + 4z = 2x + 12y z = 0is a system of three equations in the three variables x, y, z. A solution to a linear system is an assignment of numbersto the variables such that all the equations are simultaneously satised. A solution to the system above is given by

x = 1

y =2z =2since it makes all three equations valid. The word "system" indicates that the equations are to be considered collec-tively, rather than individually.In mathematics, the theory of linear systems is the basis and a fundamental part of linear algebra, a subject whichis used in most parts of modern mathematics. Computational algorithms for nding the solutions are an importantpart of numerical linear algebra, and play a prominent role in engineering, physics, chemistry, computer science, andeconomics. A system of non-linear equations can often be approximated by a linear system (see linearization), ahelpful technique when making a mathematical model or computer simulation of a relatively complex system.Very often, the coecients of the equations are real or complex numbers and the solutions are searched in the sameset of numbers, but the theory and the algorithms apply for coecients and solutions in any eld. For solutions inan integral domain like the ring of the integers, or in other algebraic structures, other theories have been developed,see Linear equation over a ring. Integer linear programming is a collection of methods for nding the best inte-ger solution (when there are many). Grbner basis theory provides algorithms when coecients and unknowns arepolynomials. Also tropical geometry is an example of linear algebra in a more exotic structure.

3.1 Elementary exampleThe simplest kind of linear system involves two equations and two variables:

2x + 3y = 6

4x + 9y = 15:

One method for solving such a system is as follows. First, solve the top equation for x in terms of y :

x = 3 32y:

Now substitute this expression for x into the bottom equation:

4

3 3

2y

+ 9y = 15:

This results in a single equation involving only the variable y . Solving gives y = 1 , and substituting this back intothe equation for x yields x = 3/2 . This method generalizes to systems with additional variables (see elimination ofvariables below, or the article on elementary algebra.)

3.2 General formA general system of m linear equations with n unknowns can be written as

3.3. SOLUTION SET 11

a11x1 + a12x2 + + a1nxn = b1a21x1 + a22x2 + + a2nxn = b2

... ... ... ...am1x1 + am2x2 + + amnxn = bm:Here x1; x2; : : : ; xn are the unknowns, a11; a12; : : : ; amn are the coecients of the system, and b1; b2; : : : ; bm arethe constant terms.Often the coecients and unknowns are real or complex numbers, but integers and rational numbers are also seen,as are polynomials and elements of an abstract algebraic structure.

3.2.1 Vector equationOne extremely helpful view is that each unknown is a weight for a column vector in a linear combination.

x1

26664a11a21...

am1

37775+ x226664a12a22...

am2

37775+ + xn26664a1na2n...

amn

37775 =26664b1b2...bm

37775This allows all the language and theory of vector spaces (or more generally, modules) to be brought to bear. Forexample, the collection of all possible linear combinations of the vectors on the left-hand side is called their span,and the equations have a solution just when the right-hand vector is within that span. If every vector within thatspan has exactly one expression as a linear combination of the given left-hand vectors, then any solution is unique.In any event, the span has a basis of linearly independent vectors that do guarantee exactly one expression; and thenumber of vectors in that basis (its dimension) cannot be larger than m or n, but it can be smaller. This is importantbecause if we have m independent vectors a solution is guaranteed regardless of the right-hand side, and otherwisenot guaranteed.

3.2.2 Matrix equationThe vector equation is equivalent to a matrix equation of the form

Ax = b

where A is an mn matrix, x is a column vector with n entries, and b is a column vector with m entries.

A =

26664a11 a12 a1na21 a22 a2n... ... . . . ...

am1 am2 amn

37775; x =26664x1x2...xn

37775; b =26664b1b2...bm

37775The number of vectors in a basis for the span is now expressed as the rank of the matrix.

3.3 Solution setA solution of a linear system is an assignment of values to the variables x1, x2, ..., xn such that each of the equationsis satised. The set of all possible solutions is called the solution set.A linear system may behave in any one of three possible ways:


(2,3)

x-y=-13x+

y=9

The solution set for the equations x y = 1 and 3x + y = 9 is the single point (2, 3).

1. The system has innitely many solutions.

2. The system has a single unique solution.

3. The system has no solution.

3.3.1 Geometric interpretation

For a system involving two variables (x and y), each linear equation determines a line on the xy-plane. Because asolution to a linear system must satisfy all of the equations, the solution set is the intersection of these lines, and ishence either a line, a single point, or the empty set.For three variables, each linear equation determines a plane in three-dimensional space, and the solution set is theintersection of these planes. Thus the solution set may be a plane, a line, a single point, or the empty set. For example,as three parallel planes do not have a common point, the solution set of their equations is empty; the solution set of theequations of three planes intersecting at a point is single point; if three planes pass through two points, their equationshave at least two common solutions; in fact the solution set is innite and consists in all the line passing through thesepoints.[2]

3.3. SOLUTION SET 13

For n variables, each linear equation determines a hyperplane in n-dimensional space. The solution set is the inter-section of these hyperplanes, which may be a at of any dimension.

3.3.2 General behavior

The solution set for two equations in three variables is usually a line.

In general, the behavior of a linear system is determined by the relationship between the number of equations and thenumber of unknowns:

Usually, a systemwith fewer equations than unknowns has innitely many solutions, but it may have no solution.Such a system is known as an underdetermined system.

Usually, a system with the same number of equations and unknowns has a single unique solution. Usually, a system with more equations than unknowns has no solution. Such a system is also known as anoverdetermined system.

In the rst case, the dimension of the solution set is usually equal to n m, where n is the number of variables and mis the number of equations.The following pictures illustrate this trichotomy in the case of two variables:


The rst system has innitely many solutions, namely all of the points on the blue line. The second system has asingle unique solution, namely the intersection of the two lines. The third system has no solutions, since the threelines share no common point.Keep in mind that the pictures above show only the most common case. It is possible for a system of two equationsand two unknowns to have no solution (if the two lines are parallel), or for a system of three equations and twounknowns to be solvable (if the three lines intersect at a single point). In general, a system of linear equations maybehave dierently from expected if the equations are linearly dependent, or if two or more of the equations areinconsistent.

3.4 Properties

3.4.1 Independence

The equations of a linear system are independent if none of the equations can be derived algebraically from the others.When the equations are independent, each equation contains new information about the variables, and removing anyof the equations increases the size of the solution set. For linear equations, logical independence is the same as linearindependence.For example, the equations

3x+ 2y = 6 and 6x+ 4y = 12

are not independent they are the same equation when scaled by a factor of two, and they would produce identicalgraphs. This is an example of equivalence in a system of linear equations.For a more complicated example, the equations

x 2y = 13x + 5y = 8

4x + 3y = 7

are not independent, because the third equation is the sum of the other two. Indeed, any one of these equations canbe derived from the other two, and any one of the equations can be removed without aecting the solution set. Thegraphs of these equations are three lines that intersect at a single point.

3.4.2 Consistency

See also: Consistent and inconsistent equationsA linear system is inconsistent if it has no solution, and otherwise it is said to be consistent . When the systemis inconsistent, it is possible to derive a contradiction from the equations, that may always be rewritten such as thestatement 0 = 1.For example, the equations

3x+ 2y = 6 and 3x+ 2y = 12

are inconsistent. In fact, by subtracting the rst equation from the second one and multiplying both sides of the resultby 1/6, we get 0 = 1. The graphs of these equations on the xy-plane are a pair of parallel lines.It is possible for three linear equations to be inconsistent, even though any two of them are consistent together. Forexample, the equations

3.4. PROPERTIES 15

The equations x 2y = 1, 3x + 5y = 8, and 4x + 3y = 7 are linearly dependent.

x + y = 1

2x + y = 1

3x + 2y = 3

are inconsistent. Adding the rst two equations together gives 3x + 2y = 2, which can be subtracted from the thirdequation to yield 0 = 1. Note that any two of these equations have a common solution. The same phenomenon canoccur for any number of equations.In general, inconsistencies occur if the left-hand sides of the equations in a system are linearly dependent, and theconstant terms do not satisfy the dependence relation. A system of equations whose left-hand sides are linearlyindependent is always consistent.Putting it another way, according to the RouchCapelli theorem, any system of equations (overdetermined or oth-erwise) is inconsistent if the rank of the augmented matrix is greater than the rank of the coecient matrix. If, onthe other hand, the ranks of these two matrices are equal, the system must have at least one solution. The solution isunique if and only if the rank equals the number of variables. Otherwise the general solution has k free parameterswhere k is the dierence between the number of variables and the rank; hence in such a case there are an innitudeof solutions. The rank of a system of equations can never be higher than [the number of variables] + 1, which means


The equations 3x + 2y = 6 and 3x + 2y = 12 are inconsistent.

that a system with any number of equations can always be reduced to a system that has a number of independentequations that is at most equal to [the number of variables] + 1.

3.4.3 Equivalence

Two linear systems using the same set of variables are equivalent if each of the equations in the second system canbe derived algebraically from the equations in the rst system, and vice versa. Two systems are equivalent if eitherboth are inconsistent or each equation of any of them is a linear combination of the equations of the other one. Itfollows that two linear systems are equivalent if and only if they have the same solution set.

3.5 Solving a linear system

There are several algorithms for solving a system of linear equations.

3.5. SOLVING A LINEAR SYSTEM 17

3.5.1 Describing the solutionWhen the solution set is nite, it is reduced to a single element. In this case, the unique solution is described by asequence of equations whose left-hand sides are the names of the unknowns and right-hand sides are the correspondingvalues, for example (x = 3; y = 2; z = 6) . When an order on the unknowns has been xed, for example thealphabetical order the solution may be described as a vector of values, like (3; 2; 6) for the previous example.It can be dicult to describe a set with innite solutions. Typically, some of the variables are designated as free (orindependent, or as parameters), meaning that they are allowed to take any value, while the remaining variables aredependent on the values of the free variables.For example, consider the following system:

x + 3y 2z = 53x + 5y + 6z = 7

The solution set to this system can be described by the following equations:

x = 7z 1 and y = 3z + 2.Here z is the free variable, while x and y are dependent on z. Any point in the solution set can be obtained by rstchoosing a value for z, and then computing the corresponding values for x and y.Each free variable gives the solution space one degree of freedom, the number of which is equal to the dimension ofthe solution set. For example, the solution set for the above equation is a line, since a point in the solution set canbe chosen by specifying the value of the parameter z. An innite solution of higher order may describe a plane, orhigher-dimensional set.Dierent choices for the free variables may lead to dierent descriptions of the same solution set. For example, thesolution to the above equations can alternatively be described as follows:

y = 37x+

11

7and z = 1

7x 1

7.

Here x is the free variable, and y and z are dependent.

3.5.2 Elimination of variablesThe simplest method for solving a system of linear equations is to repeatedly eliminate variables. This method canbe described as follows:

1. In the rst equation, solve for one of the variables in terms of the others.2. Substitute this expression into the remaining equations. This yields a system of equations with one fewer

equation and one fewer unknown.3. Continue until you have reduced the system to a single linear equation.4. Solve this equation, and then back-substitute until the entire solution is found.

For example, consider the following system:

x + 3y 2z = 53x + 5y + 6z = 7

2x + 4y + 3z = 8

Solving the rst equation for x gives x = 5 + 2z 3y, and plugging this into the second and third equation yields


4y + 12z = 82y + 7z = 2Solving the rst of these equations for y yields y = 2 + 3z, and plugging this into the second equation yields z = 2. Wenow have:

x = 5 + 2z 3yy = 2 + 3z

z = 2

Substituting z = 2 into the second equation gives y = 8, and substituting z = 2 and y = 8 into the rst equation yieldsx = 15. Therefore, the solution set is the single point (x, y, z) = (15, 8, 2).

3.5.3 Row reductionMain article: Gaussian elimination

In row reduction, the linear system is represented as an augmented matrix:

24 1 3 2 53 5 6 72 4 3 8

35 .This matrix is then modied using elementary row operations until it reaches reduced row echelon form. There arethree types of elementary row operations:

Type 1: Swap the positions of two rows.Type 2: Multiply a row by a nonzero scalar.Type 3: Add to one row a scalar multiple of another.

Because these operations are reversible, the augmented matrix produced always represents a linear system that isequivalent to the original.There are several specic algorithms to row-reduce an augmented matrix, the simplest of which are Gaussian elim-ination and Gauss-Jordan elimination. The following computation shows Gauss-Jordan elimination applied to thematrix above:

24 1 3 2 53 5 6 72 4 3 8

35 24 1 3 2 50 4 12 8

2 4 3 8

35 24 1 3 2 50 4 12 8

0 2 7 2

35 24 1 3 2 50 1 3 2

0 2 7 2

3524 1 3 2 50 1 3 2

0 0 1 2

35 24 1 3 2 50 1 0 8

0 0 1 2

35 24 1 3 0 90 1 0 8

0 0 1 2

35 24 1 0 0 150 1 0 8

0 0 1 2

35 :The last matrix is in reduced row echelon form, and represents the system x = 15, y = 8, z = 2. A comparison withthe example in the previous section on the algebraic elimination of variables shows that these two methods are in factthe same; the dierence lies in how the computations are written down.

3.5.4 Cramers ruleMain article: Cramers rule

3.5. SOLVING A LINEAR SYSTEM 19

Cramers rule is an explicit formula for the solution of a system of linear equations, with each variable given by aquotient of two determinants. For example, the solution to the system

x + 3y 2z = 53x + 5y + 6z = 7

2x + 4y + 3z = 8

is given by

x =

5 3 27 5 68 4 3

1 3 23 5 62 4 3

; y =

1 5 23 7 62 8 3

1 3 23 5 62 4 3

; z =

1 3 53 5 72 4 8

1 3 23 5 62 4 3

:

For each variable, the denominator is the determinant of the matrix of coecients, while the numerator is the deter-minant of a matrix in which one column has been replaced by the vector of constant terms.Though Cramers rule is important theoretically, it has little practical value for large matrices, since the computationof large determinants is somewhat cumbersome. (Indeed, large determinants are most easily computed using rowreduction.) Further, Cramers rule has very poor numerical properties, making it unsuitable for solving even smallsystems reliably, unless the operations are performed in rational arithmetic with unbounded precision.

3.5.5 Matrix solutionIf the equation system is expressed in the matrix formAx = b , the entire solution set can also be expressed in matrixform. If the matrix A is square (has m rows and n=m columns) and has full rank (all m rows are independent), thenthe system has a unique solution given by

x = A1b

where A1 is the inverse of A. More generally, regardless of whether m=n or not and regardless of the rank of A, allsolutions (if any exist) are given using the Moore-Penrose pseudoinverse of A, denoted Ag , as follows:

x = Agb+ (I AgA)w

where w is a vector of free parameters that ranges over all possible n1 vectors. A necessary and sucient conditionfor any solution(s) to exist is that the potential solution obtained using w = 0 satisfy Ax = b that is, thatAAgb = b: If this condition does not hold, the equation system is inconsistent and has no solution. If the conditionholds, the system is consistent and at least one solution exists. For example, in the above-mentioned case in which Ais square and of full rank, Ag simply equals A1 and the general solution equation simplies to x = A1b + (I A1A)w = A1b + (I I)w = A1b as previously stated, where w has completely dropped out of the solution,leaving only a single solution. In other cases, though, w remains and hence an innitude of potential values of thefree parameter vector w give an innitude of solutions of the equation.

3.5.6 Other methodsWhile systems of three or four equations can be readily solved by hand (see Cracovian), computers are often used forlarger systems. The standard algorithm for solving a system of linear equations is based on Gaussian elimination withsome modications. Firstly, it is essential to avoid division by small numbers, which may lead to inaccurate results.This can be done by reordering the equations if necessary, a process known as pivoting. Secondly, the algorithmdoes not exactly do Gaussian elimination, but it computes the LU decomposition of the matrix A. This is mostly an


organizational tool, but it is much quicker if one has to solve several systems with the same matrix A but dierentvectors b.If the matrix A has some special structure, this can be exploited to obtain faster or more accurate algorithms. Forinstance, systems with a symmetric positive denite matrix can be solved twice as fast with the Cholesky decompo-sition. Levinson recursion is a fast method for Toeplitz matrices. Special methods exist also for matrices with manyzero elements (so-called sparse matrices), which appear often in applications.A completely dierent approach is often taken for very large systems, which would otherwise take too much time ormemory. The idea is to start with an initial approximation to the solution (which does not have to be accurate at all),and to change this approximation in several steps to bring it closer to the true solution. Once the approximation issuciently accurate, this is taken to be the solution to the system. This leads to the class of iterative methods.

3.6 Homogeneous systemsSee also: Homogeneous dierential equation

A system of linear equations is homogeneous if all of the constant terms are zero:

a11x1 + a12x2 + + a1nxn = 0a21x1 + a22x2 + + a2nxn = 0

... ... ... ...am1x1 + am2x2 + + amnxn = 0:A homogeneous system is equivalent to a matrix equation of the form

Ax = 0where A is an m n matrix, x is a column vector with n entries, and 0 is the zero vector with m entries.

3.6.1 Solution setEvery homogeneous system has at least one solution, known as the zero solution (or trivial solution), which isobtained by assigning the value of zero to each of the variables. If the system has a non-singular matrix (det(A) 0)then it is also the only solution. If the system has a singular matrix then there is a solution set with an innite numberof solutions. This solution set has the following additional properties:

1. If u and v are two vectors representing solutions to a homogeneous system, then the vector sum u + v is also asolution to the system.

2. If u is a vector representing a solution to a homogeneous system, and r is any scalar, then ru is also a solutionto the system.

These are exactly the properties required for the solution set to be a linear subspace of Rn. In particular, the solutionset to a homogeneous system is the same as the null space of the corresponding matrix A. A numerical solutions to ahomogeneous system can be found with a SVD decomposition.

3.6.2 Relation to nonhomogeneous systemsThere is a close relationship between the solutions to a linear system and the solutions to the corresponding homoge-neous system:

Ax = b and Ax = 0.

3.7. SEE ALSO 21

Specically, if p is any specic solution to the linear system Ax = b, then the entire solution set can be described as

fp+ v : v to solution any is Ax = 0g :Geometrically, this says that the solution set for Ax = b is a translation of the solution set for Ax = 0. Specically, theat for the rst system can be obtained by translating the linear subspace for the homogeneous system by the vectorp.This reasoning only applies if the system Ax = b has at least one solution. This occurs if and only if the vector b liesin the image of the linear transformation A.

3.7 See also Linear equation over a ring Arrangement of hyperplanes Iterative renement LAPACK (the free standard package to solve linear equations numerically; available in Fortran, C, C++) Linear least squares Matrix decomposition Matrix splitting NAG Numerical Library (NAG Library versions of LAPACK solvers) Row reduction Simultaneous equations

3.8 Notes[1] The subject of this article is basic in mathematics, and is treated in a lot of textbooks. Among them, Lay 2005, Meyer

2001, and Strang 2005 contain the material of this article.[2] Charles G. Cullen (1990). Matrices and Linear Transformations. MA: Dover. p. 3. ISBN 978-0-486-66328-9.

3.9 ReferencesSee also: Linear algebra Further reading

3.9.1 Textbooks Axler, Sheldon Jay (1997), Linear Algebra Done Right (2nd ed.), Springer-Verlag, ISBN 0-387-98259-0 Lay, David C. (August 22, 2005), Linear Algebra and Its Applications (3rd ed.), Addison Wesley, ISBN 978-0-321-28713-7

Meyer, Carl D. (February 15, 2001), Matrix Analysis and Applied Linear Algebra, Society for Industrial andApplied Mathematics (SIAM), ISBN 978-0-89871-454-8

Poole, David (2006), Linear Algebra: A Modern Introduction (2nd ed.), Brooks/Cole, ISBN 0-534-99845-3 Anton, Howard (2005), Elementary Linear Algebra (Applications Version) (9th ed.), Wiley International Leon, Steven J. (2006), Linear Algebra With Applications (7th ed.), Pearson Prentice Hall Strang, Gilbert (2005), Linear Algebra and Its Applications


3.10 Text and image sources, contributors, and licenses3.10.1 Text

Dierentiable function Source: https://en.wikipedia.org/wiki/Differentiable_function?oldid=670846696 Contributors: Toby Bartels,Tosha, Mormegil, Bender235, Mpatel, Koavf, InverseHypercube, Khazar, Jim.belk, JAnDbot, Magneez, Anonymous Dissident, SieBot,StAnselm, Amahoney, Addbot, Cesiumfrog, ., Jarble, Ptbotgourou, AnomieBOT, ^musaz, Rubinbot, Shadowjams, Kusluj, Red-Bot, Toolnut, Xnn, EmausBot, Slawekb, JSquish, D.Lazard, Zueignung, 1l2, Helpful Pixie Bot, Frank.manus, Acbdyho, Brad7777,Stephan Kulla, JustAMuggle, Razibot, Loraof, Mpezzini, Nbro and Anonymous: 13

Linearization Source: https://en.wikipedia.org/wiki/Linearization?oldid=674796386 Contributors: Charles Matthews, Giftlite, Ben-FrantzDale, Jmeppley, CALR, Keenan Pepper, Oleg Alexandrov, Dionyziz, Eubot, Mathbot, Grubber, Ninly, Oneirist, Adam majewski,Gracenotes, Siva1979, Jim.belk, Novangelis, Comech, Hkiahsebag, Thijs!bot, Headbomb, Bobblehead, Italo Tasso, Laser.Y, JAnDbot,Pbroks13, AstroHurricane001, Iain marcuson, Coppertwig, AlnoktaBOT, Geometry guy, SieBot, Leviel, PixelBot, J.Gowers, Addbot,Kman543210, LaaknorBot, Bebestbe, Luckas-bot, Yobot, Unara, Xqbot, Amaury, FrescoBot, Jgalleg5, RedBot, Aturevsk, Bobogoobo,Helpful Pixie Bot, BG19bot, Wilsongerald, Saung Tadashi, Fragapanagos, Ginsuloft and Anonymous: 50

System of linear equations Source: https://en.wikipedia.org/wiki/System_of_linear_equations?oldid=664822063 Contributors: Axel-Boldt, Tarquin, Wshun, Kku, Andres, Charles Matthews, Dysprosia, Jitse Niesen, Hao2lian, Robbot, DHN, Giftlite, Everyking, DanielBrockman, Quadell, Torokun, Sam Hocevar, Azuredu, Alikhtarov, Discospinster, Paul August, Nk, Obradovic Goran, Ranveig, Jumbuck,Arthena, Caesura, Nvrmnd, Joriki, Woohookitty, LOL, Rchrd, ^demon, Je3000, Zzyzx11, Marudubshinki, Dgwarwick, Grammarbot,Hgkamath, Salix alba, FlaBot, Mathbot, Crazycomputers, Chobot, YurikBot, Wavelength, Michael Slone, KSmrq, Giro720, Zingus,Bota47, Wknight94, Arthur Rubin, KGasso, Willtron, Gesslein, InverseHypercube, Hydrogen Iodide, Larry Doolittle, Zserghei, KYN,Mhss, Anwar saadat, Chris the speller, Bluebot, Pieter Kuiper, DHN-bot~enwiki, RProgrammer, Ott0, Vegard, Cybercobra, Trifon Tri-antallidis, Blake-, Zchenyu, Lambiam, Jim.belk, RomanSpa, Mets501, Spiel496, Tawkerbot2, Dgw, Thijs!bot, Braveorca, Kablammo,Wmasterj, Urdutext, Escarbot, AntiVandalBot, Danroa, JPG-GR, Day and Nite, Khalid Mahmood, Noyder, Erkan Yilmaz, J.delanoy,Mathemaduenn~enwiki, K.menin, AntiSpamBot, SJP, Fylwind, Lights, Caribbean H.Q., VolkovBot, AlnoktaBOT, TXiKiBoT, Mar-tinOtter, Driski555, Seminalist, Brainfsck, SieBot, Ivan tambuk, Tiddly Tom, Dhateld, Yerpo, Svick, Amahoney, Denisarona, Clue-Bot, GorillaWarfare, Dobermanji, Plastikspork, Wpoely86, Wikijens, Richard B. Frost, Tim32, Alexbot, Leonard^Bloom, Kaspar.jan,Newyorxico, Thingg, XLinkBot, SilvonenBot, JinJian, Addbot, SahilK7654, Steve.jaramillov~enwiki, Fgnievinski, IgorCarron, II Mus-LiM HyBRiD II, Calle, CinchBug, KamikazeBot, AnomieBOT, Jim1138, Vanakaris, Twri, ArthurBot, Xqbot, Matttoothman, ToLLIa,Tyrol5, Frankie0607, Universalss, Constructive editor, Tavernsenses, BenzolBot, Jauhienij, Duoduoduo, , EmausBot, John ofReading, Inframaut, DotKuro, Tommy2010, Slawekb, D.Lazard, OnePt618, Donner60, ChuispastonBot, Anita5192, ClueBot NG, Oozdot ie, Movses-bot, 3rdiw, Helpful Pixie Bot, Guy vandegrift, Hz.tiang, Ren Vpenk, BattyBot, Pratyush Sarkar, Stigmatella aurantiaca,Boesball, Mark L MacDonald, Webclient101, Frosty, Graphium, Makalrfekt, Pdecalculus, Lakun.patra, Loraof and Anonymous: 187

3.10.2 Images File:Absolute_value.svg Source: https://upload.wikimedia.org/wikipedia/commons/6/6b/Absolute_value.svg License: CC-BY-SA-3.0

Contributors: Vectorised version of Image:Absolute_value.png Original artist: This hand-written SVG version by Qef File:Approximation_of_cos_with_linear_functions_without_numbers.svg Source: https://upload.wikimedia.org/wikipedia/commons/

f/fa/Approximation_of_cos_with_linear_functions_without_numbers.svg License: CC BY 3.0 Contributors: Own work Original artist:Stephan Kulla (User:Stephan Kulla)

File:Cusp.svg Source: https://upload.wikimedia.org/wikipedia/commons/a/a3/Cusp.svg License: CC BY-SA 3.0 Contributors: Ownwork Original artist: Georg-Johann

File:Intersecting_Lines.svg Source: https://upload.wikimedia.org/wikipedia/commons/c/c0/Intersecting_Lines.svg License: Public do-main Contributors: Own work Original artist: Jim.belk

File:Intersecting_Planes_2.svg Source: https://upload.wikimedia.org/wikipedia/commons/d/d6/Intersecting_Planes_2.svgLicense: CCBY-SA4.0Contributors: Original artist: Fred the Oyster

File:One_Line.svg Source: https://upload.wikimedia.org/wikipedia/commons/f/fb/One_Line.svg License: Public domain Contributors:Own work Original artist: Jim.belk

File:Parallel_Lines.svg Source: https://upload.wikimedia.org/wikipedia/commons/1/14/Parallel_Lines.svgLicense: Public domainCon-tributors: Own work Original artist: Jim.belk

File:Polynomialdeg3.svg Source: https://upload.wikimedia.org/wikipedia/commons/a/a3/Polynomialdeg3.svg License: Public domainContributors: Polynomialdeg3.png Original artist: N.Mori

File:Secretsharing_3-point.svg Source: https://upload.wikimedia.org/wikipedia/commons/a/ab/Secretsharing_3-point.svgLicense: CCBY-SA4.0Contributors: Original artist: Fred the Oyster

File:Tangent-calculus.svg Source: https://upload.wikimedia.org/wikipedia/commons/d/d2/Tangent-calculus.svg License: CC-BY-SA-3.0 Contributors:

3.10. TEXT AND IMAGE SOURCES, CONTRIBUTORS, AND LICENSES 23

Tangent-calculus.png Original artist: derivative work: Pbroks13 (talk) File:Three_Intersecting_Lines.svg Source: https://upload.wikimedia.org/wikipedia/commons/0/0f/Three_Intersecting_Lines.svg Li-

cense: Public domain Contributors: Own work Original artist: Jim.belk File:Three_Lines.svg Source: https://upload.wikimedia.org/wikipedia/commons/d/df/Three_Lines.svg License: Public domainContrib-

utors: Own work Original artist: Jim.belk File:Two_Lines.svg Source: https://upload.wikimedia.org/wikipedia/commons/1/19/Two_Lines.svg License: Public domain Contribu-

tors: Own work Original artist: Jim.belk

3.10.3 Content license Creative Commons Attribution-Share Alike 3.0

Differentiable functionDifferentiability and continuityDifferentiability classesDifferentiability in higher dimensionsDifferentiability in complex analysisDifferentiable functions on manifoldsSee alsoReferences

LinearizationLinearization of a functionExampleLinearization of a multivariable functionUses of linearizationStability analysisMicroeconomics

See alsoReferencesExternal linksLinearization tutorials

System of linear equationsElementary exampleGeneral formVector equationMatrix equation

Solution setGeometric interpretationGeneral behavior

PropertiesIndependenceConsistencyEquivalence

Solving a linear systemDescribing the solutionElimination of variablesRow reductionCramers ruleMatrix solutionOther methods

Homogeneous systemsSolution setRelation to nonhomogeneous systems

See alsoNotesReferencesTextbooks

Text and image sources, contributors, and licensesTextImagesContent license

linear ization

Documents