a1 chapter 1 mat530 as latest 5 may 2012

Chapter 1 MATHEMATICAL MODELING MAT 530

CHAPTER ONE

Mathematical

Modeling Process

At the end of this module, students should be able:

To define mathematical modeling

To explain what is a mathematical model

To describe various classification of mathematical models

To outline procedures involved in constructing a mathematical model

To solve a simple modeling problem

To apply the method of least squares

1.1 Introduction

Consider the problems of finding the shortest route from one location to a

destination in a busy city or deciding locations for bus stops in an urban

area so that greater number of passengers can be achieved. These are

some examples of complex real-world problems. In order to obtain the

required solution this person need to investigate necessary questions

about the observed world and give a simplified description of the

problem. He/she can setup equations using mathematical concepts and

language to test some ideas and then solved the model to make

prediction or decision. The developed process mentioned above is

known as mathematical modeling.

Definition 1.1

Mathematical modeling is the process of constructing

mathematical objects as a quantitative method to represent the

properties or behaviors of the real-world system, process or

phenomena. Mathematical object refers to sets of numbers,

variables or parameters and their symbols and functional

relationships in the form of equation or system of equations and

geometrical or algebraic structure.


Mathematical modeling cultivates the interaction between mathematics

and the real world by translating the real world problem into mathematical

formulation or model, solving the model and interpreting the solutions

obtained using the model so as to make the model useful to the real

world. The real world problem covers situations which may come from

diverse disciplines such as engineering, science, architecture, business,

sports science and computer science while the system can be a physical

system, a biological system, a financial system, a social system or an

ecological system. Mathematical modeling allows a modeler to

undertake experiments on mathematical representations of the real world

instead of undertaking experiments in the real world. Mathematical

models can take many forms such as dynamical systems, statistical

models, differential equations, or game theoretic models. Mathematical

modeling can be viewed as a process, as illustrated in the schematic

diagram of Figure 1.1.

Figure 1.1: Process of Mathematical Modeling

Figure 1.1 shows mathematical modeling as an iterative process or a

cycle (or loop) which starts and finishes with the real world problem. It

may start with the the upper left-handcorner, the real world problem. This

represents quantitative measurement or data of system of interest and

knowledge of how the system works such that properties, behaviors and

Given {some factors}, find {outputs}, such that {objective is

achieved}.

Test /

Analyze

Translate (Formulate)

Solve /

Analyze

Real-World Conclusion

or Prediction

Real-World

Problem

Mathematical

Solutions

Translate

(Interpret)

Data Collection and Analysis Making Assumptions

Construct Model

FINISH

START

Simulation ‘What-If’ Analysis

Validation and

Verification

Mathematical Model

http://www.answers.com/topic/dynamical-system

http://www.answers.com/topic/statistical-model

http://www.answers.com/topic/statistical-model

http://www.answers.com/topic/differential-equation

http://www.answers.com/topic/game-theory


relationship between factors (variables and parameters) could be

observed and problem pertaining to the system that needs to be modeled

identified. A mathematical models is characterized by assumptions about

variables (the things which change), parameters (the things which do not

change) and functional forms (the relationship between the two). Thus,

based on data and knowledge of the system gathered, a mathematical

model can be constructed.

Construction of a model requires: 1) a clear objective of the modeling –

which aspect of the system is involved, what do you wish the model to

produce and how accurate you want the model be; and 2) a precise idea

of the main factors involved and therefore, it may include taking a

simplified view of the system and considering the simplest practical

approach by neglecting certain factors and making assumptions. Real-

world problem could be simplified or transformed to a problem relatively

close to the original problem but all essential features of the problem

must be retained.

Mathematical modeling may encompass the following goals:

a) integrating various existing good models of various parts of the

system to represent the whole system of interest;

b) generalizing existing models of different systems through adoption,

adaptation or modification of these models to be applied to the

system of interest; and

c) simplifying or making approximation to general model since the

current models are difficult to compute or analyzed.

After a mathematical model is established, computational experiments or

computations are required to solve the model which is to generate results

or solutions. Based on the model developed, optimum or approximate

solutions maybe obtained. As the model is aimed to represent the real

system, results or solutions found based on the model should be able to

be interpreted as descriptive properties or behaviors concerning the

system, thus, enabling prediction or conclusion to be made about the real

system. The solutions, however, must first be tested (validated and

verified) against real data to determine the effectiveness of the model (to


test assumptions and for sensitivity analysis). If the solutions agree with

the real system (i.e., makes sense), then the model does indeed capture

correctly important aspects of the real-world situation and thus can be

accepted and used. In the case where the solutions found do not

substantiate the real-world phenomena, the modeling cycle is followed

through again with a revised formulation of model, new calculated results,

improved predictions, and so on. It is useful to note that the modeling

process may not necessarily results in solving the problem entirely but it

will furnish useful information regarding the situation under investigation.

The whole process of mathematical modeling can be divided into three

main activities: i) construct the model, ii) compute and analyze results,

and iii) test (validate and verify) results, as depicted in Figure 1.2 below.

Figure 1.2: Mathematical Modeling Process

Definition 1.2

A mathematical model is a system (S) containing a set of

questions (Q) relating to the system and a set of mathematical

statements (M) which can be used to answer the questions

(Velten, 2009).

A system refers to a set of interacting, interrelated, or interdependent

elements (or components) which are functionally related. When modeling,

very specific sets of questions must be addressed in order to transform

Test the Results

Construct the Model

Compute and Analyze Results

Apply the Model


data concerning the system into some mathematical statements. These

questions, which transpire the purpose of the model, are very focused and

challenging questions and can only be answered with a conclusion based

on the analysis and interpretation of evidence. In other words, the

questions invoke logical statement that progresses from what is known or

believed to be true to that which is unknown and requires validation.

Mathematical statements refer to relationships of numbers, variables and

parameters formulated using notations, symbols and operators. These

statements are used to solve the problem and provide meaningful

information useful for making conclusion, decision or prediction regarding

the system.

1.2 Definition of a model

A model is a representation of reality containing the essential structure of

some object or event in the real world. It can be a physical, symbolic

(mathematical), graphical, verbal or simplified version of a real world

system or phenomenon. Since most systems and phenomenon are very

complicated (due to their large number of components) and complex

(very highly interconnected) to be understood entirely, a model may

contain only those features that are of primary importance to the

modeler’s purpose. Thus, a model is usually a tradeoff between

generality, realism and precision. Models, therefore, may be necessarily

incomplete and may be changed or manipulated with relative ease. Due

to these, there is no model can be considered as the best model, only

better model.

Models can be categorized into three classes on the basis of their degree

of abstraction, as the following:

a) Physical (Iconic) model: An iconic, 'look-alike' model, which is the

least abstract. Examples include a model of an airplane or an

architect’s model of a building.

b) Analogous model: A more abstract type of model in which the

model behaves like a real system but only having some resemblance

http://www.businessdictionary.com/definition/feature.html


to what it represents. Chart (including flowchart), graph, map and

network diagram are considered as analogous models.

c) Symbolic or mathematical model: It is the most-abstract model

with no resemblance but only an approximation to what it represents.

The relationships between variables of the system are expressed

using a set of rules or operators according to their meaning (syntax)

rather that what they represent (semantic). Examples of these

models include mathematical equations or formula, financial

statement and set of accounts.

Definition 1.3

A model is called dynamic if at least one of its system parameters or

state variables depends on time.

Definition 1.4

A model is called static if the system parameters or state variables

do not change with respect to time.

A model can be a static or dynamic model. Figure 1.2, 1.3 and 1.4

illustrates some examples of the physical, analogous and symbolic model

in both static and dynamic forms.

a) A Static Physical Model

- A miniature replicate of a submarine b) A Dynamic Physical Model - Aerodynamics of a car in a wind tunnel

Figure 1.2: Physical Model

http://www.businessdictionary.com/definition/chart.html

http://www.businessdictionary.com/definition/network.html

http://www.businessdictionary.com/definition/diagram.html


a) An Static Analogous Model

- A model of an airport city c) A Dynamic Analogous Model

- A Traffic Anomalous Events Model

Figure 1.3: Analogous Model

a) A Static Symbolic Model - A geometry model for right

triangles

b) A Dynamic Symbolic Model - Population Growth Models

Figure 1.4: Symbolic Model

1.3 Why Model?

To gain understanding.

A model represents characteristics and behaviors of a real-world system

concerned, thus, providing an improved understanding. It facilitates

understanding by focusing and analyzing necessary components of the

system. Through modeling we can comprehend how different parts of

the system are related and which factors are most important in the

system.

To predict or simulate. Very often we wish to know what a real-world

system will do in the future, but it is expensive, impractical, or impossible


to experiment directly with the system. Examples include nuclear reactor

design, space ight, extinction of species, weather prediction, drug efficacy

in humans, and so on. Using models, simulation of the real-world system

or phenomenon can be carried out, thus providing means to explain,

control, and predict events on the basis of controlled observations.

To aid in decision making. Testing using the model developed allows

the possibility of simulating the ‘what if scenario’ or performing the

sensitivity analysis. Hence, the model provides concrete justifications for

any decison made.

Mathematics has been describe in old adage as the “Queen of Science”

and “language of science". It been used to describe real phenomena

through the mathematical modeling process. Thus, it is not a surprise

that mathematical modeling has emerged to be a much needed tool in

this modern science and technology era. Cheaper and more advanced

computers enable mathematical models to play vital role in discovering

new knowledge and solving problems aside from becoming increasingly

cost-effective alternative to direct experimentation.

Despite the clear advantages of modeling, there is no definite steps for

constructing a “one for all” model. There is no model that can be applied

to all situations. Modeling is an art. It involves the creativity in utilizing a

sound mathematical knowledge together with the knowledge of the

system to create something useful to provide meaningful information and

interpretation of the system. Different expertise and perspectives yields

different models for the same system. Such scenario justifies that there

is no “best” or “perfect” model. A model is generally a trade-offs between

the following:

a) accuracy

b) flexibility

c) cost.

More accurate model requires higher cost and reduces flexibility. Higher

flexibility model, on the other hand, may jeopardize the accuracy of the

model and may comes at increased cost. In addition, expensive model is


not preferrable since it does not necessarily guarantees accuracy or

flexibility. Thus, modeling is always aimed at producing a sufficiently

accurate and flexible model preferrably at low cost.

Categorize the following models as one of the following: a) A static physical model d) An dynamic analogous model b) A dynamic physical model e) A static symbolic model c) An static analogous model f) A dynamic symbolic model

Universal Harmonics Model Malignancies Dynamic Pathway Model

Answer: _______________________

Answer: _______________________

A Hurricane Forecast Model A Volume Optimization Model

Answer: _______________________ Answer: _______________________

Warm up exercise


1.4 Classification of Mathematical Models

When studying or developing models, it is helpful to identify the type of

models that we are dealing with. Mathematical models can be classified

into four different types: a) deterministic model, b) stochastic models, c)

empirical models and d) mechanistic models

Definition 1.5

A deterministic model is a model which is represented by a

function that allows predictions of the dependent variable to be

made based on the independent variable(s).

Deterministic models have no components that are inherently uncertain,

i.e., no parameters in the model are characterized by probability

distributions. Deterministic models ignore random variation, and so

always predict the same outcome from a given starting point. In other

words, for fixed starting values, a deterministic model will always produce

the same result. Deterministic model is usually applied for situations

where the outputs are direct consequence of the initial conditions of the

problem.

Definition 1.6

A stochastic model is a model where randomness is present

and variables are not described by unique values, but rather by

probability distributions.

A stochastic model is mainly used for situations where a random effect

plays a vital role in the model formulation and solving. This model, which

is statistical in nature, will produce many different results depending on

the actual values that the random variables take in each realization.

A stochastic model predicts the distribution of possible outcomes.


Definition 1.7

A model is known as empirical model if it was constructed from

and based entirely on experimental data only, using no priori

information about the system.

For an empirical model, no account is taken of the mechanism by which

changes to the system occur and minimal information is used a priori in

the development of the model. The model just simply notes that some

phenomena occur and tries to account quantitatively for changes

associated with different conditions. Thus, an empirical model usually

provides a quantitative summary of these observed relationships among

a set of measured variables. Models developed using statistical

techniques are examples of empirical models.

In an empirical model, relationship between variables is determined by

inspecting the data on the variables and selecting the best fit

mathematical formula to represent the relationship. Therefore, it is a

compromise between accuracy of fit and simplicity of mathematics.

Definition 1.8

A model is known as mechanistic model if some of the

mathematical statements are based on priori information about

the system.

A mechanistic model uses a large amount of theoretical information by

attempting to explain phenomena that occur at a more detailed level of

structure. It generally describes what happens at one level in the

hierarchy by considering processes at lower levels. It takes account of

the mechanisms through which these changes occur and generates

prediction concerning the variables based on this knowledge. The model

is built on causal relationships represented by equations.

Table 1.1 shows an example of classification of some models according

to the four classes of models.


Table 1.1: Classification of Models

Empirical Mechanistic

De

term

inis

tic

Short term forecasts (nowcasts) of seasonal climate based on historical climate records (climatological database) of rainfall or temperature using regression analysis

A hypothetical study on predator-prey relationships of the larval salamanders and predacious aquatic invertebrates – a deterministic model with mechanistic explanantion to patterns observed

Sto

ch

as

tic

A mass flux balance model to estimate heavy-metals in agriculture soil. Normal or log normal probability distribution is used to represent the metal concentration.

A risk projection model in epidemiology (in the form of a stochastic mechanistic model) developed to estimate radiation-induced cancer risks to the end of life for Japanese atomic bomb survivors.

A model can also be classified according to the level of understanding on

which the model is based. The simplest explanation is to consider the

hierarchy of organizational structures within the system being modeled.

For example, in the case of animals, models can be distinguished

according to different levels of this hierarchical structure. For example, a

model concerning an animal can be classified as a molecular model,

cellular model or an individual model of the animal. This classification

can be illustrated as in Figure 1.6.

High Herd

Individual

Organs

Cells

Low Molecules

Figure 1.6: A Classification of Model for Animals based on Hierarchy

Aside from classifying models based on the categories mentioned,

models can be distinguished based on whether they are static or dynamic

models. As described in Definition 1.3 and Definition 1.4, a static model


does not account for the element of time whereas a dynamic model is

time dependent. The static model describes the steady-state or

equilibrium situations of the system at a specific time instant, where the

output does not change if the input is the same. In a dynamic model, the

output changes with respect to time. A resistor system is a static model

where the voltage is directly proportional to the current, independent of

time. On the other hand, a capacitor is modeled as dynamic model since

the voltage is dependent on the time and previous time history.

Dynamic models can be divided further into continuous and discrete

models, as described in the following definitions.

Definition 1.9

A model is called discrete if the quantities have varied at discrete

times or places (or that we only consider discrete variations even

if they may change continuously).

Definition 1.10

A model is called continuous if there is continuous variation in

quantities.

Discrete models use a discrete or combinatoric description such as

integer numbers, graphs and difference equations. On the other hand,

continuous models often involve real quantities, i.e., real numbers,

physical quantities and differential equations rather than the difference

equations. Differential equations denotes changes in time over an

infinitesmall interval whereas difference equations represent changes

over a finite interval (time jumps).

One further type of model, the system model, is worthy of mention. This

is built from a series of sub-models, each of which describes the essence

of some interacting components. The above method of classification then

refers more properly to the sub-models: different types of sub-models

may be used in any one system model.


Much of the modelling literature refers to ’simulation models’. Why are

they not included in the classification? The reason for this apparent

omission is that ’simulation’ refers to the way the model calculations are

done - i.e. by computer simulation. The actual model of the system is not

changed by the way in which the necessary mathematics is performed,

although our interpretation of the model may depend on the numerical

accuracy of any approximations.

1.5 Constructing Mathematical Models

Mathematical modeling is the art of transforming real world problem from a

specific application area into a structured mathematical formulation which is

solvable, thus the theoretical and numerical analysis can lead to useful

understanding, prediction, solution and interpretation related to the original

problem. Mathematical modeling has been successfully applied in wide areas of

applications, indispensable in some of these applications. The model developed

provides precise guide to solution of the problem, enables comprehensive

understanding of the system involved and allows the application of advanced

computing software and techniques. Mastering the skill in mathematical

modeling should be one of the expertise that must be acquired by students,

aside from the theoretical mathematical learning in class, to enable them to

handle real world challenging problem of our time.

Figure 1.t illustrates the flow of interaction of the four main components of the

mathematical modeling process. Problem statement defines the problem and

wishes to are to be achieved by the by the model. The mathematical theory

includes theory related to the application, mathematical theory to be employed

and theories discussed in literature. Mathematical solution methods underline

the mathematical approaches which could be utilized in generating the solutions

based on the mathematical model. Finally, computational experiments refer to

the execution of computer programs or simulation runs using certain data and

according to the mathematical model and solution techniques constructed.


Figure 1.7: Interaction of the Components of the Mathematical Modeling Process

The modeling process itself is usually an iterative process characterized by a

sequence of steps which sometimes required to be repeated. The steps include

the following:

a) define the problem and objectives of the model;

b) conduct a thorough literature review of the problem and existing models;

c) gather complete data through data collection, empirical observations or

experiments;

d) analyze data to identify the important factors (main variables, parameters

and constants) and determine properties and relationships between

these factors as well as their influence towards the behavior of the

system;

e) construct diagram, flowchart, network or computer visualization model to

get better understanding of the complex interaction of factors within the

system;

f) list all assumptions and ways to verify and validate the model;

g) transform the problem using abstraction or formulization into a

mathematical model. Start with a simple model as among models with

similar predictive power, the simplest one is the most desirable;

Mathematical Model

Problem Statement

Mathematical Theory

Mathematical Solution Methods

Computational Experiments


h) develop solution techniques that can be used to determine the solutions

of the model;

i) perform computational experiments,which can consist of simulations,

analytical and qualitative analysis using mathematical techniques;

j) analyze the results;

k) validate and verify results based on interpretation and comparison with

the real system using data set that have not been used to build the

model;

l) make conclusions, predictions and decisions based on j) and k);

m) carry out the changes in the real system based on the findings and

mechanisms proposed by the model; and

n) refine the model, if necessary, by repeating step a) to h) for enhanced

performance of the real system.

Steps : Constructing Mathematical Model

1. Identify the real problem

2. Formulate a mathematical model

i. Identify and classify the variables

ii. Determine interrelationships between the variables

4. Obtain the mathematical solution to the model

5. Interpret the mathematical solution

6. Verify the model

i. Does it address the problem?

ii. Does it make common sense?

iii. Test or compare solution with real world data

7. Write a report/Conclusion

8. Implement the model

9. Maintain the model


Modeling Approaches Construction of a mathematical model includes the following steps:

1. Identify the real problem

a. This concern with questions such as the following:

◦ How to classify the problem?

◦ Is there an underlying physical/scientific behavior to be taken

account?

◦ What kind of sources of facts and data are relevant?

◦ Is data available? Is itr primary or secondary data?

◦ What assumptions and simplifications about the problem can be

made?

◦ What kinds of models are applicable?

◦ What is the purpose and objective of the model to be developed?

◦ Will the solution based on the model be unique or not?

◦ Who will use the model?

b. Formulate a precise problem statement

In most problems we can develop the problem statement by examining:

◦ What is known or given?

◦ What is to be found, estimated or decided

◦ What are the conditions to be satisfied?

◦ What are the objectives to be achieved?

2. Formulate mathematical model

Various activities need to be carried as listed below:

a. Identify and list the relevant factors and/or relationships.

Factors are quantifiable and can be classified as variables, parameters

or constants, and each of these can be continuous, discrete or random.

◦ Continuous variables: takes all real values over an interval, e.g.

time, speed, length, density, etc.

◦ Discrete variables: takes on certain isolated values only, e.g. the

number of people, cars, houses, months, etc.


◦ Random variables: unpredictable, but governed by some

underlying statistical model, e.g. bus’s arrival time.

◦ Input Variables: quantities which determine subsequent

evaluations within the model. Input variables are to be known, or

give, or assumed, or can be considered to have an arbitrary value.

◦ Output Variables: quantities whish are consequences of given

values of input variables and parameters and cannot be given

arbitrary values. These represent outcomes from a model.

◦ Parameters: Quantities which are constant for a particular

application of model, but can have different values for another

application of the same model, e.g. fixed costs, the dimension of a

room, price of a ticket, etc.

◦ Constants: quantities whose values we cannot change, e.g. π,

gravity, speed of light, etc.

All the factors must have suitable algebraic symbols and represent

measurements with certain unit of measurement.

b. List the assumptions

Basic considerations for assumptions include:

i. whether or not to include certain factors,

ii. the relative magnitudes of certain effects of various factors (help

to identify the main factors), and

iii. the forms of relationship between factors – this is the heart of the

model.

Appropriate assumptions keep the model simple.

c. Collect data

Utilize data as much as possible.

i. At initial stage – analyze data to identify factors and their

relationships.

ii. In the model development stage – fix values of parameters and

constants that may occur in the model based on data.

iii. In the final stage – valicate model by comparing its solution or

prediction with real data


Challenges in data collection:

◦ Getting permission to collect data.

◦ Getting right instruments / equipments.

◦ Getting respondents to complete survey questionnaire.

◦ Getting the right and complete data.

d. Formulate model

i. Define variables representations and symbols.

ii. Assign units.

iii. Determine the relations and equations connecting the problem

variables.

iv. Formulate objective of the model.

v. Identify the form of outputs to be obtained.

vi. Specify constraints.

vii. Check model’s structure:

Given {values of input variables, parameters and constants},

find {values of output variables} such that {conditions are

satisfied or objective achieved}.

e. Obtain the mathematical solution of the model

Solving process may involve the following:

◦ Plot graphs

◦ Use Calculus approaches

◦ Apply numerical methods

◦ Apply optimization techniques

◦ Run simulations

◦ Execute computer programs

3. Analyze and Interpret solution

Analyze and examine results or solutions obtained.

a. Check whether values of variables have the correct sign and size?

b. Ask the following questions:

◦ Are solutions produced reasonable?

◦ Has data measurement and accuracy been achieved?

◦ Have the best solution found?


4. Compare with reality

a. Test results against real data.

b. Check the following:

◦ Do proposed predictions based on solution make sense?

◦ Do predictions agree with real data?

◦ Evaluate the model. Is the model efficient?

◦ Do results suggest more accuracy need to be achieved?

5. Write a report

The following questions can be considered in preparing the report.

◦ Who is the report for?

◦ What do the readers want to know?

◦ How detail should the report be?

◦ Are important features clearly described?

◦ Are the results stand out?

Figure 1.8: Mathematical Modeling Process

Identify the real problem

Formulate Model

Solve the Model

Interpret Solutions

Compare with Reality

Write a report

Validation

Problem Definition

Modeling

Translation

Conclusion


STEPS

1. Identify the Problem

Given {the box is to have square base and double thickness top and

bottom; cardboard costs RM1.50 per square meter}, find {the dimension of

the box} such that {the cost is minimized}.

Draw diagram to understand the problem better.

TOP TOP

SIDE SIDE SIDE SIDE SIDE

BOTTOM BOTTOM

Figure 1.9: Illustration for the Problem

Example 1: Most Economical Size Box

A company decides to produce its own box for packaging products manufactured by the company. It has been decided the box should hold 0.1m3 (0.1 cubic meters). The box should have a square base and double thickness top and bottom. Cardboard costs RM1.50 per square meter. You are given the task of designing the box and it is up to you to decide the most economical size. What is the minimum cost to produce such box.

h

w

w

h

w w

TOP

SIDE

BOTTOM

w

w


2. Formulate mathematical model

Assumption: Ignore the thickness of cardboard for this model.

Volume

The volume of the box is given to be 0.1m3. Thus,

(1.1)

Area of the 4 Sides

Area of double tops and bases

Total cardboard needed Area of Cardboard

Total Cost (C) Area of Cardboard (in m2) x price (in RM) per square meter

(1.2)

Let us use method of substitution to solve a system of two equations,

equations (1.1) and (1.2), with two unknowns, . From (1.1), rearrange the equation to get:

(1.3)

Substitute equation (1.3) into equation (1.2):

(1.4)

3. Solve the model

To find the dimension of the box which gives a minimum total cost,

differentiate equation (1.4) with respect to .

(1.5)

Set the derivative equals to 0 to get the value of .

m 0.37 m or cm 37 cm.


Substituting the value of in (1.3) gives us:

m 0.74 m or cm 74 cm

Find the second derivative of with respect to

(1.6)

Substituting into equation (1.6), we obtain,

Since

, we conclude that is minimum when meter and

meter. Substituting the values of and in equation (1.4), we

get the minimum cost, .

4. Analyze and Interpret the solution

Plotting the graph of equation (1.4) gives us the graph as shown in

Figure 1.10, where the point (0.37, 2.44) is the lowest point on the graph.

By examining the graph, the width ( ) could be anywhere between 0.35 m

to 0.38 m without affecting the minimum cost very much, that is

.

Figure 1.10: Graph of Cost (C) vs Width (w)

0.3, 2.54 0.4, 2.46

0

1

2

3

4

5

6

7

0 0.2 0.4 0.6 0.8 1 1.2

Co

st p

er

Bo

x (i

n R

M)

Width or length of the box, w (in m)

(0.37, 2.44)


Conclusion

Based on the mathematical model, the following can be recommended:

Length or width ( ) m,

Height ( ) m,

Cost ( )

But any width between 0.35 m and 0.38 m would be fine, since the cost

will not differ much.

Thus, a more practical solution may be:

Length or width ( ) m,

Height ( ) m,

Cost ( )

Refinement of the model

Improvements of this model may include the following:

Include cost of glue or staples

Include wages for assembly workers

Include wastage when cutting box shape from cardboard.

Should the design be changed to optimize use of cardboard?


1.6 Method of Least Squares The Method of Least Squares is a procedure to determine the best fit line to

data; the proof uses simple calculus and linear algebra. The basic problem is to

find the best fit straight line given that, for , the pairs

are observed. The method easily generalizes to finding the best fit of the

form:

where is not necessarily a linear function, but must be a linear

combination of these functions.

Given data, , we may define the error associated to

saying by

(1.7)

The goal is to find values of and that minimize the error. In multivariable

calculus we learn that this requires us to find the values of (a; b) such that

.

.

Differentiating and setting the derivatives equals to 0 gives:

and dividing by 2 yields:

.

.

We may rewrite these equations as:

.

.


It can be shown that solving the equations simultaneously yields:

.

.

From (1.12) and (1.13), we found that values of and which minimize the error

(defined in (1.7) satisfy the following matrix equation:

This, we have

Since the matrix A is invertible, this implies:

Define

,


= -1.03

120.88 Alternatively,

The equation of the best fit line is:

x y 1 26 92 676 8464 2392 2 30 85 900 7225 2550 3 44 78 1936 6084 3432 4 50 80 2500 6400 4000 5 62 54 3844 2916 3348 6 68 51 4624 2601 3468 7 74 40 5476 1600 2960

Example 2: Curve Fitting using the Method of Least Squares

Comsider the dat given below:

x 26 30 44 50 62 68 74

y 92 85 78 80 54 51 40

Find the best fit line using least squares method that describes the linear relationship between y and x.

Solution


Figure 1.10: Best Fit Line using Trendline in Excel

Identify the expected function : bxaey

Identify the unknown : a, b

y = -1.0344x + 120.88 R² = 0.9249

0

10

20

30

40

50

60

70

80

90

100

0 20 40 60 80

y

Example 3: Method of Least Squares and Linearization

A rectangular house is to be built with exterior walls that are 8 feet high. One

wall of the house will face north. The total enclosed area of the house will be

1500 square feet. Annual heating costs for the house are determined as

follows: each square foot of exterior wall with a northern exposure adds RM4 to

the annual heating cost; each square foot of exterior wall with an eastern or

western exposure adds RM2 to the annual heating cost; each square foot of

exterior wall with a southern exposure adds RM1 to the annual heating cost.

For what value of L is the annual heating cost minimized, and what is the

annual heating cost for this choice of L? The values given for the heating costs

per square foot can only be known approximately. How does the solution

depend on the given values for the heating costs?

Solution

Solution


Linearize the function :

bx

bx

aelnyln

aey

bxaln

)1elnelnbxaln

elnalnyln bx

Transform the data and/or unknown :

ylnY alna0 ba1

Identify normal equation

n

1i

2i

n

1i

i

n

1i

i

xx

xn

1

0

a

a =

n

1iii

n

1ii

Yx

Y

Determine the respective values:

i xi xi2 yi Yi xi Yi

1 0.4 0.16 800 6.6846 2.6738 2 0.8 0.64 975 6.8824 5.5059 3 1.2 1.44 1500 7.3132 8.7759 4 1.6 2.56 1950 7.5756 12.1209 5 2.0 4.00 2900 7.9725 15.9449 6 2.3 5.29 3600 8.1887 18.8340

3.8xn

1i

i

09.14yn

1i

i

6170.44Y

n

1i

i

8555.63Yxn

1i

ii

Substitute the respective values in the normal equation.

09.143.8

3.86

1

0

a

a =

8555.63

6170.44

Solve for the coefficients: 0a and 1a

3037.6a0 and

8187.0a1

Substitute to the transform unknown :

5906.546

ea

3037.6alna

3037.6

0

8187.0ba1

The expected function is x8187.0e5906.546y


kAxy

Taking natural logarithm both sides:

ln y = l n A + k ln x

Y Xaa 10

The necessary transformation for the data and unknowns are:

Y yln X xln Alna0 1ak

Rewrite

ln y = ln A + k ln x

as

Y = a0 + a1 X

=ROUND(LN(C4);4)

Example 4: Least Squares Technique

Use least-squares procedure to fit y = Axk to the following data.

xi 1.00 1.15 1.40 1.43 1.60 2.00

yi 4.33 4.58 4.98 5.06 5.28 5.80

Solution


Hence, the required normal equations:

6

1i

2i

6

1ii

6

1ii

xx

x6

1

0

a

a =

6

1iii

6

1ii

Yx

Y

9620.09971.1

9971.16

1

0

a

a =

3333.3

6358.9

Solving for a0 and a1:

a0 = 1.4649

Since a1 = 0.4239 then k = a1 = 0.4239

But ln A = a0 then

A = 0ae

= e1.4649

= 4.327

Hence, the expected function is y = Axk = 4.327x0.4239

Find the values of a and b of the form y = aex + be-x to the following data:

X 0 0.5 1.0 1.5 2.0 2.5 Y 5.02 5.21 6.49 9.54 16.02 24.53

Warm up exercise


1.7 Continuous Least-Squares Polynomials

The previous section considered the problem of least-squares

approximation to fit a collection of discrete data. The other approximation

problem concerns the approximation of functions or fitting continuous

data.

Consider a function y = f(x), which is continuous in [a, b]. Let

mm

ii

2210m xa...xa...xaxaa)x(P be a polynomial which

represent the function y = f(x), then the total least squares error is defined

as follows

∫b

a

2m dx(x)P- f(x) E

∫b

a

2mm

ii

2210 dxxa...xa...xaxaa- f(x)

We seek to minimize the sum of error squares. From the calculus of

functions of several variables, a necessary condition for the values a0, a , …,

and am to minimize E such that

0=a

E...=

a

E...=

a

E=

a

E

mi10 ∂

∂

∂

∂

∂

∂

∂

∂ mi0

Hence,

∫b

a

2mm

ii10

0

0dxxa...xa...xaa- f(x)a

(1)

∫b

a

2mm

ii10

1

0dxxa...xa...xaa- f(x)a

(2)

)(0dx......- f(x)

.

.

.

)(0dx......- f(x)

.

.

.

∫

∫

b

a

2

10

b

a

2

10

mxaxaxaaa

ixaxaxaaa

m

m

i

i

m

m

m

i

i

i


Simplifying (1) yields

∫b

a

mm

ii10 0dx)1(xa...xa...xaa- f(x)2

b

a

b

a

mm

b

a

ii

b

a

1

b

a

0

b

a

b

a

mm

b

a

ii

b

a

1

b

a

0

b

a

mm

ii10

dx)x(fdxxa...dxxa...xdxadxa

dx)x(fdxxa...dxxa...xdxadxa

0dx)x(fxa...xa...xaa∫

Simplifying (2) to (m) using similar procedures yield the normal equations

b

a

b

a

1mm

b

a

1ii

b

a

21

b

a

0 dx)x(xfdxxa...dxxa...dxxaxdxa

b

a

ib

a

imm

b

a

i2i

b

a

i21

b

a

i0 dx)x(fxdxxa...dxxa...dxxadxxa

: .

b

a

mb

a

m2m

b

a

mii

b

a

m21

b

a

m0 dx)x(fxdxxa...dxxa...dxxadxxa

In matrix form the normal equations can be represented as:

b

a

m

b

a

i

b

a

b

a

m

i

1

0

b

a

m2b

a

imb

a

1mb

a

m

b

a

mib

a

i2b

a

1ib

a

i

b

a

1mb

a

1ib

a

2b

a

b

a

mb

a

ib

a

b

a

dx)x(fx

.

.

dx)x(fx

.

.

dx)x(xf

dx)x(f

a.

.

a.

.

a

a

dxx...dxx...dxxdxx

..................

dxx...dxx...dxxdxx

..................

dxx...dxx...dxxxdx

dxx...dxx...xdxdx


Example 5:

Derive the normal equation in matrix form to approximate y = f(x) with a

straight line on the interval [a, b].

Let xaa)x(P 101

0=a

E=

a

E

10 ∂

∂

∂

∂

∫b

a

21 dx(x)P- f(x) E

∫b

a

210 dxxaa- f(x)

b

a

b

a

1

b

a

0

b

a

b

a

1

b

a

0

b

a

10

b

a

10

b

a

210

0

dx)x(fxdxadxa

dx)x(fxdxadxa

0dx)x(fxaa

0dx)1(xaa- f(x)2

0dxxaa- f(x)a

∫

∫

∫

Solution

Steps : Fitting Continuous Least Squares Polynomial Determine the degree m of the polynomial : m Write the general expression of the polynomial to be fitted

Pm(x) = mm

2210 xa ... xa x a a ++++

Write the normal equations in matrix form.

Compute : b

a

idxx for m2i0 and 1

0

i dx)x(fx for mi0

Solve for the coefficients a0, a1…am. Thus, the least-squares function is:

f(x) = mm

2210 xa ... xa x a a ++++

Solve for a0 and a1. Thus, the least-squares line is f(x) = a0 + a1x.


∫

∫

∫

b

a

10

b

a

10

b

a

210

1

0xdx)x(fxaa

0dx)x(xaa- f(x)2

0dxxaa- f(x)a

b

a

b

a

21

b

a

0

b

a

b

a

21

b

a

0

dx)x(xfdxxaxdxa

dx)x(xfdxxaxdxa

b

a

b

a

1

0b

a

2b

a

b

a

b

a

dx)x(xf

dx)x(f

a

a

dxxxdx

xdxdx

Example 6: Find the least squares polynomial approximation of degree

one to y = x 2 + 4x + 4 on the interval [0,2].

2

0

2

0

1

02

0

22

0

2

0

2

0

dx)x(xf

dx)x(f

a

a

dxxxdx

xdxdx

3

683

56

a

a

3

82

22

1

0

997.5

336.3

a

a

1

0

Hence, x997.5336.3)x(P1

Solution


Example 7:

Find the least squares polynomial approximation of degree two to f(x) =

ex on the interval [ 0, 1]

Determine the degree m of the polynomial

m = 2

Write the general expression of the polynomial to be fitted

P2(x) = 2210 xa x a a

Write the normal equations in matrix form.

1

0

x2

1

0

x

1

0

x

2

1

0

1

0

41

0

31

0

2

1

0

31

0

21

0

1

0

21

0

1

0

dxex

dxxe

dxe

a

a

a

dxxdxxdxx

dxxdxxxdx

dxxxdxdx

Compute : 1

0

idxx for 4i0 and 1

0

i dx)x(fx for mi0

7183.0

1

7183.1

a

a

a

2.025.03333.0

25.03333.05.0

3333.05.01

2

1

0

Solve for the coefficients a0, a1…am.

8449.0

8445.0

0140.1

a

a

a

2

1

0

Thus, the least-squares function is:

P2(x) = 0.8449x 0.8445x 1.0140 2

2.7 Legendre Polynomial

The previous section considered the problem of least-squares

approximation to fit a collection of discrete data. The other approximation

problem concerns the approximation of functions or fitting continuous data.

In this section we shall learn to fit data in continuous form using sequences

of Legendre polynomials.

Solution


Definition

The functions listed below are called Legendre polynomials and are

defined for

-1 x 1:

P0(x) = 1

P1(x) = x

1- 3x2

1)x(P 2

2

3x - 5x2

1)x(P 3

3

3+30x - 35x8

1)x(P 24

4

15x+70x - 63x8

1)x(P 35

5

.

.

.

1- mm1m P 1+m

mP

1+m

1+2m)x(P

-

This set of Legendre polynomials is said to be orthogonal on [-1,1] with

respect to the weight function w(x) 1 .The criteria required is that these

functions are designed to satisfy the following orthogonality condition:

1

1

nm mnif1n2

2mnif0

dx)x(P)x(P

(details of orthogonality and weight function shall not be discussed here)

In general, the Legendre polynomials can also be derived by the formula:

n2

n

n

nn 1) - x(dx

d

!n2

1)x(P

Suppose y(x) is a function continuous on [-1,1]. Here, the approach to

finding the least-squares approximating polynomial f(x) to fit the function

y(x) (or, the continuous data) is done in a similar manner. Let f(x) be of

polynomial of degree m defined using sequences of Legendre polynomials

such that

f(x) = a0P0(x) + a1P1(x) + a2P2(x) + . . . + amPm (x) .


We seek to minimize the sum of error squares; i.e.

∫1

1-

2dxy(x)- f(x) L

∫1

1-

2mm1100 dxy(x)- (x)Pa + (x)Pa +(x)Pa

(1)

where, Pi (x) is a Legendre polynomial and a i is a constant coefficient.

From the calculus of functions of several variables, a necessary condition

for the values a0, a , …, am to minimize L is that

0,=a

L

k∂

∂for each k = , , …, m.

Hence, using (1),

0dx)x(y- )x(Pa )x(Pa )x(Pa a

L1

1-

2mm1100

k

∫

With the orthogonality property of Legendre polynomials, this term can be

simplified to:

0dx)x(P)x(y- )x(P)x(Pa

1

1-

2kkkk ∫

∫∫1

1-

k

1

1-

2kk dx)x(P)x(yx)x(P.a

∫1

1-

kk dx)x(P)x(ya1k2

2

Thus,

∫1

1-

kk dx)x(P)x(y2

1k2a

Notice that ak calculates the coefficients a0, a1, …, am with the condition that

x is defined for y(x) on the interval [-1,1].


Theorem

Suppose y(x) is continuous and defined on [-1,1], then y(x) can be

approximated by a least-squares polynomial f(x) of degree m, using

series of Legendre polynomials such that:

f(x) = a0P0(x) + a1P1(x) + a2P2(x) + . . . + amPm(x)

where, the coefficients a0, a1, …, am is be determined by

1

1

kk dx)x(P)x(y2

1k2a for k = , , , ,……….,m

Example 8:

Find the least squares polynomial approximation of degree one to y = x 2 +

4x + 4 on the interval [-1,1].

Identify the observed function :

y(x) = x 2 + 4x + 4

Determine the Legendre polynomials of degree 1:

f(x) = a0P0(x) + a1P1(x)

= a0 + a1x

where P0(x) = 1 and P1(x) = x

Determined the coefficients a0 and a1:

1

1

kk dx)x(P)x(y2

1k2a for k = 0,1

Solution

Steps : Fitting continuous data (or function) using Legendre polynomials if independent variable is defined on [-1,1] Identify the observed function: y(x) Determine the Legendre polynomials of degree m:

f(x)= a0P0(x) + a1P1(x) + a2P2(x) + . . . + amPm(x)

Determine the coefficients a0, a , …, am :

1

1

kk dx)x(P)x(y2

1k2a for k = , , , ,…m


3

13

dx)1)(4x4x(2

1

dx)x(P)4x4x(2

1)0(2a:0k

1

1

2

1

1

02

0

4

dxx)4x4x(2

3

dx)x(P)4x4x(2

1)1(2a:1k

1

1

2

1

1

12

1

Thus, the polynomial of degree one to fit y(x) is:

f(x) = a0P0(x) + a1P1(x)

= 3

13 + 4x

Other method which can be applied is

1

1

1

1

1

01

1

21

1

1

1

1

1

dx)x(xy

dx)x(y

a

a

dxxxdx

xdxdx

In the case where x (or the independent variable) is defined on [m,n], a

suitable linear transformation is required so that the interval range of the

independent variable is normalized to be on [-1,1].

Steps : Fitting continuous data (or function) using Legendre polynomials if independent variable is not defined on [-1,1] Transform x to t linearly: ]1,1[t]n,m[x

Let batx

Solve for a and b Rewrite y(x) in term of y(t) Determine the Legendre polynomials of degree m

f(t) = a0P0(t) + a1P1(t) + a2P2(t) + . . . + amPm(t)

Determine the coefficients a0, a , …, am :

1

1

kk dt)t(P)t(y2

1k2a for k = 0,1,2,3,..,m

The polynomial of degree m to fit y(t) is: f(t) = a0P0(t) + a1P1(t) + a2P2(x) + . . . + amPm(t)

Rewrite f(t) in term of f(x).


Example 9:

Find the least squares polynomial approximation of degree one to y = x 2

+ 4x + 4 on the interval [0,2]

Since ]2,0[x and ]1,1[x then transformation is required.

Transform x to t linearly:

]1,1[t]2,0[x

Let . x = at + b

Solve for a and b

When x = 0: t = - 1 : 0 = a(- + b …

When x = 2: t = : = a + b …

Solving (1) and (2):

a=1 and b=1

Thus

x = t + 1 or t = x – 1

Rewrite y(x) in term of y(t):

y(x) = x 2 + 4x + 4, but x = t+1

y(t) = (t+1) 2 + 4(t+1) + 4

= t 2 + 6t +9

Determine Legendre polynomials of degree 1:

f(t) = a0P0(t) + a1P1(t)

= a0 + a1t

where P0(t)=1 and P1(t)=t

Determine the coefficients a0 and a1:

1

1

kk dt)t(P)t(y2

1k2a for k = 0,1

3

28dt)1)(9t6t(

2

1

dt)t(P)9t6t(2

1)0(2a:0k

1

1

2

1

10

20

Solution


6dt)t)(9t6t(2

3

dt)t(P)t(y2

1)1(2a:1k

1

1

2

1

111

The polynomial of degree 1 to fit y(t) is:

f(t) = a0P0(t) + a1P1(t)

= 3

28 + 6t

Rewrite f(t) in term of f(x):

f(x) = 3

28 + 6(x-1) since t = x – 1

= 3

10 + 6x

Example 10:

Find the least squares polynomial approximation of degree two to f(x) =

ex on the interval

a) [-1, 1] b) [ 0, 1]

No transformation required since ]1,1[x .

Given

y(x) = ex

and

f(x) = a0P0(x) + a1P1(x) + a2P2(x) polynomial of degree two

)1x3(2

1axaa

)x(Pa)x(Pa)x(Pa

221o

221100

since

)1x3(

2

1)x(Pandx)x(P,1)x(P 2

2110

Determined the coefficients a0 ,, a1 and a2:

1

1

kk dx)x(P)x(y2

1k2a for k = 0,1,2

Solution


1752.1

dxe2

1

dx)x(Pe2

1)0(2a:0k

1

1

x

1

1

0x

0

1036.1

dxxe2

3

dx)x(Pe2

1)1(2a:1k

1

1

x

1

1

1x

1

3578.0

dx)1x3(2

1e

2

5

dx)x(Pe2

1)2(2a:2k

1

1

2x

1

1

2x

1

Hence,

.x5367.0x1036.19963.0

)1x3(2

13578.0x1036.11752.1

)1x3(2

1axaa

)x(Pa)x(Pa)x(Pa)x(f

2

2

221o

221100

b)

Need a suitable linear transformation since ]1,1[x and ]1,0[x

Make a suitable linear transformation of x to t:

]1,1[t]1,0[x

i.e. x = at + b, solve for a and b

when x = 0, t = - 1 : 0 = a(-1) + b = - a + b …

when x = , t = : = a + b = a + b …

Solving (1) and (2):

2

1band

2

1a


Thus,

1x2t1tx2or2

1t

2

1t

2

1x

Write y(x) in the term of y(t):

2

1t

x

e)t(y

e)x(y

Write the least-squares polynomial in the new variable t in the form of

Legendre polynomials:

f(t) = polynomial of degree two (quadratic)

)1t3(2

1ataa

)t(Pa)t(Pa)t(Pa)t(f

221o

221100

Determine the coefficients a0 , a1 and a2:

1

1

kk dt)t(P)t(y2

1k2a for k = 0,1,2

7183.1

dte2

1

dt)t(Pe2

1)0(2a:0k

1

1

2

1t

1

10

2

1t

0

8452.0

dtte2

3

dt)t(Pe2

1)1(2a:1k

1

1

2

1t

1

11

2

1t

1

1398.0

0559.02

5

dt)1t3(2

1.e

2

5

dt)t(Pe2

1)2(2a:2k

1

1

22

1t

1

12

2

1t

2

Thus,

)t(Pa)t(Pa)t(Pa)t(f 221100


2

2

t8388.0t8452.06484.1

)1t3(2

1)1398.0(t8452.07183.1)t(f

Hence

2

2

x0968.13x7872.143097.1

)1x2(8388.0)1x2(8452.06484.1)x(f

Find the least squares polynomial approximation of degree two to:

a) f(x) = sin 2x on the interval [-1,1] and [0, 1]

b) f(x) = ln x on the interval [1,3]

Warm up exercise


Exercise 4

1. Fit a straight line to the given data.

a) X y b) x y c) x y -2 1 -6 -5.3 -3 -6

-1 2 -2 -3.5 1 -4

0 3 0 -1.7 3 -2

1 3 2 0.2 5 0

2 4 6 4.0 9 4

2. Fit a parabola to the given data.

a) X y b) x y c) x Y 2.0 5.1 -3 15 -2 2.80

2.3 7.5 -1 5 -1 2.10

2.6 10.6 1 1 0 3.25

2.9 14.4 3 5 1 6.00

3.2 19.0 2 11.50

3. The table below shows the time (in seconds) required for water to drain

through a hole in the bottom of a bottle as a function of depth (in meters) to which the bottle has been filled.

Depth 0.05 0.10 0.15 0.20 0.30 0.35 0.40 Time 65.99 120.28 166.69 207.85 279.95 313.04 344.24

a) Find the regression line to fit the data b) Estimate the time required for water to drain at depth of 2.5 meters. 4. Find the best-fit line for the given values.

x 5 6 10 14 16 20 22 28 28 36 38 y 30 22 28 14 22 16 8 8 14 0 4

5. Fit a least-squares quadratic function (parabola) to the given data.

x 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 p(x) 0.0 2.0 5.0 8.0 12.0 25.0 40.0 57.0

6. The following tabulated values are taken from an experiment. Complete

the table and calculate the sum of error squares.

x 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 Observed y 0.0 2.0 5.0 8.0 12.0 25.0 40.0 57.0 Calculated y

Error


7. Find the values of A and B to fit a curve of the following form to the given

data

a) BAx

1y

b) B

x

Ay

8. The data given are for the solubility S of n-butane in anhydrous

hydrofluoric acid at high temperature T needed in the design of petroleum

refineries. Determine the constants and to fit the data to the model

TeS . Estimate the solubility when temperature is 95F.

9. One of the following data sets, (x,y1) and (x,y2) , follows an exponential

law bxAey and the other follows a power law bAxy . Which is which?

Fit the data to the respective model and calculate the sum square errors.

X 2.0 2.5 3.0 3.5 4.0 4.5 5.0

y1 14.79 27.75 47.09 74.07 109.99 156.10 213.69

y2 12.13 19.58 31.59 50.97 82.21 132.59 213.82

10. Find the least- squares polynomial approximation of degree one to the

function f(x) on the indicated interval.

a) f(x) = e2x ; [-1, 1] b) f(x) = sinx ; [-1, 1]

c) x

1)x(f ; [-3, -1] d) f(x) = cos 2x ; [0, 0.5] ,

e) f(x) = x3 – 1; [0,2] f) f(x) = ln x; [1, 2]

11. Find the least- squares polynomial approximation of degree two to the

function f(x) on the indicated interval.

a) f(x) = e-x ; [0, 1] b) f(x) = cos x; [-1, 1]

c) f(x) = x3 – x + 1; [0, 3] d) f(x) = sin2x; [0, ]

x -1 0.1 1 2 3

y 6.62 3.94 2.17 1.35 0.89

Temperature F 77 100 185 239 285

Solubility (weight %) 2.4 3.4 7.0 11.1 19.6

a1 chapter 1 mat530 as latest 5 may 2012

Documents