v. rouillard 2003 1 introduction to measurement and statistical analysis assessing experimental...

V. Rouillard 2003

1 Introduction to measurement and statistical analysis

ASSESSING EXPERIMENTAL DATA : ERRORS

• Remember: no measurement is perfect – errors always exist.

• Measurement error is defined as the difference between the true value and the

measured value.

• We can only estimate the size of the error or its likelihood that it exceeds a certain

value.

• Errors can be estimated statistically when large number of measurements are taken.

• However, must ensure that measurement systems are calibrated.

V. Rouillard 2003


TYPES OF ERROR

• Most errors can be put into two classes: Bias errors and Precision errors.

• Bias errors are also referred to as systematic errors and remain the same for every

measurement made.

• Precision errors are also called random errors and are different for each measurement

made. However, the average value of the random error is zero.

• If enough measurements are repeated, the distribution of precision errors will be

revealed and the likely size of the error can be estimated statistically.

• Because bias errors are fixed and do not produce a statistical distribution, they cannot

be estimated using statistical techniques.

• They can only be estimated by comparison with a standard or another instrument or

even by experience and common sense.

V. Rouillard 2003


TYPES OF ERRORLarge bias error &

small random error

Small bias error & large random

error

Fre

qu

enc

y o

f o

ccu

rren

ce

Fre

qu

enc

y o

f o

ccu

rren

ce

V. Rouillard 2003


COMMON SOURCES OF ERROR

Bias errors:

• Calibration (eg: zero-offset and scale adjustments)

• Certain consistently recurring human errors (eg: parallax, poor

synchronisation)

• Certain errors caused by defective equipment (eg: poor design, fabrication

and maintenance)

• Loading errors (eg: microphone, vehicle speed gun)

• Resolution limitations (eg: lack of significant figures in digital displays)

Random errors:

• Certain human errors (eg: lack of concentration)

• Disturbances to equipment (eg: ground vibrations, atmospheric conditions)

• Fluctuating experimental conditions (eg: poor experimental design, must

account for inherent oscillations/variations of the measurand)

• System sensitivity imitations (eg: use bathroom scale to measure mass of

small animal)

V. Rouillard 2003


COMMON SOURCES OF ERROR

Combined errors:

• Backlash, friction and hysteresis (eg: in mechanical indicators such as

pressure gauges)

• Calibration drift or reaction to changing environmental conditions.

• Variations in procedure (eg: when short cuts are taken or personnel changes)

Illegitimate errors (mistakes):

• Blunders and mistakes (eg: forgot to switch on amplifier, write phone number

instead of reading)

• Computational errors (eg: use wrong calibration constant)

V. Rouillard 2003


INSTRUMENT PERFORMANCE : TERMINOLOGY

• Accuracy: (expected) closeness with which a measurement approaches the true

value.

• Precision: indication of the reproducibility of measurements. If a variable is fixed,

precision is the measure of the degree to which successive measurements differ

from one another.

• Resolution: The smallest change in the measurand that the instrument will detect.

• Sensitivity: The ratio of the instrument response to an change in the measured

quantity. Eg: and accelerometer with a sensitivity of 100 mV/g is more sensitive

than one with a sensitivity of 10 mV/g.

• Error: Difference between the true value and the measured value.

V. Rouillard 2003


UNCERTAINTY : ESTIMATING THE LEVEL OF MEASUREMENT ERROR.

• Total uncertainty, U, combines the bias and random uncertainties as follows:

• This method is based on the assumption that the sources of bias and random

errors are independent and they are therefore unlikely to coincide.

• Remember: The bias uncertainty is estimated from calibration checks while the

random uncertainty is estimated by statistical analysis of repeat measurements.

U B Rx x x ( )2 2

V. Rouillard 2003


ESTIMATING THE RANDOM (PRECISION) UNCERTAINTYStatistical Analysis

• A measurement sample is drawn from the population to make an estimate of the

measurand.

• In may be that no two samples (4 blades) will have precisely the same value.

• But each sample (and specimen) should approximate the average value for the

populationPopulation

V. Rouillard 2003



• A measurement sample is drawn from the population to make an estimate of the

measurand.

• In may be that no two samples (4 blades) will have precisely the same value.

• But each sample (and specimen) should approximate the average value for the

populationPopulation Sample

(random selection from population)

V. Rouillard 2003



• Manufacturing (production) uncertainty: Analyse repeat measurements from the sample

(each specimen is measured only once).

• Experimental uncertainty: Analyse repeat measurements from one individual specimen

only.

V. Rouillard 2003



• Statistical analysis and interpretation meaningful only of a (relatively) large number of

measurements are made.

• Systematic errors should be kept small. Statistical treatment cannot remove systematic

(bias) errors.

• Arithmetic mean:

• Deviation from the mean: Difference between an individual reading and the mean of

the group of readings: (note: the algebraic sum of all deviations = zero)

xxn

d x x d x x d x xn n1 1 2 2 .......

V. Rouillard 2003



• Average deviation: an indication of the precision of the measurements:

• Standard deviation: the root-mean-square (RMS) deviation of the measurements. For

a finite number of readings:

• Variance: mean-square deviation = 2

Dd d d d d

n

d

nn

1 2 3 4 .......

d d d d d d

nd

nn t1

222

32

42

42 2 2

1 1.......

V. Rouillard 2003



• Case Study: Measurement of the mass of turbine blades for use in jet propulsion

systems. Blades are supplied by different manufacturers. Mass must be established

based on random sample.

V. Rouillard 2003



• Probability Distribution of errors: The frequency distribution of observations can

be calculated and displayed graphically using a histogram or frequency distribution

plot:

Mass [g]

Nu

mb

er o

f o

bse

rva

tio

ns

1 2

12

24

34

27

16

311

V. Rouillard 2003



• If more observations were made it is expected that the frequency distribution of the

observations will become more defined:

Mass [g]

Nu

mb

er o

f o

bse

rva

tio

ns

V. Rouillard 2003



• This bell-shaped curve has been shown to approach the distribution function called

the Normal or Gaussian distribution.

Mass [g]

Nu

mb

er o

f o

bse

rva

tio

ns

2( ) expf x n x

V. Rouillard 2003



• This bell-shaped curve has been shown to approach the distribution function called

the Normal or Gaussian distribution.

0

0.1

0.2

0.3

0.4

0.5

-4 -3 -2 -1 0 1 2 3 4

p(x)

(x-)/

21 1

( ) exp22

xp x

V. Rouillard 2003



• The normal distribution function characteristically has few observations at the high

and low ends and many in the middle. It has been shown to be very useful in for

evaluating random errors.

Mass [g]

Nu

mb

er o

f o

bse

rva

tio

ns

1 2

12

24

34

27

16

311

V. Rouillard 2003



Comments on the normal distribution of random errors:

• All observations include small, disturbing effects called random errors.

• Random errors can be positive or negative with equal probability.

• Small errors are more likely to occur that large errors.

• Very large errors (> 3) are very improbable

• The probability of a given error will be symmetrical about zero.

V. Rouillard 2003



Interpretation of the normal distribution of random errors:

Error Level Probability

Terminology [%]

Probable error 0.6754 50.0

Std deviation 1 68.3

90% error 1.645 90.0

2-Sigma error 1.96 95.0

3-Sigma error 3 99.7

4-Sigma error 4 99.994

Error Level

21 1

( ) exp22

xp x

V. Rouillard 2003



• Turbine blade mass case study:

Mass [g]

Nu

mb

er o

f o

bse

rva

tio

ns

Standard deviation:

0.75 g

Mean: 319.75 g

V. Rouillard 2003


ESTIMATING THE STATISTICAL DISTRIBUTION OF A RANDOM PROCESS

• Case Study: Analysis of maximum daily wave height for the design of an offshore structure. Measurements made continuously by a wave rider buoy which stores the daily maximum wave height and transmits the data to a base station.

V. Rouillard 2003



Day of year

Dai

ly m

ax.

wav

e h

eig

ht

[m]

Sample: record for one year (random?)

V. Rouillard 2003



• The frequency distribution of observations can be calculated and displayed

graphically using a histogram or frequency distribution plot:

Wave height [m]

Nu

mb

er o

f o

bse

rva

tio

ns

4

1214

46

67

72

62

38

13

24

Maximum Daily Wave Height

[m]

Number of observations

2 43 124 145 466 677 728 629 3810 2411 1312 313 014 015 1

31

V. Rouillard 2003



• With more observations, it is expected that the frequency distribution will approach the Normal or Gaussian distribution

Wave height [m]

Nu

mb

er o

f o

bse

rva

tio

ns

V. Rouillard 2003



• The normal distribution has been shown to be very useful in for describing many

random variables such as test scores, people height, weight etc.,

Wave height [m]

Nu

mb

er o

f o

bse

rva

tio

ns

Maximum Daily Wave Height

[m]

Number of observations

2 43 124 145 466 677 728 629 3810 2411 1312 313 014 015 1

V. Rouillard 2003



• The normal (Gaussian) distribution is a function of the mean and standard deviation

of the sample:

• Where is the mean

And the standard deviation.

In this example:

The mean daily max. height = 7.5 m

The standard deviation is = 2.0 m

3 (99.7% 332 days per 333 days)

the expected ann. max. wave height is:

7.7 + 3(2.0) = 13.7 m

4 (99.994% all but 1 day per 45 yrs)

the expected max. wave height over

45 yrs is: 7.7 + 4(2.0) = 15.7 m

f xx

( ) exp F

HIK

LNMM

OQPP

1

2

12

2

Wave height [m]

Nu

mb

er o

f o

bse

rva

tio

ns

2.0

7.5 m

v. rouillard 2003 1 introduction to measurement and statistical analysis assessing experimental...

Documents

measurement error

computational errors

perfect errors

systematic errors

certain human errors

maintenance loading

digital displays random

measurement systems