A Practical approach to
measurement uncertainty in
a civil materials laboratory
settingSession M309
Objectives
➢ Background;
➢ Systematic Approach to Measurement Uncertainty;
➢ Uncertainty of Measurement (UoM) example focussing
on compacted voids determination;
➢ Summary.
❖ With support of: G. Mturi & K. Mogonedi, CSIR
B. Pearce, Learning Matters etc
H. Badenhorst, NLA
2
Background
“A measurement without a clear
understanding
of uncertainty lacks merit” ¹
¹ O’Connell et al., 2011
3
Background
² Cambridge University Press, 2019
4
What is measurement uncertainty:
Measurement:
“a value, discovered by measuring, that corresponds to the size, shape, quality, etc. of
something”²
Uncertainty:
“a situation in which something is not known, or something that is not known or certain” ²
Measurement ≠ True Value
𝑴𝒆𝒂𝒔𝒖𝒓𝒆𝒎𝒆𝒏𝒕 = 𝑩𝒆𝒔𝒕 𝒆𝒔𝒕𝒊𝒎𝒂𝒕𝒆 ± 𝑼𝒏𝒄𝒆𝒓𝒕𝒂𝒊𝒏𝒕𝒚
Why is measurement uncertainty determination and use important and what
effect does it have:
It provides the range of error within which the true value of a measurement will
fall;
Requirement for testing and calibration facilities endeavouring to obtain or
maintain SANAS 17025 status under clause 7.6 of SANS 17025 standard;
Results coming from civil materials laboratories are used for pass/fail decision
making in engineering and construction projects;
Results are used to determine whether materials are suitable for use in specific
applications:
What is the cost of erroneous results or results provided without a known
degree of uncertainty to our clients/stakeholders?
Background
5
Background
6
How is measurement uncertainty determined in a civil materials setting:
Measurement uncertainty can be daunting to grasp, calculate and use effectively,
but…
The trick is not to panic and realise that civil materials can be
inherently variable and can cause significant error in results by
itself and we just have to take it one step at a time.
Measurement Uncertainty Approach
7
How do your results become traceable back to international standard?
𝑈𝐼𝑆𝑂
𝑈𝑁𝑆𝑂 = 𝑈𝑁𝑅𝑆 + 𝑈𝐼𝑆𝑂
𝑈17025 𝐶𝑎𝑙 = 𝑈𝑁𝑅𝑂 + 𝑈𝐿𝑎𝑏 𝑠𝑡𝑎𝑛𝑑𝑎𝑟𝑑𝑠 𝑢𝑐 =
𝑖=1
𝑛
𝑢𝑖2
Measurement Uncertainty Approach
8
ISOGUM – ISO Guide to the Expression of Uncertainty
Develop a model of the uncertainty measurement process;
Determine the uncertainty components based on the model;
Calculate the sensitivity coefficients;
Calculate the component uncertainties;
Calculate the associated degrees of freedom as required;
Convert all uncertainties into uncertainties expressed in the same
units as the measurand;
Combine all the uncertainties. ¹
¹ O’Connell et al., 2011
Measurement Uncertainty Approach
9
ISOGUM – ISO Guide to the Expression of Uncertainty
Develop a model of the uncertainty measurement process;
The model must be realistic and must categorise variation inducing
factors into their respective variation types including their analysis regimes.
¹ O’Connell et al., 2011
Type A Analysis
Statistical calculation
Type B Analysis
Non-statistical, ex. Calibration certificate
uncertainties, reference tables or books,etc.
Random Error
- Always present
- Unpredictable
- Caused by instrumentation, changes in
environmental conditions, etc.
- Can use statistical analysis to analyse data
Systematic Error
- Predictable and typically constant
- Causes can be accounted for or eliminated
- Often caused by imperfect observation
methods or calibrations
Measurement Uncertainty Approach
10
ISOGUM – ISO Guide to the Expression of Uncertainty
Determine the uncertainty components based on the model:
Isolate each component in the model that is anticipated to cause
variation in the measured results.
Calculate the sensitivity coefficients:
Sensitivity coefficients are multipliers that are used to convert uncertainty
components into the correct units of measure and magnitude required. ³
Not needed if the component is already in the unit and magnitude required. ³
Calculate the component uncertainties:
Determine component uncertainty 𝑼𝒊 as laid out in model.
¹ O’Connell et al., 2011
³ ISOBUDGETS, 2019
Measurement Uncertainty Approach
11
ISOGUM – ISO Guide to the Expression of Uncertainty
Calculate the associated degrees of freedom as required:
“The number of values in the final calculation which are free to vary” ³
Convert all uncertainties into uncertainties expressed in the
same units as the measurand:
Use sensitivity coefficients determined to convert uncertainties into same
measurement unit.
Combine all the uncertainties:
𝑢𝑐 =
𝑖=1
𝑛
𝑢𝑖2
³ ISOBUDGETS, 2019
Measurement Uncertainty Example
12
To illustrate an approach toward measurement uncertainty determination,
let us utilise the process to determine the uncertainty involved with the
determination of voids in compacted asphalt specimens.
To start, one must first isolate the variables of the voids calculation:
𝑉𝑜𝑖𝑑𝑠 =𝑀𝑉𝐷 − 𝐵𝐷
𝑀𝑉𝐷𝑥 100
Where:
BD – Bulk density of compacted density determined from SANS 3001 AS10
MVD – Maximum voidless density determined from SANS 3001 AS11
Overall uncertainty of measurement for compacted voids is thus
compounded from the UoM of the MVD and the BD test procedures.
Measurement Uncertainty Example
13
Something to keep at the back of your mind as we go through the process….
Sources of Error
Samples
Procedure
PeopleEnvironment
Equipment
Measurement Uncertainty Example
BD uncertainty analysis model:
14
Measurement Uncertainty Example
BD uncertainty analysis:
Possible error inducing factors
Equipment People Procedure Environment Sample
Thermometer Competency Specimen
temperature
Ambient
temperature
Incoming sample
representativeness
Waterbath Multiple operators
performing tests
Soaking time Humidity In-lab sample
handling to obtain
test specimens
Digital Balance Time constraints Handling time
before SSD
Multiple test
influence
UoM stemming from
compaction process
Cloth Attention to detail Cloth dampness Space limitations
Stopwatch/Timer
Measurement Uncertainty Example
BD uncertainty analysis:
1. Standard Uncertainty (𝑈𝑠) for inherent variability of the material _ Type A
… Experimental Standard deviation of the mean (EDSM):
𝑬𝑫𝑺𝑴 =𝝈
𝒏=𝟖. 𝟏𝟑
𝟏𝟔= 𝟐. 𝟎𝟑 𝒌𝒈/𝒎𝟑
2. Temperature effect uncertainty (𝑈𝑇) _ Type B
Thermometer uncertainty -> ± 0.2°C
thus
0.4307 𝒌𝒈/𝒎𝟑/ °C x 0.2 °C = ± 0.09 𝒌𝒈/𝒎𝟑
𝑼𝑻 =𝟎. 𝟎𝟗
𝟑= 𝟏. 𝟕𝟑𝒌𝒈/𝒎𝟑
y = -0.4307x + 2588.1R² = 0.94942576
2576
2577
2577
2578
2578
2579
2579
20 22 24 26 28 30
BD
Temperature
Change in BD with Variation in temperature
Note: This is
only an exerpt
from the full
analysis for
illustration
purposes
Measurement Uncertainty Example
BD uncertainty analysis:
3. Soaking Time effect (𝑈𝑠𝑡) _ Type A
Taking variation within the 3 – 5 minutes soaking time limits from the
method into account:
𝑈𝑠𝑡 =𝝈
𝒏=
𝟑.𝟑𝟗
𝟖= 𝟏. 𝟐𝟎 𝒌𝒈/𝒎𝟑
4. Handling time effect (𝑈ℎ𝑡) _ Type A
As method does not specify handling time, standard procedure for lab of <20 seconds was
checked in conjunction with 10 seconds handling time variance.
𝑼𝒉𝒕 =𝟕. 𝟐𝟖
𝟑𝟐= 𝟏. 𝟐𝟗𝒌𝒈/𝒎𝟑
Note: This is
only an exerpt
from the full
analysis for
illustration
purposes
Measurement Uncertainty Example
BD uncertainty analysis:
BD combined uncertainty (𝑈𝐵𝐷𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑)
The BD of the specimens tested for this particular sample is therefore:
2572 ± 3 𝒌𝒈/𝒎𝟑
𝑢𝑐 =
𝑖=1
𝑛
𝑢𝑖2
𝑢𝑐 = 𝑢𝑠2 + 𝑢𝑇
2 + 𝑢ℎ𝑡2 + 𝑢𝑠𝑡
2
𝑢𝑐 = 2.032 + 1.732 + 1.292 + 1.202
𝒖𝒄 = 𝟑. 𝟐𝟎 𝒌𝒈/𝒎𝟑 ≅3 𝒌𝒈/𝒎𝟑Note: This is
only an exerpt
from the full
analysis for
illustration
purposes
MVD uncertainty analysis model:
Measurement Uncertainty Example
19
Measurement Uncertainty Example MVD uncertainty analysis:
Possible error inducing factors
Equipment People Procedure Environment Sample
Thermometer Competency Sample
temperature
Ambient
temperature
Incoming sample
representativeness
Waterbath Multiple operators
performing tests
Time under vacuum Humidity In-lab sample
handling to obtain
test specimens
Digital Balance Time constraints Vacuum pressure Multiple test
influence
Pressure gauge/
manometer
Attention to detail Flask handling Space limitations
Stopwatch/Timer Shaking intervals
Soap solution effect
Measurement Uncertainty Example
MVD uncertainty analysis:
1. Standard Uncertainty (𝑈𝑠) for inherent variability of the material _ Type A
… Experimental Standard deviation of the mean (EDSM):
𝑬𝑫𝑺𝑴 =𝝈
𝒏=𝟐. 𝟎𝟖
𝟏𝟐= 𝟎. 𝟔 𝒌𝒈/𝒎𝟑
2. Temperature effect uncertainty (𝑈𝑇) _ Type B
Thermometer uncertainty -> ± 0.2°C
thus
0.5806 𝒌𝒈/𝒎𝟑/ °C x 0.2 °C = ± 0.12 𝒌𝒈/𝒎𝟑
𝑼𝑻 =𝟎. 𝟏𝟐
𝟑= 𝟏. 𝟕𝟑𝒌𝒈/𝒎𝟑
Note: This is
only an exerpt
from the full
analysis for
illustration
purposes
y = -0.5806x + 2686.4R² = 0.998
2669
2670
2671
2672
2673
2674
20 22 24 26 28 30
MVD
Temperature
Change in MVD with Variation in temperature
Measurement Uncertainty Example
MVD uncertainty analysis:
3. Pressure effect (𝑈𝑃𝑟) _ Type A
(Note: can also have a type B component depending on equipment setup)
Method specifies 30mm Hg vacuum pressure, variation checked at 20, 25 a& 30mm Hg:
𝑈𝑃𝑟 =𝝈
𝒏=
𝟑.𝟐𝟐𝟏
𝟐𝟏= 𝟎. 𝟕𝟎 𝒌𝒈/𝒎𝟑
Note: This is
only an exerpt
from the full
analysis for
illustration
purposes
Measurement Uncertainty Example
MVD uncertainty analysis:
MVD combined uncertainty (𝑈𝑀𝑉𝐷𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑)
The MVD of the specimens tested for this particular sample is therefore:
2672 ± 2 𝒌𝒈/𝒎𝟑
𝑢𝑐 =
𝑖=1
𝑛
𝑢𝑖2
𝑢𝑐 = 𝑢𝑠2 + 𝑢𝑇
2 + 𝑢ℎ𝑡2
𝑢𝑐 = 0.62 + 1.732 + 0.702
𝒖𝒄 = 𝟏. 𝟗𝟔 𝒌𝒈/𝒎𝟑 ≅ 𝟐 𝒌𝒈/𝒎𝟑Note: This is
only an exerpt
from the full
analysis for
illustration
purposes
Combined uncertainty of both MVD and BD testing results in:
Measurement Uncertainty Example BD
Error
MVD
Error
Voids
Error
24
𝑉𝑜𝑖𝑑𝑠 =𝑀𝑉𝐷 −𝐵𝐷
𝑀𝑉𝐷𝑥 100 =
2672 −2572
2672𝑥 100 = 3.7%
𝑈𝐵𝐷 = 3 𝑘𝑔/𝑚3 𝑈𝑀𝑉𝐷 = 2 𝑘𝑔/𝑚3
𝑅𝑒𝑠𝑢𝑙𝑡𝐵𝐷 = 2572 𝑘𝑔/𝑚3 𝑅𝑒𝑠𝑢𝑙𝑡𝑀𝑉𝐷 = 2672 𝑘𝑔/𝑚3
Resultant voids:
But …
What about the uncertainty?????????????
Combined uncertainty of both MVD and BD testing results in:
Measurement Uncertainty Example
25
𝑽𝒐𝒊𝒅𝒔 =𝑀𝑉𝐷 −𝐵𝐷
𝑀𝑉𝐷𝑥 100 =
2672 −2572
2672𝑥 100 = 𝟑. 𝟕%± 𝟎. 𝟐%
𝑈𝐵𝐷 = 3 𝑘𝑔/𝑚3 𝑈𝑀𝑉𝐷 = 2 𝑘𝑔/𝑚3
𝑅𝑒𝑠𝑢𝑙𝑡𝐵𝐷 = 2572 𝑘𝑔/𝑚3 𝑅𝑒𝑠𝑢𝑙𝑡𝑀𝑉𝐷 = 2672 𝑘𝑔/𝑚3
Due to the uncertainty of both MVD and BD:
BD
Error
MVD
Error
Voids
Error
Summary
Determining and communicating the degree of uncertainty of
measurement that your results have, give validity to your results no
matter how insignificant it might seem;
Going through the process of determining your uncertainty, gives you the
tools to narrow down on sources of error in your testing environment and
assists in addressing/eliminating them;
Always check the validity of your results within the context of your UoM
range. Repeat results that fall outside of UoM range, are invalid;
It is not necessary to determine uncertainty of measurement of every
single aspect, if there is suitable evidence as to how the factor is
addressed and maintained constant.
26
Q A
27