uncertainty analysis for flow measurements and techniques using standardized methodology marian...
TRANSCRIPT
Uncertainty Analysis for
Flow Measurements and Techniquesusing
Standardized Methodology
Marian MusteMarian Muste11
Juan Gonzalez-CastroJuan Gonzalez-Castro22
Dongsu KimDongsu Kim11
Kwonkyu YuKwonkyu Yu11
1 IIHR- Hydroscience & Engineering, The University of Iowa2 South-Florida Water Management District, West Palm Beach
Overview
BackgroundBackground Uncertainty Analysis (UA) FrameworksUncertainty Analysis (UA) Frameworks
AIAA (1995)AIAA (1995)
UA Implementation ExampleUA Implementation Example MethodologyMethodology Assessment of Elemental UncertaintiesAssessment of Elemental Uncertainties Customized GUI for UA ImplementationCustomized GUI for UA Implementation
Conclusions Conclusions OutlookOutlook
Background
SAMPLE REQUEST regarding uncertainty analysis originated from a Hydrologic Service
…. Has anybody out there had to defend the validity of an ADCP flow measurement against a legal challenge from a third party?
……When current meters were used to undertake these measurements we could claim that the flow measurement was undertaken in conformance with British
and International standards for current meter gauging and that the current meter had a valid calibration certificate…
In the case where flow measurements are now taken using ADCPs we feel more vulnerable to legal challenges. This is for two reasons:
1. There is no ISO document in place. The Agency has to rely on its own internal document on gauging procedures which is based on the draft ISO document.
2. ADCPs do not have "certificates of calibration" . The only checks on the performance that can be made are against other ADCPs or other types of flow
monitoring equipment.
(posted on the USGS’ Hydro-Acoustics Work Group webpage by R. Iredale, The Environment Agency of England and Wales, 2005)
Over the last 50 years, considerable efforts have been made by professional societies to develop and implement uncertainty
analysis (UA). One of the rigorous UA methodology (based on sound statistical and
engineering concepts):
Guide to the Expression of Uncertainty Measurement (ISO, 1993) - adopted widely by various scientific & research communities, e.g., NIST (1994), NF
ENV 13005 (1999)- the guide recognizes the need for further adaptation for specific areas
Specific adaptations for engineering:- Assessment of Wind Tunnel Data Uncertainty (AIAA, 1995)- Test Uncertainty (ASME, 1998)
Key assumptions/concepts for ISO (1993)-based standards - Gaussian pdf-s for the error sources - 2 sample standard deviations for 95% confidence level - for large samples (N ≥ 10), special procedures for handling small samples - RSS used for combining uncertainties - Taylor-series expansion for propagation of uncertainties
- total uncertainties expressed using confidence intervals
UA Frameworks
Terminology for ISO (1993) - based standards
UA Frameworks
The 3 standards provide the same total measurement uncertainty
ISO (1993) AIAA-S-071-1995 ASME PTC 19.1-1998 Uncertainty of a measurement
input quantity type A standard uncertainty type B standard uncertainty combined standard uncertainty
individual variable bias limit precision limit (differently estimated
for single and multiple tests) total uncertainty
independent parameter systematic uncertainty random uncertainty (differently estimated single and multiple tests)
measurement uncertainty
Uncertainty of a result functional relationship sensitivity coefficients combined standard uncertainty (accounts for correlated errors)
expanded uncertainty (accounts for the level of confidence)
coverage factor f( degrees of freedom and t Distribution)
data reduction equation
bias limit precision limit (differently estimated
for single test with single readings and averaged readings, multiple tests)
sensitivity coefficients combined standard uncertainty
(accounts for correlated errors) uncertainty at specified confidence
level coverage factor f( degrees of freedom
and t Distribution)
derived result systematic uncertainty random standard deviation
(differently estimated for single and multiple tests)
sensitivity coefficients uncertainty of the result (accounts
for correlated errors and the level of confidence)
coverage factor f(degrees of
freedom and t Distribution)
= to tal e rro r = b ias e rro r = p recis ion error
M A G N ITU D E O F X
FR
EQ
UE
NC
Y O
F O
CC
UR
RE
NC
E
X
X
X
tru e
X tru e
k
k+ 1 k
k+ 1
k+ 1XX k
(a ) tw o re a d in g s
(b ) in f in ite n u m b e r o f re a d in g s
Bias error (): fixed, systematic
Bias limit (B): estimate of
Precision error (): random
Precision limit (P): estimate of
Total error:
Engineering approach, simple, clear, widely applied
AIAA (1995)
Determine the data reduction equation
JXXXrr ,,, 21
Assess relative significance of uncertaintysources (order of magnitude estimates)
For the experimental result r, determine theprecision and bias limits and overall uncertainty
Considering the significant sources, estimate theprecision and bias limits for each iX
Identify sources of uncertainty for each iX
Implementation Sequence
Key feature: data-reduction equation
r = r(X1, X2, X3,…, Xj)
AIAA (1995)
r = r (X , X ,......, X ) 1 2 J
1 2 J
M EASUREM ENTOF INDIVIDUALVARIABLES
INDIVIDUALM EASUREM ENTSYSTEMS
ELEM ENTALERROR SOUR CES
DATA REDUCTIONEQUATIO N
EXPERIM EN TALRESULT
XB , P
1
1 1
XB , P
2
2 2
XB , P
J
J J
rB , P
r r
Implementation Aspects
Measurement systems for each individual variable Xi : instrument, data acquisition and reduction procedures, operational environment (laboratory, in situ), the flow and its interaction with the instrument and the environment
Estimates of errors are meaningful only when considered in the context of the process leading to the value of the quantity under consideration
Uncertainties estimated following the signal propagation from sensor to the final result
Uncertainties estimated with a pre-established confidence level (95% for most engineering areas)
UA differently conducted dependent on the type of experiment:
Single test (for complex or expensive experiments): one set of measurements (X1, X2, …, Xj) for r
Multiple tests (ideal situations): many sets of measurements (X1, X2, …, Xj) for r at a fixed test condition with the same measurement system
AIAA (1995)
MULTIPLE TESTS (recommended)
Given a data reduction equation for a measurement ),,,( 21 JXXXrr
ii X
r
;M
tSP r
r
2/1 P + B = U 2r
2rr
The result and its uncertainty is
and the precision limit of the result is
where the bias limit of the result is
The uncertainty in the final result
rUr
M
kkrM
r1
1with
;
1
2/1
1
2
M
k
kr M
rrS 102 Mfort
J
i
J
i
J
ikikkiiir BBB
1
1
1 1
222 2
AIAA (1995)
SINGLE TEST
Given a data reduction equation for a measurement
ii X
r
2/1 P + B = U 2r
2rr
The result and its uncertainty is
and the precision limit of the result is
where the bias limit of the result is
The uncertainty in the final result
2/1
1
2
1
M
k
kr M
rrS
),,,,( 321 jXXXXrr
rUr
J
i
J
i
J
ikikkiiir BBB
1
1
1 1
222 2
Based on prior informationrr tSP
AIAA (1995)
AIAA (1995)
Implementation Aspects
sound engineering judgment to optimize the output with minimum costs, e.g.: use of end-to-end uncertainty estimation approach uncertainty sources < 1/4 or 1/5 of the largest sources are usually considered negligible
specific procedures for single and multiple measurements
specific procedures for dealing with small statistical samples
methodology for assessment of calibration uncertainties
methodology for data validation
Implementation Aspects
Integration of UA in all phases of the measurement
AIAA (1995) D E F IN E P U R P O S E O F T E S T A N D
R E S U LT S U N C E R TA IN T Y R E Q U IR E M E N T S
U N C E R TA IN T YA C C E P TA B L E ?
IM P R O V E M E N TP O S S IB L E ?
D E T E R M IN E E R R O R S O U R C E SA F F E C T IN G R E S U LT S
Y E SN O
N O
Y E S Y E S
Y E S
N O
S E LE C T U N C E R TA IN T Y M E T H O D
E S T IM AT E E F F E C T O FT H E E R R O R S O N R E S U LT S
- M O D E L C O N F IG U R AT IO N S (S )- T E S T T E C H N IQ U E (S )- M E A S U R E M E N T S R E Q U IR E D- S P E C IF IC IN S T R U M E N TAT IO N- C O R R E C T IO N S T O B E A P P L IE D
- D E S IR E D PA R A M E T E R S (C , C , ... .)D R
D E S IG N T H E T E S T
- R E F E R E N C E C O N D IT IO N- P R E C IS IO N L IM IT- B IA S L IM IT- T O TA L U N C E R TA IN T Y
D O C U M E N T R E S U LT S
N O T E S T
C O N T IN U E T E S T
IM P L E M E N T T E S T
S O LV E P R O B LE M
R E S U LT SA C C E P TA B L E ?
M E A SU R E-M E N T
S YS T E MP RO BLE M ?
N O
P U R P O S EA C H IE V E D ?
Y E S
N O
S TA R T T E S T
E S T IM AT EA C T U A L D ATAU N C E R TA IN T Y
IMPLEMENTATION EXAMPLE
Extensively used in laboratory measurements and field conditions,
from simple (Pitot tube) to complex (LDV) instruments
Widely applied for teaching and research purposes
Successful implementation to discharge measurements: conventional instruments (Muste et al. 2007)
contemporary, nonintrusive techniques: Large-Scale Particle Image Velocimetry (Y-S. Kim et al, 2007)
Acoustic-Doppler Current Profilers (Gonzalez-Castro & Muste, 2007)
AIAA (1995)
Currently, ADCPs are the most efficient instrument for riverine environment characterization (monitoring and research needs)
If properly operated, the instrument can accurately document discharges, mean velocities, and selected turbulence characteristics
Despite their extensive use, there are aspects regarding their capabilities, operation, and uncertainty analysis not documented yet
ADCP UA: Implementation
ADCP Uncertainty Analysis (UA) status
Past efforts (non-standardized methodologies)
Discharge: Simpson & Oltman (1992), Gordon (1993), Lipscomb (1995), Morlock (1996), Simpson (2001), Gartner (2002), Muller (2002), Yorke & Oberg (2002), USGS-RDI (2005)
Turbulence measurements: Droz (1998), Stacey (1999), Nystrom (2002), Schemper & Admiraal (2002)
On-going efforts (standardized methodology) UA formulated within the framework of authoritative
engineering standards
ADCP UA: Implementation
Discharge Measurement with ADCP mounted on a boat
Measurable Area
Unmeasurable Near-bank Areas
Unmeasurable Top Area
Unmeasurable Bottom Area
etQ
mQ
elQerQ
ebQ
erelebetemmt QQQQQQQ
;lllel ZLKVQ rrrer ZLKVQ where
ADCP UA: Implementation
Error identification
ADCP UA: Elemental Uncertainty Assessment
Source Biases Estimation of Accounted in Reduction Equations through 1
Depends upon Can be estimated from
e1: Spatial resolution Water and boat velocities, depths † ADCP, mode, settings, boat speed End-to-end calibration 2
e2: Doppler noise Water and boat velocities Bva, Bvb ADCP frequency, mode, settings,
speed of sound, gating time UA of signal processing algorithms,
instrument intercomparison
e3 : Velocity ambiguity Water and boat velocities † Mode, settings End-to-end calibration
e4 : Side-lobe interference Discharge through unmeasured areas * Beam angle, settings, bathymetry End-to-end calibration
e5: Temporal resolution High frequency velocity components † Settings End-to-end calibration
e6: Sound speed Water and boat velocities, depths BC Water properties UA of C(Salinity, Temperature) with data
from reference meter
e7 : Beam angle Water and boat velocities, depths B ADCP Manufacturer’s specifications
e8 : Boat speed Water and boat velocities, depths † Site, flow, boat operation End-to-end calibration
e9: Sampling time Long-term means †
Frequency of large-scale flow structures 3
Instrument intercomparison based on long data records under steady conditions
e10 : Near-transducer Velocities near the ADCP Bnt ADCP, draft, settings, velocity, flow
depth Experimental Measurements and CFD
Modeling
e11: Reference boat velocity Water and boat velocities, depths Bvb Sediment concentration, flow 4 Manufacturer’s Specifications
e12: Depth Discharge through unmeasured areas BDa, BDp, BDp, BDo, BDavg 5 ADCP, settings, draft, bathymetry,
water properties, time gating
UA of depths as f(C and gating time) and BC, Bt and BDADCP and concurrent depth
range measurements
e13: Cell positioning Measured and unmeasured discharge Bt , BDa, BDp, BDo, BDavg ADCP, setting, water properties
e14: Rotation Water and boat velocities, depths and geographic orientation
Bp , Br , Bh ADCP, setup, site Manufacturer’s Specifications
e15: Timing Distances by gating and discharge Bt ADCP, speed of sound, gating time Manufacturer’s Specifications
e16: Edge Discharges through channel edges B ,BL ADCP settings, bathymetry, cross
section, edge distances Manufacturer’s Specifications
e17: Vertical profile model Discharge through unmeasured top and bottom areas
BQ1 6 Velocity distribution model, turbulence
intensity Field and Laboratory Experiments with
reliable CFD-LES Modeling
e18: Discharge model Discharge through measured area BQ2 6 Discharge model
Highly resolved data / End-to-end calibration
e19: Finite summation Discharge through measured area BQ3 6 ADCP settings, boat velocity
e20: Site conditions & operation Total discharge † Site, boat operation Concurrently measured data
fVADCP9
Data Reduction Equations (Teledyne/RDI’s ADCP)
fff vuV ,
bbb vuV ,
x
y
E
dtdzkVVQT tz
tz
bfm
U
L
0
)(
)(
ADCP UA: Implementation
Exact approach – discharge in the direct measured area
Using BT
1
1 1,134212134
11
sin2
sN
i
n
jjibbbb
iijj
m vvvvvvvvttzz
Q
jiQ ,
unit area for jiQ ,
i 1isNi 1i
n
j1j
ADCP UA: Implementation
Exact approach: in-bin discharge Using BT
zthprFFFFCFFFFFfQ abDbDbDbDSDDDDm ji ,,,,,,,,,,,,,,, 11114321,
If , the discharge is a functional relationship of the form:
WATER VELOCITY WITH RESPECT TO ADCP
34432121 cscsin2seccossin+csccos24
1vvPvvvvPRvvRua
432134 secsin-csccos24
1vvvvPvvPva
BOAT VELOCITY WITH RESPECT TO CHANNEL BED
34432121 cscsin2seccossin+csccos24
1bbbbbbbbb vvPvvvvPRvvRu
432134 secsin-csccos24
1bbbbbbb vvvvPvvPv
ACTUAL WATER VELOCITY
bababaf vvuuVVV ,
DEPTHS
BADCPtop DDD ;
20 ap
bB
DDDDD
for Mode 1;
2ap
bB
DDDD
for Mode 5
2
cos 0max
DDDDD p
ADCPavgLG
for Mode 1;
2cosmax
pADCPavgLG
DDDD for Modes 5, 8 & 11
ADCPavgtotal DDD cos ; 21
aLGtotal
DDDZ ;
22a
toptotal
DDDZ ; totalDZ 3
PARTIAL DISCHARGES
11
11
1
12
1
bb
m
jjabbaii
ba
t ZZ
vuvuttZD
Qi
11
11
11
12
23
bb
m
jjabbaii
bba
b ZZ
vuvuttZZD
Qi
lllel ZLCVQ ; rrrer ZLCVQ
TOTAL DISCHARGE
erelem
N
ibt
N
i
m
jmtotal QQQQQQQ
s
ii
s
ji
1
1
1
1 1,
iijjjiabbam ttzzvuvuQji
11,1,
SD F
CFv
2
ADCP UA: Implementation
Exact approach – top and bottom discharges (extrapolation)
BTM Q
MID Q
TOP Q 3 Z
1 Z
Z
DEPTH CELL Da
ADCP TRANSDUCER FACE
D total
POWER FIT
SCALAR TRIPLE (m2/s2) PRODUCT
ADCP
MEASURED DISCHARGE
TOP LAYER (ESTIMATED)
BOTTOM LAYER (ESTIMATED)
DISCHARGE (m3/s)
ACTUAL PROFILE
ADCP VELOCITIES
D
2 Z CONSTANT
POWER 3-POINT SLOPE
POWER
CONSTANT POWER IN LOW 0.2 D total
D avg
D ADCP
D top B D
D LG
11
11
11
12
23
bb
m
jjabbaii
bba
b ZZ
vuvuttZZD
Qi
11
11
1
12
1
bb
m
jjabbaii
ba
t ZZ
vuvuttZD
Qi
ADCP UA: Implementation
Uncertainty Propagation to Final Result: Bias Limit2222222
erelebetemmt QQQQQQQ BBBBBBB
i
N
i
n
jj
rvv
jar
m
rja
m
n
j
j
ti
m
ti
m
zj
m
zj
m
rv
br
m
v
ra
mm
r
m
p
m
Q
s
jrarja
jiji
i
ji
i
ji
i
ji
i
ji
rb
ji
ra
jijijiji
m
BBv
Q
v
Q
Bt
QB
t
QB
z
QB
z
Q
Bv
QB
v
QB
QB
r
QB
p
Q
B
1
1
1
1
4
1 )1(
12
2
2
2
1
2
2
2
2
1
4
1
2
2
2
2
2
2
2
2
2
2
2
)1(
,,
,
1
,,
1
,
,,,,,
15.0
i
N
i
n
jj
rvv
jar
et
rja
ett
i
ett
i
et
Db
etD
avg
etD
o
etD
p
etD
a
et
et
j
n
j tv
bt
etv
ta
etr
etp
et
Q
s
jrarja
ii
i
i
i
i
b
i
avg
i
o
i
p
i
a
i
i
tb
i
ta
iii
et
BBv
Q
v
QB
t
QB
t
Q
BD
QB
D
QB
D
QB
D
QB
D
Q
BQ
Bv
QB
v
QB
r
QB
p
Q
B
1
1
1
1
4
1 )1(
2
2
2
2
1
2
2
2
2
2
2
2
2
2
2
2
2
1
4
1
2
2
2
2
2
2
2
2
2
)1(115.0
2
2
2
2
2
2
2
2
2
2
2
BQ
BD
QB
D
QB
L
QB
V
QB el
D
ADCP
elD
avg
elL
e
elV
el
elQ ADCPavgeelet
2
2
2
2
2
2
2
2
2
2
2
BQ
BD
QB
D
QB
L
QB
V
QB er
D
ADCP
erD
avg
erL
er
erV
er
erQ ADCPravg
r
ererer
ADCP UA: Implementation
M
tSP t
t
Q
Q
22
ttt QQQ PBU
Uncertainty Propagation to Final Result: Precision Limit
Uncertainty Propagation to Final Result: Total Uncertainty
ADCP UA: Implementation
bV
= beam angle, = angle of the flow to instrumentβ = angle of the boat velocity = in beam water velocities = boat velocity
where
41 ~ VV
Practical approach(pitch and roll neglected in DRE; errors accounted through end-to-end calibrations)
velocity (instrument coordinates neglecting the pitch and roll angle)
erel
N
i
n
jiijjijiibjift QQttzzVVQ
s
1
1 1111,11,,1 )sin(
total discharge
2
1
221
2432
2
221
243 )cos(
sin2
)()(2
sin4
)()(,1
bbf VVVVV
VVVVV
Vji
ADCP UA: Implementation
• Software Configuration
ADCPOutput Binary Files
ADCPOutput ASCII Files
Statistical Analysis for UA
Processing Database - Vertical, Horizontal Velocity Profile - Discharge (WinRiver & WinADCP Homologuous) - Visualizations
Prior Information ( ) - Calibrations - Manufacturer specifications - Literature compilation - prior measurements - User UA archive
Error Propagation to Final Results (Embedded in Uncertainty Analysis GUI)
- Velocity Uncertainty Analysis - Discharge Uncertain Analysis
Uncertainty Analysis Graphical User Interfaces
Graphical DisplayNumerical Display
12
34
5
Uncertainty Analysis Output
ADCP Uncertainty Analysis & GUI Flow Chart
Configuration File - operational and environmental specifications
New Measurements191 ~ uu
ADCP UA Software - architecture• Developing tools - Borland C++ Builder (v.6) & Microsoft Access
Archive database
- Elemental uncertainties are archived in categories based on river characteristics. - Users with limited level of preparedness can estimate uncertainties using default values obtained in similar environment and operating conditions. - The stored information is updated as soon as new measurements are processed. - User can also create new archives using new classification categories
ADCP UA Software - GUIs
Information for archiving
ADCP UA Software - GUIs
Discharge
River Characteristics
Vessel Moving Path
Flow Direction
Channel Profile
Assessment of bias limit
ADCP UA Software - GUIs
Individual Error Source Input
River Characteristics
Default Values based on Archive Database
Relevant Literature for error sources
1
2
3
4
Assessment of precision limit
ADCP UA Software - GUIs
Repeated Measurements for a transect
Result
Variation of Discharge Measurements
Assessment of total uncertainty
ADCP UA Software & GUIs
Calculated Uncertainty in Discharge Measurement
Feasibility of UA engineering standards for implementation to ADCP measurements
The methodology is comprehensive, simple to implement Easily upgradeable as new info occur UA allows tracing of the measurement accuracy to primary
standards withstand legal and strict QA/QC requirements Finalization of UA – an extensive and expensive effort
Collaboration between manufacturers and users in a coordinated effort = key to complete UA for the variety of measurement situations and operating conditions encountered in monitoring practice
The framework was adopted by ASCE’s HME Task Committee and the UNESCO group on Data Requirements for Integrated Urban Water Management (Fletcher et al., 2007)
Currently evaluated by the ISO committee (Herschy)
Conclusions
Conclusions
The UA customized software for ADCP velocity and discharge measurements requires minimum user preparation
Autoarchiving uncertainties for specific environments and operating conditions can provide information about dominant sources of uncertainties at various sites.
By continuously increasing the sample size through archiving, the UA output is progressively enhanced.
Outlook
Work closely with manufacturers and users to assess elemental error sources (manufacturer, operator, environment, or combinations) and integrate them in the AIAA (1995) uncertainty assessment framework for rigorous documenting velocity and discharge measurement accuracy
Conduct sensitivity analysis and field tests for compiling uncertainty minimizations guidelines
Develop operational guidelines for conducting accurate measurements in various flow regimes
Outlook
Need for coordination and extensive collaboration among ADCP manufacturer, operators, data users, and third-party evaluators
Need for evaluation of the status of current developments and to strategize for integrative efforts to assess methodologies for operation and accuracy assessment of the ADCP as well as other flow measurement techniques over an extend the range of flow conditions (present WMO effort)
IIHR is willing to be actively involved in the WMO initiative
Thank you!
Questions?