variance vs entropy base sensitivity indices julius harry sumihar
TRANSCRIPT
Variance vs Entropy BaseSensitivity Indices
Julius Harry Sumihar
Outline
• Background
• Variance-based Sensitivity Index
• Entropy-based Sensitivity Index
• Estimates from Samples
• Results
• Conclusions
Background
• Applications of computational models to complex real situations are often subject to uncertainty
• The aim of sensitivity analysis is to quantitatively express the degree of impact of the uncertainty from the specific sources on the resulting uncertainty of final model output
Variance Base Sensitivity Index
• Result from the principle of “expected reduction in variance”
• This principle leads to the expression:
• Interpreted as “the amount of variance of output Y that is expected to be removed if the true value of parameter Xi will become known”
varY – E[var(Y|Xi)]
• Main characteristic: it considers the variance of a probability distribution as an overall scalar measure of the uncertainty represented by this distribution
• Intuitively, over a bounded interval, the highest possible degree of uncertainty is expressed by the uniform distribution
• A scalar measure of uncertainty should attain its maximum value for uniform distribution
• Inconsistency: This is not the case for variance
p=1/3 p=1/3
p=1/6 p=1/6
0 1/3 2/3 1
p=1/4 p=1/4p=1/4 p=1/4
0 1/3 2/3 1
Var(X) = 19/108 Var(X) = 15/108
H(X) = 1.32966 H(X) = 1.38629
• Entropy: an overall scalar uncertainty measure maximized by the uniform distribution
• ‘a measure of the total uncertainty of Y coming from all parameters’
dyyfyfYH ))(ln()()(
Entropy Base Sensitivity Index*
*Bernard Krzykacz-Hausmann,”Epistemic Sensitivity Analysis Based On The Concept Of Entropy”
• ‘a measure of uncertainty of Y coming from the other parameters if the value of parameter X is known to be x’:
• ‘expected uncertainty of Y if the true value of parameter X will become known’:
dyxyfxyfxXYH ))|(ln()|()|(
dydxxfxyfxyfXYH )())|(ln()|()|(
• ‘the amount of entropy of output Y that is expected to be removed if the true value of parameter X will become known’:
• By some manipulations:
)|()( XYHYH
dxdyyfxf
yxfyxfXYHYH
)()(
),(ln),()|()(
b1 b2 b3 bjmaxbj-1 bj
a1
a2
ai-1
ai
aimax
Y
X
Estimates From Samples
i
aa xi
ai
ani
nxf
ii)(1
1
1
..
.)( ),[ 1
j
bb yj
bj
bnj
nyf
jj)(1
1
1
..
.)( ),[ 1
ji
bbaa yxj
bj
bi
ai
anij
nyxf
jjii
,
),[),[ )(1)(11
1
1
1
..),(
11
ji ji
ij
ij
nn
nn
nn
n
nXYHYH
,
..
.
..
.
..
..
]))((
ln[)|()(
2
..
.
2
...
11X)]|E[var(Y– varY
j
jj
i j
ijj
i n
nyny
nn
Entropy Base:
Variance Base:
Results• Model:
o Y = U1 + U2
o Y = U1 + 2U2
o Y = N1 + N2
o Y = N1 + 2N2
U1,U2 ~ U[0,1], N1,N2 ~ N(0.5, 0.3)
• Number of samples: 1,000 and 10,000 (@10 times)
• Grid Size: 0.025, 0.05, 0.1, 0.2
Model: Y = U1 + 2U2
0
0,02
0,04
0,06
0,08
0,1
0,12
0.025 0.05 0.1 0.2
Grid Size
var(
E[Y
|U1]
)
Analytical
1,000 samples
10,000 samples
Model: Y = U1 + 2U2
00,20,40,60,8
11,21,41,61,8
0.025 0.05 0.1 0.2
Grid Size
H(Y
)-H
(Y|U
1)
analytical
1,000 samples
10,000 samples
• 10,000 samples is better than 1,000 samples
• use 10,000 samples from now on
Effect of Sample Number
Model Xi 0.025 0.05 0.1 0.2 Analytical
Y = U1 + U2
U1 0,5629084 0,4847560 0,4381850 0,3740526 0,5
U2 0,5618695 0,4838189 0,4378267 0,3759858 0,5
Y = U1 + 2U2
U1 0,4104407 0,2723420 0,2263539 0,1890477 0.25
U2 0,9948456 0,9062422 0,83651837 0,7288735 0.943147
Y = N1 + N2
N1 0,5493945 0,4147645 0,35574700 0,3236304 0,346573
N2 0,5524757 0,4150046 0,35787368 0,3255338 0,346573
Y = N1 + 2N2
N1 0,5112249 0,2500562 0,15475538 0,1155110 0.111572
N2 0,9992094 0,8611518 0,79986998 0,7280601 0.804719
H(Y)-H(Y|Xi)
Effect of Grid Size
Model Xi 0.025 0.05 0.1 0.2 Analytical
Y = U1 + U2
U10,08324767 0,0829774 0,0822966 0,0796876 0,083333
U20,08371050 0,0833495 0,0825837 0,0802683 0,083333
Y = U1 + 2U2
U10,08395208 0,0832341 0,0823905 0,0794308 0,083333
U20,33438259 0,3335535 0,3309643 0,3212748 0,333333
Y = N1 + N2
N10,08959163 0,0892577 0,0884411 0,0856991 0,09
N20,08977242 0,0894751 0,0887430 0,0860685 0,09
Y = N1 + 2N2
N10,09116694 0,0901768 0,0886759 0,0859255 0,09
N20,35729517 0,3565734 0,3537290 0,3437929 0,36
Var(Y)-E[Var(Y|Xi)]
Model: Y = U1 + U2
0
0,1
0,2
0,3
0,4
0,5
0,6
0,025 0,05 0,1 0,2
Grid Size
H(Y
)-H
(Y|X
i)
Analytical
Numerical U1
Numerical U2
Model: Y = N1 + N2
0
0,1
0,2
0,3
0,4
0,5
0,6
0,025 0,05 0,1 0,2
Grid Size
H(Y
)-H
(Y|X
i)
Analytical
Numerical N1
Numerical N2
Model: Y = U1 + U2
0,01
0,03
0,05
0,07
0,09
0,025 0,05 0,1 0,2
Grid Size
var(
E[Y
|Xi]
)
Analytical
Numerical U1
Numerical U2
Model: Y = N1 + N2
0,01
0,03
0,05
0,07
0,09
0,025 0,05 0,1 0,2
Grid Sizeva
r(E
[Y|X
i])
Analytical
Numerical N1
Numerical N2
• Entropy base is very sensitive to grid size
• No rule exist for choosing grid size
Model XiH(Y)-H(Y|Xi) Analytical var(E[Y|Xi]) Analytical
Y = U1 + U2
U1 0.484756 0.5 0.083248 0.083333
U2 0.483819 0.5 0.083350 0.083333
Y = U1 + 2U2
U1 0.272342 0.25 0.083234 0.083333
U2 0.906242 0.943147 0.333553 0.333333
Y = N1 + N2
N1 0.355747 0.346573 0.089591 0.09
N2 0.357874 0.346573 0.089772 0.09
Y = N1 + 2N2
N1 0.115511 0.111572 0.090177 0.09
N2 0.799870 0.804719 0.357295 0.36
Best Estimates
Model: Y = U1 + U2
0,04
0,05
0,06
0,07
0,08
0,09
0,1
1 2 3 4 5 6 7 8 9 10
Sample
var(
E[Y
|Xi]
)
U1
Average U1
U2
Average U2
Analytical
Model Y = U1 + U2
0,45
0,46
0,47
0,48
0,49
0,5
0,51
1 2 3 4 5 6 7 8 9 10
Sample
H(Y
)-H
(Y|X
i)
U1
U2
Average U1
Average U2
Analytical
Model: Y = U1 + 2U2
0,23
0,24
0,25
0,26
0,27
0,28
0,29
1 2 3 4 5 6 7 8 9 10
Sample
H(Y
)-H
(Y|U
1)
U1
Average
Analytical
Model: Y = U1 + 2U2
0,04
0,05
0,06
0,07
0,08
0,09
0,1
1 2 3 4 5 6 7 8 9 10
Sample
var(
E[Y
|U1]
)
U1
Average
Analytical
Model: Y = U1 + 2U2
0,88
0,89
0,9
0,91
0,92
0,93
0,94
0,95
1 2 3 4 5 6 7 8 9 10
Sample
H(Y
)-H
(Y|U
2)
U2
Average
Analytical
Model: Y = U1 + 2U2
0,31
0,32
0,33
0,34
0,35
0,36
0,37
1 2 3 4 5 6 7 8 9 10
Sample
var(
E[Y
|U2]
)
U2
Average
Analytical
H(Y)-H(Y|Xi) Var(Y)-E[Var(Y|Xi)]
Model: Y = N1 + N2
0,3
0,32
0,34
0,36
0,38
1 2 3 4 5 6 7 8 9 10
Sample
H(Y
)-H
(Y|X
i)
U1
U2
Average U1
Average U2
Analytical
Model: Y = N1 + N2
0,05
0,06
0,07
0,08
0,09
0,1
1 2 3 4 5 6 7 8 9 10
Sample
var(
E[Y
|Xi]
)
N1
Average N1
N2
Average N2
Analytical
Model: Y = N1 + 2N2
0,08
0,09
0,1
0,11
0,12
0,13
0,14
1 2 3 4 5 6 7 8 9 10
Sample
H(Y
)-H
(Y|N
1)
N1
Average
Analytical
Model: Y = N1 + 2N2
0,06
0,07
0,08
0,09
0,1
0,11
0,12
1 2 3 4 5 6 7 8 9 10
Sample
var(
E[Y
|N1]
)
N1
Average
Analytical
Model: Y = N1 + 2N2
0,75
0,77
0,79
0,81
0,83
0,85
1 2 3 4 5 6 7 8 9 10
Sample
H(Y
)-H
(Y|N
2)
N2
Average
Analytical
Model: Y = N1 + 2N2
0,3
0,32
0,34
0,36
0,38
0,4
1 2 3 4 5 6 7 8 9 10
Sample
var(
E[Y
|N2]
)
N2
Average
Analytical
Var(Y)-E[Var(Y|Xi)]H(Y)-H(Y|Xi)
Conclusions
• Entropy-based sensitivity index is difficult to estimate
• Variance-based sensitivity index is better than the Entropy-based one
dxxyfxfyf
UXXXXY
xxy )()()(
]1,0[~,
21
2121
otherwise
xyxxyf
otherwise
xxf
x
x
,0
1,1)(
,0
10,1)(
2
1
otherwise
yydxxyfxf
yydxxyfxf
yfy
xx
y
xx
y
,0
21,2)()(
10,)()(
)(
1
1
0
21
21
Model: Y = U1 + U2
otherwise
xyxyf
xxUxXY
UXXXXY
xy ,0
1,1)(
]1,[~|
]1,0[~,
|
1
2121
01)1ln(1)|(
1
0
1
1 x
x
dydxXYH
2
1)2ln()2()ln()(
1
0
2
1 dyyydyyyYH
2
1)|()( 1 XYHYH
Model: Y = U1 + U2
)())|((
))|(()]|([)(22 YEXYEE
XYEVarXYVarEYVar
1
0
2
1
2 1)2()()( dyyydyydyyyfYE
xdyydyyfyxXYE
x
x
xy
2
11)()|(
1
|
12
1)]|([)( XYVarEYVar
12
131)
2
1()()|())|((
1
0
222 dxxdxxfxXYEXYEE x
Model: Y = U1 + U2
]2,0[~]1,0[~,2 3213121 UXUXXXXXXY
dxxyfxfyf xxy )()()(31
otherwise
yxyxyf
otherwise
xxf
x
x
,0
2,2
1)(
,0
10,1)(
3
1
Model: Y = U1 + 2U2
otherwise
yydxxyfxf
ydxxyfxf
yydxxyfxf
yf
y
xx
xx
y
xx
y
,0
32,2
1
2
3)()(
21,2
1)()(
10,2
1)()(
)(1
2
1
0
0
31
31
31
Model: Y = U1 + 2U2
otherwise
xyxyf
xxUxXY
UXXXXY
xy
,0
2,2
1)(
]2,[~|
]1,0[~,2
1|
1
2121
)2ln(1)2
1ln(
2
1)|(
1
0
2
1 x
x
dydxXYH
9431471807.0
)22
3ln()
22
3()
2
1ln(
2
1)
2ln(
2)(
1
0
2
1
3
2
dyyy
dydyyy
YH
25.0)|()( 1 XYHYH
Model: Y = U1 + 2U2
1
0
2
1
3
2
22
2
3)
2
1
2
3(
2
1
2
1)()( dyyyydydyydyyyfYE
xdyydyyfyxXYE
x
x
xy
12
1)()|(
2
|1 1
12
1
2
3
3
7)]|([)(
2
1
XYVarEYVar
3
71)1()()|())|((
1
0
21
21
2
1 dxxdxxfxXYEXYEE x
Model: Y = U1 + 2U2
otherwise
xyxyf
xxUxXY
UXXXXY
xy ,0
1,1)(
]1,[~|
]1,0[~,2
2|
2
2121
02
1)1ln(1)|(
2
0
1
2 x
x
dydxXYH
9431471807.0)|()( 2 XYHYH
9431471807.0
)22
3ln()
22
3()
2
1ln(
2
1)
2ln(
2)(
1
0
2
1
3
2
dyyy
dydyyy
YH
Model: Y = U1 + 2U2
1
0
2
1
3
2
22
2
3)
2
1
2
3(
2
1
2
1)()( dyyyydydyydyyyfYE
xdyydyyfyxXYE
x
x
xy
2
11)()|(
1
|2 2
3333.02
3
12
31)]|([)(
2
2
XYVarEYVar
12
31
2
1)
2
1()()|())|((
1
0
22
22
2
2 dxxdxxfxXYEXYEE x
Model: Y = U1 + 2U2
Model: Y = a1N1+a2N2+a3N3+…
),(~,...,,... 212211 NXXXXaXaXaY nnn
Bernard Krzykacz-Hausmann:
22)]|([)( iii aXYVarEYVar
22
22
1ln2
1)|()(
kk
iii
a
aXYHYH
dyyfyfYH ))(ln()()(
)(
),()|(| xf
yxfxyf xy
dxyxfyf ),()(
dxdyyfyxf ))(ln(),(
dxdyxfxyfxyf
dxxfxYHXYH
)())|(ln()|(
)()|()|(
dxdyxf
yxfyxf )
)(
),(ln(),(
dxdyyfxf
yxfyxfXYHYH )
)()(
),(ln(),()|()(
;
;
Derivation of H(Y)-H(Y|X)
dxdyyfxf
yxfyxfXYHYH )
)()(
),(ln(),()|()(
1
1
1
1
1
1
..
.
1
1
..
.
1
1
1
1
..ln
1
1
1
1
..
ia
ia
jb
jb
jb
jbn
jn
ia
ian
in
jb
jb
ia
ian
ijn
jb
jb
ia
ian
ijn
j i
i
aa xi
ai
ani
nxf
ii)(1
1
1
..
.)( ),[ 1
j
bb yj
bj
bnj
nyf
jj)(1
1
1
..
.)( ),[ 1
ji
bbaa yxj
bj
bi
ai
anij
nyxf
jjii
,
),[),[ )(1)(11
1
1
1
..),(
11
j i
nj
n
ni
n
nij
n
nij
n
..
.
..
.
..ln..
Estimate of H(Y)-H(Y|X)
))|(())|(()( XYEVarXYVarEYVar
))|(())|(( 22 XYEEXYEE
)())|(( 22 YEXYEE
j j
ijjjj
jj
ijjy n
nybb
bbn
nydyyfyYE
..
1
1..
1)()(
dyxf
yxfydyxyfyxXYE xy )(
),()|()|( |
j
iiaa
ii
i
iiii
ijj
bbx
aan
nbbaan
ny
ii 1),[
1..
.
11.. )(11
11
1
j
aa
i
ijj x
n
ny
ii)(1 ),[
.1
Estimate of Var(Y)-E(Var(Y|X))
2
.....
.
2
.
11
i j
ijj
i
i
i j i
ijj ny
nnn
n
n
ny
dxxfxXYEXYEE )()|())|(( 22
2
..
.
2
...
11
j
jj
i j
ijj
i n
nyny
nn))|(()( XYVarEYVar
Estimate of Var(Y)-E(Var(Y|X))
