lian duan and mats p. e. heimdahlfm.csl.sri.com › verisure2015 › talks › verisure beta.pdfdr....
Post on 30-Jan-2021
1 Views
Preview:
TRANSCRIPT
-
VeriSure'15 7/18/2015
Dr. Mats Heimdahl 1
Software Engineering Center
Software Engineering Center
Representing Confidence in
Assurance Case Evidence
Lian Duan and Mats P. E. Heimdahl University of Minnesota Software Engineering Center
Department of Computer Science and Engineering
University of Minnesota
4-192 EE/CS; 200 Union Street SE
Minneapolis, MN 55455
VeriSure'15 1 7/18/2015
Funded by CNS-0931931
and CNS-1035715
Software Engineering Center
Assurance Case Example
2 7/18/2015 VeriSure'15
Abstractions OK?
Environment OK?
Tool OK?
Fraud?
Oracles OK?
Test harness OK?
Test vs Production?
Fraud?
-
VeriSure'15 7/18/2015
Dr. Mats Heimdahl 2
Software Engineering Center
Confidence
• Need to have confidence in the claims made in the assurance case
– Does not matter what approach to the case we take
• Approving a system is always a judgement call
• Can model it qualitatively or quantitatively
– Qualitatively: • Separate confidence case, acceptance criteria, …
– Quantitatively: • Use a single number, a range, or a distribution
7/18/2015 VeriSure'15 3
Software Engineering Center
Previous Work: Bayesian Networks
• Evidence nodes (quantitative information)
• Links (qualitative information)
7/18/2015 VeriSure'15 4
-
VeriSure'15 7/18/2015
Dr. Mats Heimdahl 3
Software Engineering Center
Previous work:
Dempster-Shafer Theory
• Separate out uncertainty
• Belief, Disbelief, Uncertainty
7/18/2015 VeriSure'15 5
Software Engineering Center
Jøsang’s Opinion Triangle
• Standard logic: bottom part (disbelief to belief)
of triangle
• Top vertex: uncertainty
• An opinion is
{belief, disbelief, uncertainty}
• Base rate: fourth variable
(prior belief)
7/18/2015 VeriSure'15 6
-
VeriSure'15 7/18/2015
Dr. Mats Heimdahl 4
Software Engineering Center
Our Proposal
• Use the beta distribution to represent confidence
• Mapping to Jøsang’s opinion triangle
7/18/2015 VeriSure'15 7
Software Engineering Center
Beta Distribution
• Continuous version of
binomial distribution
• Finite range
• Versatile
• 2nd order distribution:
probability of
probabilities
7/18/2015 VeriSure'15 8
-
VeriSure'15 7/18/2015
Dr. Mats Heimdahl 5
Software Engineering Center
Uncertainty vs. Split Opinion
• {0.0, 0.0, 1.0} – Full uncertainty
• {0.5, 0.5, 0.0} – Split opinion
7/18/2015 VeriSure'15 9
Software Engineering Center
More Beta/Opinion Triangle:
7/18/2015 VeriSure'15 10
-
VeriSure'15 7/18/2015
Dr. Mats Heimdahl 6
Software Engineering Center
More Beta/Opinion Triangle:
7/18/2015 VeriSure'15 11
Software Engineering Center
More Beta/Opinion Triangle:
7/18/2015 VeriSure'15 12
• Visualize as spikes at specific belief values
0 1 .5
-
VeriSure'15 7/18/2015
Dr. Mats Heimdahl 7
Software Engineering Center
More Beta/Opinion Triangle:
7/18/2015 VeriSure'15 13
Software Engineering Center
Testing Evidence
7/18/2015 VeriSure'15 14
-
VeriSure'15 7/18/2015
Dr. Mats Heimdahl 8
Software Engineering Center
Logical Argument Example
7/18/2015 VeriSure'15 15
Software Engineering Center
Software Node: Ev1 Ev2
• Consensus operator:
7/18/2015 VeriSure'15 16
-
VeriSure'15 7/18/2015
Dr. Mats Heimdahl 9
Software Engineering Center
Hardware Node: Ev3 Ev4
• Consensus operator
7/18/2015 VeriSure'15 17
Software Engineering Center
No Over-Radiation
• Logical OR
7/18/2015 VeriSure'15 18
-
VeriSure'15 7/18/2015
Dr. Mats Heimdahl 10
Software Engineering Center
Work in Progress
• Propose the use of the Beta distribution, and its duality with the opinion triangle/subjective logic, to represent confidence in assurance cases
• Work in progress
– Combination of distributions in an assurance case
– Weighing evidence
– Elicitation of confidence estimates
– Sensitivity analysis
7/18/2015 VeriSure'15 19
Software Engineering Center
Summary
7/18/2015 VeriSure'15 20
Thank You
-
VeriSure'15 7/18/2015
Dr. Mats Heimdahl 11
Software Engineering Center
Summary
7/18/2015 VeriSure'15 21
Thank You
Questions: • Should we quantify confidence/trust?
• Can we elicit belief in any reliable manner?
• Is Beta the answer?
• Can we have “belief templates” for various types
of evidence?
• Will this ever be better than educated guesses?
top related