bekee, expert knowledge modeling with bayesian belief networks

©2012 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express

written permission

Plan

Modeling by Brainstorming

Bayesia ExpertKnowledge Elicitation Environment

Bayesia Expert Knowledge Elicitation Environment - BEKEE

An innovative Brainstorming Tool

Dr. Lionel JOUFFE

February 2012

1

1

Plan




written permission2

All models are wrong; the practical question is how wrong do they have to be to not be useful (Box&Draper 87)

MODELING BY BRAINSTORMINGMODELING BY BRAINSTORMING

2

Plan




written permission3

Why?

There is a clear need for Decision Support Systems

Every Decision Maker is faced to complex decisions

Human Beings are not so good at taking rational

decision under uncertainty

3

Plan




written permission4

How?

Explicit Knowledge

Tacit Knowledge

Data is not always available for automatically learning a Decision Support

System with Data Mining algorithms

But experts have gathered invaluable Tacit Knowledge through

their Experience

We need to convert this Tacit Knowledge into Explicit

Knowledge and use it for building models

4

Plan




written permission5

What?

3,6

3,65

3,7

3,75

3,8

3,85

3,9

3,95

A priori Flowery Feminine Original Tenacious Fruity

Bayesian Belief Networks (BBNs) are ideal models for Expert Knowledge Modeling

Graphical Representation Powerful Probabilistic Engines

What-if scenarios

Drivers analysis

5

Plan




written permission6

BBNs are made of Two Distinct Parts

Qualitative part: the Structure

Directed Acyclic Graph (DAG), i.e. no directed loop

Nodes represent the variables

Each node has a set of exclusive states (e.g.: Poor, Good)

Arcs represent the direct probabilistic relationships between the variables (possibly causal)

6

Plan




written permission7

BBNs are made of Two Distinct Parts

Quantitative part: the Parameters

Probability tables are associated to each node

MARGINAL PROBABILITY DISTRIBUTIONHalf of the products are of Good quality

40% of the Brand Images are Poor

The size of the Conditional Probability Tables grows

exponentially with respect to the number of Parents

CONDITIONAL PROBABILITY DISTRIBUTIONThere are 60% of chance that the Perceived

Quality is Good for Poor Quality products with Good Brand Image

7

Plan




written permission8

BBNs are Powerful Observational Inference Engines ...

We get some evidence on the states of a subset of variables: Hard positive and negative evidence, Likelihood, Probability distributions, Mean values

We take these findings into account in a rigorous way to update our belief on the states of all the other variables

Probability distributions on their values

Multi-Directional Inference (Simulation and/or Diagnosis)

The evidence on Perceived Quality (a new

probability distribution) allows to update the probability distribution of

Brand Image (Diagnosis) and Satisfaction (Simulation)

Prior Distribution Posterior Distribution

8

Plan




written permission9

We DO some state modification on a subset of variables: Hard positive and negative actions, Likelihood, Probability distributions, Mean values

We take these actions into account in a rigorous way to update our belief on the states of all the descendant variables

Simulation of the effects of these actions

Probability distributions on their values

Simulating a new population made of 85% of Good

Perceived Quality products rather than focusing on a sub-population made of

such products

Prior Distribution Posterior Distribution

... and Powerful Causal Inference Engines

9

Plan




written permission10

BBN Modeling by Brainstorming

Brain Storming Sessions with this group of Experts to manually build the BBN, conceptual dimension per conceptual dimension

Clear definition of the BBN’s objective(s) (e.g.: Improvement of the Product/Service Quality,

improvement of the Purchase Intent, improvement of the Company’s performance, ...)

Identification of the conceptual dimensions that are linked to these objectives

(e.g.: Human resources, Management, Production, Marketing, ...)

Definition of the group of experts that will fully cover all the dimensions (and the different geographical zones),

with a small redundancy for allowing fruitful expert debates

10

Plan




written permission

The StructureThe Qualitative Part

11

One sub-network per Conceptual Dimension

11

Plan





BAYESIA ExpertKnowledge Elicitation Environment

Interactive

Batch

12

Each expert gives his/her belief on the probability distributions

Plan




written permission

The ParametersThe Quantitative Part

13

Probabilities do not have to be exact to be useful

BIASES

Cognitive (Plausibility

, Control, A

vailability,

Anchoring)

Emotional (Mood, Motivation)Group (Anchoring, Herding)

Facilitator (can be biased toward charismatic experts or toward the last expressed opinion)

☛ Bayesia Expert Knowledge Elicitation Environment for reducing these biases, improving traceability, gathering all the useful

knowledge, ....

13

Plan





Interactive Sessions

14

Plan





Batch Sessions

15

Plan




written permission

Expert Management

* Available on subscription only16

The Expert Editor allows defining:The Expert’s name, its Credibility (that will be use globally during

the consensus computation), her/his Picture, a Comment to describe her/his area of expertise. The last field contains the number of assessments realized by the expert on the current

network

Group of experts can be Imported and Exported

Communication with the BEKEE web server*

Allows generating a Bayesian network by using the assessments of the selected experts only

16

Posting a Question to the Server

Plan





Selecting a cell in the probability table activates the Assessment

button for assessing the question corresponding to the selected line,

i.e. what is the marginal probability distribution of Mobility over the 3

defined states?

The Assessment Editor allows the Facilitator manually adding,

deleting and modifying Experts’ Assessments.

The Post Assessment button is used by the Facilitator to send the question to the BayesiaLab’s secured website for

an online assessment

17

Plan




written permission

https://www.bayesialab.com/expertise2/

18

18

http://www.bayesialab.com/expertise2/

http://www.bayesialab.com/expertise2/

Plan




written permission

Web Tool

19

19

Plan





Interactive Session

Waiting for a question send by the Facilitator

Pressing Play allows participating to the interactive session

Click the Lock to fix that probability

Confidence level of the expert used for weighting the assessment

Comment field for explaining,detailing the assessment

20

Plan





Interactive SessionNode with Parents

The context variables in the BBN

21

Plan




written permission

The Facilitator’s tool

22

Pressing OK makes BayesiaLab harvesting the assessments

Once the Expert validates her/his assessment, this assessment is sent to the BayesiaLab’s

server and the Facilitator’s listener is automatically updated

This listener allows following the status of the Experts’

assessments

22

Plan




written permission

The Facilitator’s tool

Sorting the assessments by state probabilities can be used for:- detecting Experts’ misunderstanding

- Knowledge sharing, especially by making the 2 “extremes” Experts debate

If some useful knowledge comes out from the debate, the Facilitator can post again the question for new Expert Assessments. Each Expert will then be allowed to update her/his assessment online (each Experts’ webpage is initialized with the

information she/he set in the previous round)

23

The content of this editor is sortable by each column just by clicking on the corresponding

header.It is sorted here in the ascending order on the

probabilities assessed for the state Weak

23

Plan




written permission

The Consensus

24

Once the assessments validated, a Mathematical consensus is computed by using the Experts’

credibility and their assessment’s confidence. This automatic consensus can be manually modified by the Facilitator to set a Behavioral consensus, i.e.

one issued after a fruitful expert debate

Hovering over this icon returns the minimum and the maximum assessments, and the

number of assessments

A small icon is associated to each probability for graphically

representing the consensus degree. That icon goes from full transparency, when all

the votes are identical, to no transparency at all, when the assessment range is 1 (one expert set

0% and another one set 100%)

A Consensus icon is also associated to the nodes for indicating the global consensus over all the

distributions. The darker the icon is, the lower the global consensus is

24

Plan




written permission

The Consensus

This information panel contains:- the number of rows (probability distributions) that have Experts assessments- the total number of assessments that have been set in the probability table

- the number of Experts that have assessed at least one probability distribution in the table - a measure of the global disagreement that takes into account the deviations from the

mathematical consensus- the maximum disagreement corresponding to the greatest difference between two

assessments in the probability table

25

Pressing the “I” key while hovering over the expert icon allows displaying the

information panel below

25

Plan




written permission

The Assessment Report

26

This report first gives information on the Experts, then returns a sorted list of the nodes wrt the

global disagreements, and another one wrt the maximal disagreements.

Finally, for each node, a summary contains all the global information on the assessments of the

(Conditional) Probability Table

Right clicking on the Expert Icon in the lower left corner of the Graph

window allows generating an HTML report

26

Plan




written permission

The Graph Report

27

The information given by the Assessment and Graph reports is useful for the Model Verification. High divergences can be due to state inversion, fuzzy definitions of the variables and/or their

states, different contexts

The Graph report allows generating HTML Conditional Probability Tables. These tables comes with the consensual probability distributions and the

maximum divergences.

From white (0) to blue (100) for the probabilities

Colors are associated to each cell

From green (0) to red (50) for the divergences

27

Plan




written permission

Batch Session

28

In-person meetings are essential for building the qualitative part of the models.

Probability elicitation is time consuming and that quantitative part can be too long to allow the interactive elicitation of all the parameters during the meetings.

Batch sessions allow then each expert to remotely:

Complete the parameter elicitation process

Verify the assessed probabilities

The Facilitator can select the nodes for which the probability distributions

have to be assessed and/or verified

Warning are generated for the distributions that are greater than

30% threshold

28

Plan




written permission

Web Tool

29

This expert has assessed 3 distributions out of 12

The Play button allows participating to the batch

session

The pie chart represents that completion rate

Nodes to assess or verify

29

Plan




written permission

Web Tool

30

30

Plan




written permission

Exportation of a Bayesian Network per Expert

31

This exportation tool allows the creation of one Bayesian Belief Network per Expert.

The parameters (probabilities) are those assessed by the Expert. For probabilities not assessed by the Expert, the model is based on the consensual probabilities, either the mathematical one, or

the behavioral one entered by the Facilitator

31

Plan




written permission

Exportation of the Probability Assessments

32

Generation of a CSV file with all the assessed probabilities, one line per cell/

probability

Context in terms of states Assessed Node

32

Plan




written permission

Exportation of the Expert Assessments

33

Generation of a CSV file with all the assessments of the Experts, one

line per cell/probability.

1/number of states of the assessed variable

33

Plan




written permission

Analysis of the Expert Assessments

34

The Expert Assessment file can be analyzed with the unsupervised learning algorithms of BayesiaLab for finding the direct probabilistic relationships that hold

between the Experts’ assessments

Each node represents the discretized probabilities assessed by the Expert

34

Plan




written permission

Automatic Segmentation of the Experts

Based on the obtained Expert Segments, one Bayesian network per segment can be generated (by using the Expert Editor). This can be useful for analyzing the sensibility of the model, but also to get specific networks (depending on the geographical localization

for example)

35

Based on the obtained network, Experts can be clustered into homogeneous groups by using the BayesiaLab’s Variable Clustering algorithm

Dendrogram corresponding to that segmentation Each color corresponds to a

cluster.

The real experts behind those anonymized experts have indeed 3 different profiles (functionally and

geographically)

35

Plan




written permission

Parameter Sensibility Analysis

36

The Assessment Sensitivity Analysis tool allows measuring

the uncertainty associated to the consensus

Generation of a set of networks by randomly selecting Experts’ assessments

Measurement of the uncertainty associated to each probability distribution

The probability of Strong goes from 30% to 86%

36


written permission

Plan



Contact

37

6 rue Léonard de Vinci BP0119

53001 LAVAL CedexFRANCE

Dr. Lionel JOUFFEPresident / CEO

Tel.: +33(0)243 49 75 58Skype: +33(0)970 44 64 28Mobile: +33(0)607 25 70 05Fax: +33(0)243 49 75 83

37

http://www.remarkgroup.com/






















bekee, expert knowledge modeling with bayesian belief networks

Business

evidence onperceived

goodperceived quality

productservice quality

bbn modeling

bbns objectivese

explicitenvironment

negative evidence

bayesian belief networks