bpm cluster meeting 2014
DESCRIPTION
Slides of my presentation at BPM Cluster meeting, 21 August 2014, Eindhoven, NLTRANSCRIPT
1/31www.janclaes.info
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
Jan ClaesTeaching assistant : PhD 2009 – 2015
Supervisors UGent : Geert Poels & Frederik GaillySupervisors TU/e : Paul Grefen & Irene Vanderfeesten
Why do people struggle with the complexity of constructing a process model?
2/31www.janclaes.info
Business Process Modeling (BPM)Assumptions
Constructing process models is useful Constructing the models is cognitively challenging
Business Process Modeling (BPM) Process model quality
“what is a good process model?” Process of process modeling (PPM)
“how to make a good process model (effectiveness) cccccccc using a good approach (efficiency)?”
3/31www.janclaes.info
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
How do people construct process models?
4/31www.janclaes.info
PPMChart visualization
CREATE_ACTIVITY CREATE_START_EVENT CREATE_END_EVENT CREATE_AND CREATE_XOR CREATE_EDGE MOVE_ACTIVITY MOVE_START_EVENT MOVE_END_EVENT MOVE_AND MOVE_XOR DELETE_ACTIVITY DELETE_START_EVENT DELETE-END_EVENT DELETE_AND DELETE_XOR DELETE_EDGE NAME_ACTIVITY RENAME_ACTIVITY NAME_EDGE RENAME_EDGE
Start event Edge Activity
Gateway
Edge
Activity
Edge
Edge
Activity
Edge Gateway
Edge
123
5
7
9
4
6
8
1112
10
time
mod
el e
lem
ents
5/31www.janclaes.info
Flow-oriented modeling
6/31www.janclaes.info
Aspect-oriented modeling
7/31www.janclaes.info
Undirected modeling
8/31www.janclaes.info
Observations (objective)
Obs. 1. Majority used serialized approachObs. 2. Large group flow-oriented modelingObs. 3. Small group aspect-oriented
modelingObs. 4. Large group combine FO and AOObs. 5. Small group undirected modelingObs. 6. Remainder is uncategorizedObs. 7. Undirected sessions lasted longer
9/31www.janclaes.info
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
What about the quality of the corresponding models?
10/31www.janclaes.info
Impressions (subjective)
Impr. 1. Modelers need serializationImpr. 2. Structured serialization
helps avoiding ‘mistakes’Impr. 3. Structured serialization
does not support every modeler to the same extent
11/31www.janclaes.info
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
Is there a rational for these observations and impressions?
12/31www.janclaes.info
Kinds of human memory
input Sensory memory
Workingmemory
Long term memory
performanceOBSERVATION SELECTION
ORGANIZATION
RETRIEVALTRANSFER
Workingmemory
13/31www.janclaes.info
Types of cognitive load
Time
Inst
anta
neou
s lo
ad
overall load
intrinsicload
extraneousload
germane load
freecapacity
assumed capacity
instantaneous load
overload
14/31www.janclaes.info
Cognitive Load Theory (CLT)
case complexity prior knowledgeinput material
representation fitworking memory
capacity
extraneous cognitive load
germane cognitive load
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
overall construction effort
cognitive overload
intrinsic cognitive load
++
+
+
+++
+ +
–
– – –
–
–
15/31www.janclaes.info
Cognitive Fit Theory (CFT)
Need for structure
Learning styleField dependencySensory
Visual
Inductive
Active
Sequential
Intuitive
Auditory
Deductive
Reflective
Global
I ♥︎� chaos I #$@&! chaos
DON’T SEE THE FOREST THROUGH THE TREES
16/31www.janclaes.info
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
What can we do with this knowledge?
17/31www.janclaes.info
ImpressionsCognitive theoryObservations
Theory building
Structured Process Modeling Theory
17/32www.janclaes.info
18/31www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines BA B The more A, the more B+ A B The more A, the less B– A B A translates into B
learning style
degree of serialization
adopted serialization style
need for structure
field-dependency
– +
intrinsic cognitive load for modeling phases
intrinsic cognitive load for aggregation phases
cognitive overload
intrinsic cognitive load for strategy building phases
+ + +
serialization style fitstructuredness of serialization
– – – –
1 2 3
19/31www.janclaes.info
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
How will the developed theory be tested?
20/31www.janclaes.info
Methodology1. Cognitive tests prior to the test session
Collect modeler data2. Observational modeling session
At TU/e in BPM lecture of September 26 Collect modeler, modeling and model data
3. Post-experiment interviews with selection of participants (e.g.,
outliers) Collect additional, qualitative data
21/31www.janclaes.info
Experiment set-upNo treatmentMeasure every involved construct of the theory
Modeler, modeling process, process modelCalculate (cor)relations of the theory
22/31www.janclaes.info
PRIOR TO THE EXPERIMENT SESSION
HOMEWORK
23/31www.janclaes.info
Experiment set-up1. Prior to the experiment session
Software to be installed on personal laptop ( = download from website and unzip)
5 cognitive tests (time range 5-30 min)• Reading span test (RSPAN)• Operation span test (OSPAN)• Counting span test (CSPAN)• Hidden figures test (HFT)• Need for structure (NFS)
24/31www.janclaes.info
Experiment set-up1. Prior to the experiment session
To be completed at home Before deadline Software collects data in one-zip file per test
which participants have to submit
25/31www.janclaes.info
Experiment set-up1. Prior to the experiment session
Students will receive their individual results as incentive to not cheat on these tests
Late responses or too much errors reveals cheating Student is not allowed to start actual experiment if
the test results are not submitted in time
26/31www.janclaes.info
AT THE EXPERIMENT SESSION
MODELING TASK
27/31www.janclaes.info
Experiment set-up2. At the experiment session
Modeling task and post-task survey In the same, already installed program
(using a different start-up code) Software automatically collects data
• Each operation in the tool ( = each activity in the modeling process)
• Constructed process model, number of operations, modeling time
in one-zip file which they have to submit
28/31www.janclaes.info
Experiment set-up2. At the experiment session
Student does not get credit if not all data (6 zip files) are submitted
Technical problems > replacement assignment(e.g., help me with coding the collected data)
29/31www.janclaes.info
AFTER THE EXPERIMENT SESSION
INTERVIEW
30/31www.janclaes.info
Experiment set-up3. After the experiment session
Data will be analyzed and outliners will be defined Selection of students will be invited for post-
experiment interview (voluntarily) Collect qualitative data
• Explain how they have modeled• Explain their cognitive profile• Explain which results we expected and why• Ask about their opinion: why did they (not) achieve the
predicted results?
31/31www.janclaes.info
Thanks for you attention!
> Do you have feedback on my research plans? <> set-up is useful for theory testing? <
> set-up is feasible (enough)? <
> other suggestions? <
Does someone has some time to test the software?
http://www.janclaes.infoTwitter: @janclaesbelgium
32/31www.janclaes.info
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
How can the concepts of the theory be measured?
33/31www.janclaes.info
Cognitive Load Theory (CLT)
case complexity prior knowledgeinput material
representation fitworking memory
capacity
extraneous cognitive load
germane cognitive load
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
overall construction effort
cognitive overload
intrinsic cognitive load
++
+
+
+++
+ +
–
– – –
–
–
34/31www.janclaes.info
overall construction effort
Cognitive Load Theory (CLT)
case complexity prior knowledgeinput material
representation fit
extraneous cognitive load
germane cognitive load
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
cognitive overload
intrinsic cognitive load
++
+
+
+++
+ +
–
– – –
–
–
working memory capacity
Reading span - Operation span - Counting span
Reading spanFor many years, my family and friends have been working on the farm. SPOT
Because the stuffy was room, Bob went air for some fresh outside. TRAILWe were fifty miles out at sea before we lost sight of the land. BAND
Counting span
Count the dark blue circles
Operation spanIS (8 / 4) – 1 = 1 ? BEAR
IS (6 X 2) – 2 = 10 ? BEANSIS (10 X 2) – 6 = 12 ? DAD
35/31www.janclaes.info
overall construction effort
Cognitive Load Theory (CLT)
case complexity prior knowledgeworking memory
capacity
extraneous cognitive load
germane cognitive load
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
cognitive overload
intrinsic cognitive load
++
+
+
+++
+ +
–
– – –
–
–
input material representation fit
Fit with modeling task = constantFit with modeler through questions
Example questionsI did experience problems using the modeling tool (5-point Likert)
Using the modeling language was not difficult (5-point Likert)I found it easy to understand the case description (5-point Likert)
36/31www.janclaes.info
Cognitive Load Theory (CLT)
prior knowledgeinput material
representation fitworking memory
capacity
extraneous cognitive load
germane cognitive load
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
overall construction effort
cognitive overload
intrinsic cognitive load
++
+
+
+++
+ +
–
– – –
–
–
case complexity
Case complexity = constant(number of activities and arrows in solution model can be used as proxy for number of concepts and relations)
37/31www.janclaes.info
overall construction effort
Cognitive Load Theory (CLT)
case complexityinput material representation fit
working memory capacity
extraneous cognitive load
germane cognitive load
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
cognitive overload
intrinsic cognitive load
++
+
+
+++
+ +
–
– – –
–
–
prior knowledge
Prior domain, modeling language and modeling method knowledgethrough questions
Example questionsI am familiar with the mortgage handling process (5 point-Likert)
How many process models did you ever create?I am experienced in process modeling (5-point-Likert)
38/31www.janclaes.info
overall construction effort
Cognitive Load Theory (CLT)
case complexity prior knowledgeinput material
representation fitworking memory
capacity
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
++
+
+
+++
+ +
–
– – –
–
–
extraneous cognitive load
germane cognitive load
cognitive overload
intrinsic cognitive load
Through questions
Example questionsI found it hard to complete the task (5-point-Likert)
39/31www.janclaes.info
Cognitive Load Theory (CLT)
case complexity prior knowledgeinput material
representation fitworking memory
capacity
extraneous cognitive load
germane cognitive load
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
overall construction effort
cognitive overload
intrinsic cognitive load
++
+
+
+++
+ +
–
– – –
–
– schema construction
40/31www.janclaes.info
Cognitive Load Theory (CLT)
case complexity prior knowledgeinput material
representation fitworking memory
capacity
extraneous cognitive load
germane cognitive load
schema construction
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
overall construction effort
cognitive overload
intrinsic cognitive load
++
+
+
+++
+ +
–
– – –
–
–
process model quality
Syntactical quality: number of cognitive mistakesSemantic quality: assessment by experts using checklist (penalty for over- and underfit)
41/31www.janclaes.info
Cognitive Load Theory (CLT)
case complexity prior knowledgeinput material
representation fitworking memory
capacity
extraneous cognitive load
germane cognitive load
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
cognitive overload
intrinsic cognitive load
++
+
+
+++
+ +
–
– – –
–
–
overall construction effort
Time between start of reading case description and last operation in the toolNumber of operations
42/31www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines BA B The more A, the more B+ A B The more A, the less B– A B A translates into B
learning style
degree of serialization
adopted serialization style
need for structure
field-dependency
– +
intrinsic cognitive load for modeling phases
intrinsic cognitive load for aggregation phases
cognitive overload
intrinsic cognitive load for strategy building phases
+ + +
serialization style fitstructuredness of serialization
– – – –
43/31www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines BA B The more A, the more B+ A B The more A, the less B– A B A translates into B
learning style
degree of serialization
adopted serialization style
need for structure
field-dependency
– +
intrinsic cognitive load for modeling phases
intrinsic cognitive load for aggregation phases
cognitive overload
intrinsic cognitive load for strategy building phases
+ + +
serialization style fitstructuredness of serialization
– – – –
Through questions
Example questionsWhen I write a text:
• I usually work on content, structure, wording and format right from the start• I usually first decide on the content, then the structure, next I write sentences
and finally I work on the formatting
44/31www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines BA B The more A, the more B+ A B The more A, the less B– A B A translates into B
learning style
degree of serialization
adopted serialization style
need for structure
field-dependency
– +
intrinsic cognitive load for modeling phases
intrinsic cognitive load for aggregation phases
cognitive overload
intrinsic cognitive load for strategy building phases
+ + +
serialization style fitstructuredness of serialization
– – – –
Through questions
Need for Structure scale12 questions with 6-point Likert scale
45/31www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines BA B The more A, the more B+ A B The more A, the less B– A B A translates into B
learning style
degree of serialization
adopted serialization style
need for structure
field-dependency
– +
intrinsic cognitive load for modeling phases
intrinsic cognitive load for aggregation phases
cognitive overload
intrinsic cognitive load for strategy building phases
+ + +
serialization style fitstructuredness of serialization
– – – –
Hidden Figures Test
46/31www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines BA B The more A, the more B+ A B The more A, the less B– A B A translates into B
learning style
adopted serialization style
need for structure
field-dependency
– +
intrinsic cognitive load for modeling phases
intrinsic cognitive load for aggregation phases
cognitive overload
intrinsic cognitive load for strategy building phases
+ + +
serialization style fit– – – –
degree of serialization
structuredness of serialization
Assessment by experts
Flow-oriented – Aspect-oriented – CombinationHappy Path First – Undirected
Uncategorized
47/31www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines BA B The more A, the more B+ A B The more A, the less B– A B A translates into B
learning style
degree of serialization
adopted serialization style
need for structure
field-dependency
– +
intrinsic cognitive load for modeling phases
intrinsic cognitive load for aggregation phases
cognitive overload
intrinsic cognitive load for strategy building phases
+ + +
structuredness of serialization
– – – –
serialization style fit
Theoretical matchPost-experiment interview
Global learner = aspect-oriented, sequential = flow-orientedHigh need for structure = no undirected
Field-dependent = flow-oriented
48/31www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines BA B The more A, the more B+ A B The more A, the less B– A B A translates into B
learning style
degree of serialization
adopted serialization style
need for structure
field-dependency
– +
+ + +
serialization style fitstructuredness of serialization
– – – –
intrinsic cognitive load for modeling phases
intrinsic cognitive load for aggregation phases
cognitive overload
intrinsic cognitive load for strategy building phases
See earlier: through questions