bpm seminar understandability of process models a user’s perspective on learning
TRANSCRIPT
BPM Seminar
understandability of process models
A user’s perspective on learning
Topics
1. Context of the research
2. Theoretical Background
3. Theoretical Design & Conceptualisation
4. Measurement
5. Questions?
1. Context of the research
Goal:
Defining understanding as a result of learning, how do differences in user
characteristics contribute to differences in understanding process models?
An example
Kick-off
• Curtis et al. (1992)
Goal:
“facilitate human understanding and communication”
Sub-goal:
“represent processes in forms understandable by humans”
Gap-analysis
• Independent variables
The effect of:
task complexity
technology/process model characteristics
on performance
Visualisation
Human
Data Model
Task
Performance
Gap-analysis II
• Understandability defined as an intrinsic property of a process model
• Complexity, learnability, usability, etc.
• Pragmatic quality
Why is this important?
• Practically:) helps representing processes in forms understandable by humans) additional insights allow for more effective
training
• Academically:) effects of users often averaged out by assuming homogeneity (or x vs. y)) Maturity of the discipline will benefit from an interdisciplinary approach
2. Theoretical background
“To learn is to be human”
(Goward)
Theory of Cognitive Load
• Define information integration as a learning process
• Control factors of:Content
&Content Presentation
An example
WOOL
MEDDOW
GRASS
GOAT
Learning Conceptualised
PRESAGE PROCESS PRODUCT
USERCHARACTERISTICS
LEARNING CONTEXT
KNOWLEDGECONSTRUCTION
LEARNINGOUTCOMES
Revisions
• Controlled Learning Context
• Limited Mutuality
USERCHARACTERISTICS
KNOWLEDGECONSTRUCTION
LEARNINGOUTCOME
Controlling the Learning Context
HOW?
3. Theoretical Design& Conceptualisation
a Cognitive Perspective onLearning
Structure
• Gap Analysis
• Introduction of a framework
• Integration into learning theory
Prior to Learning
USERCHARACTERISTICS
KNOWLEDGECONSTRUCTION
LEARNINGOUTCOME
User CharacteristicsGap analysis
distalvariables
affective variables psychosocial variables
skills & expertise
Agarwal et al., 1999 Cognitive Fit
Aranda et al., 2007 Domain ExpertiseLanguage Expertise
Bandara et al., 2005 User Competence
Chiew & Wang, 2004
Syntactic KnowledgeSemantic KnowledgeSchematic KnowledgeStrategic Knowledge
Maes & Poels, 2007 Perceived Ease of UsePerceived Usefulness
Perceived Semantic QualityUser Satisfaction
Mendling et al., 2007
ExperienceDomain Expertise
Patig, 2008 AgeGender
ExperienceDomain Knowledge
Reijers & Mendling,2008
EducationJob type
Domain KnowledgeCompany Experience
Field Experience
Input: User Characteristics
• Identification of a framework which conceptualises user characteristics in an integrative manner
• Problem-solving is inherent to human nature
Usage of goal-setting behavioural theory
Goal-setting Behavioural Theory
Eight VariablesPresage
User Characteristics
Learning Context
Self-Efficacy
Skills
Attitude
Traits,Beliefs
SubjectiveNorm
PositiveAnticipatedEmotions
NegativeAnticipatedEmotions
PerceivedBehavioural
Control
Process
LearningStrategies
GoalMotives
GoalIntention
Knowledge Construction
Product
LearningOutcome
NoLearning
MeaningfulLearning
FragmentedLearning
During Learning
USERCHARACTERISTICS
KNOWLEDGECONSTRUCTION
LEARNINGOUTCOME
Knowledge ConstructionGap analysis
• Static Approach; hence limited availability
Cardoso et al. (2006): short- and long-term memory
chunking and tracing
Hungerford et al. (2004):
Task Planning
Others, e.g.: Central vs. Peripheral, Queues
Throughput: Knowledge Construction
• Identification of a framework that describes Knowledge Creation
• Mandviwalla & Hovav (1995)
Motivational process
Motivational Hub (Locke, 1991)
Motives Goals Strategy
Personal Agency Beliefs &
Expectancy
Volition
Volitional dimension
Goal
Approach to Learning
Commitment
Three VariablesPresage
User Characteristics
Learning Context
Self-Efficacy
Skills
Attitude
Traits,Beliefs
SubjectiveNorm
PositiveAnticipatedEmotions
NegativeAnticipatedEmotions
PerceivedBehavioural
Control
Process
LearningStrategies
GoalMotives
GoalIntention
Knowledge Construction
Product
LearningOutcome
NoLearning
MeaningfulLearning
FragmentedLearning
After Learning
USERCHARACTERISTICS
KNOWLEDGECONSTRUCTION
LEARNINGOUTCOME
Learning OutcomeGap analysis
Cognitive Performance Behavioural Performance
Affective Performance
Agarwal & Karahanna, 2000
Behavioural Intention to Use
Aranda et al., 2007
Correctness of understanding Time to completion ConfidencePerceived Difficulty
Goodhue et al., 2000
Time to completion User Evaluation of consistencyUser Evaluation of training
Igbaria et al., 1997
System Usage
Mendling et al., 2007
Number of questions correct Perceived understandability
Recker & Dreiling, 2007
Number of questions correctCloze Recall testProblem Solving task
Time to task completion Ease of Understanding
Venkatesh & Bala, 2008
Use Behaviour
Output: Learning Outcome
• Identification of a framework which distinguishes between different types of
understanding
Motivational Hub (Locke, 1991)
Test Performance
Learning Outcome Cognitive Description Retention Transfer
No learning No knowledge Poor Poor
Rote learning Fragmented knowledge Good Poor
Meaningful learning Integrated knowledge Good Good
Three VariablesPresage
User Characteristics
Learning Context
Self-Efficacy
Skills
Attitude
Traits,Beliefs
SubjectiveNorm
PositiveAnticipatedEmotions
NegativeAnticipatedEmotions
PerceivedBehavioural
Control
Process
LearningStrategies
GoalMotives
GoalIntention
Knowledge Construction
Product
LearningOutcome
NoLearning
MeaningfulLearning
FragmentedLearning
Time Out
Variables Explained
• Eight Presage
• Three Process
• Three Product
User Characteristics
Categories Variables Sub-variables
Distal Variables Distal Variables Demographics & CultureAttitude towards targetsPersonality traits Exposure to mediaOther individual difference variables
Affective Variables AttitudePositive Anticipated EmotionsNegative Anticipated Emotions
Psychosocial Variables Subjective NormPerceived Behavioural ControlSelf-efficacy
Skills and Expertise Skills and Expertise Syntactic SkillsSemantic SkillsPragmatic SkillsExpertise
Knowledge Construction
Factors Variables Sub-variables
Approach to Learning Goal Motives Fear of failureAim for qualificationIntrinsic interestCommitment to work
Goals PerformanceLearning
Learning Strategies SurfaceDeepNon-directed
Learning Outcome
Variables
No learning
Rote learning
Meaningful learning
4. Measurement
Configuration
• What to measure?
• How to measure?
• Who to sample?
What to measure?• Due to model complexity,
test a part of the model:
a) Personal data prior to learning
b) Information of their learning approach during learning
c) Questions on understanding post learning
a) Personal Data
• What to collect?
Distal Variables: Personality?Demographics?Culture?
Skills: Reading skills?Studying skills?Modelling skills?
Examples of Frameworks
• Frederiks & Weide (2006) Analysis Skillsincl. Handle implicit knowledge, grammatical analysis, abstract sentence structure, think on an abstract level
• Bandara et al. (2007) Content for IS SubjectsFrom focus group: analytical skills, understand the problem, ability to communicate with client
• Lindland et al. (1994) QualitySyntactic, Semantic & Pragmatic Quality
• Vanderfeesten et al. (2007) Complexity metricsincl. Coupling, Cohesion, Modularity
Skills• Syntactic Skills a) Word comprehension
b) Vocabulary & syntaxII) Identify obstacles
• Semantic Skills c) Spatial abilityc) BMP modelling discoursed) experience/past behavioural skillsI) studying habitsII) identify obstacles
• Pragmatic Skills e) Working memorye) Integration capacityf) Real time self-evaluationI) Develop effective strategiesII) Problem-solvingIV) Emotional stability
b) Learning Approach
• Methods available
Yet, is this useful?
c) Understanding
• Methods available
• Only cognitive measurement?Indicators:Recall & Transfer
Question answered correctly Problem solving
Time?
• Incorporation of affective constructs
Cognitive Performance Behavioural Performance Affective Performance
How to measure?
Qualitative vs. Quantitative
+
Ability to measure all skills
Control over context
Scope
No additional layer of interpretivism
Fixed vs. Loose
• Should respondents be provided with a goal?
• Should meaningful learning be the outcome to aim for?
Tests, tests, tests
some examples• Spatial ability test• Working memory
integration test• Reading ability• Understanding of story• Problem solving• Cloze Recall test• Cognitive Coupling
Who to sample?
• Controlling the Learning Context
vs.homogeneity of population
• Assume group differences (e.g. expert/novice)
vs.differences by measurement
Encore un foisPresage
User Characteristics
Learning Context
Self-Efficacy
Skills
Attitude
Traits,Beliefs
SubjectiveNorm
PositiveAnticipatedEmotions
NegativeAnticipatedEmotions
PerceivedBehavioural
Control
Process
LearningStrategies
GoalMotives
GoalIntention
Knowledge Construction
Product
LearningOutcome
NoLearning
MeaningfulLearning
FragmentedLearning
5. Questions?
?