erpsim : a platform for multi-method experimental research in is prof. pierre-majorique léger,...
TRANSCRIPT
ERPsim : A platform for multi-method experimental research in IS
Prof. Pierre-Majorique Léger, Ph.D.Director ERPsim Lab, HEC Montréal
Videoconference, University of Arkansas, Sept 17th 2010
Summary of the presentation:ERPsim research platform
Usage data
Psychophysiological data
Performance data
Psychometric data
Psychometric data
Psychometric data
Psychometric data
Psychometric data
Virtual time
Direct observation
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 11 2 3
02468
Synchronization of IT usage, psychophysiological and psychometric data
Research platform enabling realistic business IT environments in laboratory settings
2
About ERPsim Lab
ERPsim
Researchplatform
Commercialplatform
Academicplatform
The mission of ERPsim Lab is to conduct research, development and transfer in the field of serious games in information technology.
More than 100 universities worldwide are using ERPsim to teach ERP concepts to more than 3000 students annually
Many large organisations such as Deloitte, SAP, Coca Cola and Croyola, are using ERPsim to train their end-users on ERP systems
Several researchers have started using ERPsim to run experimental protocol in IS
The academic version of ERPsim is available via the SAP University Alliance sdn.sap.com/ua
3
Simulated flight environment
ERPsim: A flight simulator for ERP systems
Simulated plane
4
ERPsim: A flight simulator for ERP systems
Simulated business environment
Real ERP system
5
What does ERPsim do ?
ERP system (SAP) Participants
1) Business analytics2) Business decisions
1) Automation of administrative tasks
2) Simulation of a market3) Simulation of passing of
time 6
© Léger et al. 2009. ERPsim est une technologie propriétaire développée par des chercheurs de HEC Montréal, de l’École Polytechnique de Montréal et du Western Michigan Univ. ERPsim est détenu par Polyvalor et Valorisation Gestion et commercialisé par Baton Simulations. Plus d’info à erpsim.hec.ca Version : Sept 2009
Scope of the simulation
7
Sales
Automated by ERPsim
Automated by ERPsim
Planning & Procurement
Automated by ERPsim
Production
Automated by ERPsim
ERPsim : Automation of administrative tasks
Léger, P.-M. (2006): Using a Simulation Game Approach to Teach Enterprise Resource Planning Concepts,” Journal of Information Systems Education 17: 441-448Léger, P.-M., Robert, J., Babin, G., Pellerin, R., Wagner, B. (2007): ERPsim, ERPsim Lab (erpsim.hec.ca), HEC Montreal, Montreal, Qc.
8
ERPsim academic platform
Academic platform: Pedagogical objectives
Shows how the ERP system supports business strategies
To develop a hands-on understanding of the concepts underlying enterprise systems
To experience the tangible benefits of enterprise integration firsthand
To develop technical skills at taking decision using an enterprise software
10
Paradigm shift in IT education
• Paradigm shift in the teaching– From instructors to coaches– From specialized technical knowledge to holistic
business knowledge• Paradigm shift in the learning
– From “point-and-click” to discovery of how SAP can help achieve business goals
– From technical questions to business strategy issues
Charland, P., Leger, P-M, Feldstein, H., Robert, J., Babin, G., Lyle, D. « Business simulation training: Guideline for new training approaches in IT training », Proceeding of the ICT 2010 Conference, June 30 – July 2 nd, Singapour 11
Moving toward a competency approach
Kang and Santhanam, 2003 12
Research results on attitude and learning (Cronan et al, 2009a 2009b)
• The results suggest that the ERP simulation game was instrumental in ERP business process learning.
• Participant attitudes toward SAP and its use by employees were significantly higher following the simulation game experience.
• After three months on the job, new hires indicate that the learning levels as well as attitudes toward ERP/SAP remain high.
Cronan, T. P.; Douglas, D. E.; Schmidt, P.; and Alnuaime, O. (2009a), “ERP Simulation Game; Learning and Attitudes toward SAP Samples Company First Time Hires, ITRI-WP123-1009, Information Technology Research Institute, University of Arkansas.
Cronan, T. P. ; Douglas, D. E.; Schmidt, P.; and Alnuaime, O. (2009b), “Evaluating the Impact of an ERP Simulation Game on Student Knowledge, Skills, and Attitudes‘ ITRI-WP123-1008, Information Technology Research Institute, University of Arkansas. 13
Research results on self-perceived vs objective measure
• The results indicate significant correlation between the objective measures and the self-assessed measures of knowledge.
• Assessment of the validity of using perceived (self-assessed) measures as well as provide effective objective measures of learning in this ERP simulation context.
Cronan, T. P.; Léger, P.-M.; Charland, P.; Robert, J.; Babin, G. (2010) “Comparing Objective Measures and Perceptions of Cognitive Learning in an ERP Simulation Game: A Research Note, ITRI-WPXXX-XXXX, Information Technology Research Institute, University of Arkansas.
Self-Assessed Measures
Enterprise Systems (ES) Management
Knowledge
Business Process
Knowledge
ERP Transaction
Skills
Objective Knowledge Corr. Sig1 Corr. Sig1 Corr. Sig1 Objective knowledge Overall (30 questions) 0.41 ** 0.43 ** 0.46 **
Objective knowledge ERP (15 questions) 0.38 ** 0.40 ** 0.41 ** Objective knowledge Business/collaboration (15 questions) 0.37 ** 0.39 ** 0.42 **
Objective knowledge - Easy (20 questions)
0.43 ** 0.45 ** 0.47 **
Objective knowledge - Complex (10 questions) 0.23 ** 0.22 ** 0.27 **
Converted GPA 0.03 Ns 0.09 Ns 0.11
Ns
1 Bilateral test for Pearson correlation (** p < 0.05)
14
Research results : ERP and Business Knowledge
Correlation between ERP knowledge and performance
Correlation between Business knowledge and performance
Overall sample 0,05 NS –0,01 NSOnly top 10 % 0,410 * 0,496 **Only lower 10 % –0,258 NS –0,216 NS
Correlation between Performance and ERP and Business knowledge (n = 120)
(*) significant at 10%; (**) significant at 5%, (NS) Non significantThe correlation in the 2 last lines are non-parametric correlations.
Léger, P.-M., Charland, P., Robert, J., Cronan, P. et Babin, G., Cronan, P., (2010) « Assessing the dimensionality of an objective measurement of ERP knowledge », SAP Curriculum Congress, March 20th 2010
15
Research results : the role of GPA and complex knowledge
Léger, P.-M. et al. (unpublished). PLEASE DO NOT CITE.
Beta Sig.(Constant) 0.200 NSGPA -0.074 NSKnowledge Easy -0.259 NSKnowledge Complex 0.442 *Experience 0.538 ***R2 30,4 % ***Wald chi2 154.95 ***
* p<0.05, ** p<0.01, *** p<0.001
Regression (panels, n=480)
16
A platform for multi-method experimental research in IS
A platform for multi-method experimental research in IS
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Usage data
Psychophysiological data
Performance data
Psychometric data
Psychometric data
Psychometric data
Psychometric data
Psychometric data
Virtual time
Direct observation
18
Example of performance data in simulation game
Net profits
Virtual time
19
Example of usage data in SAP
20
Research results : Role transformation (1/2)
GR1 GR4 GR2 GR3
Integrated marketing Marketing functional Production planning SchedulingCosting_Report 1.000 0.390 2.080 0.220
Financial_Report 2.620 0.430 9.710 9.830
MRP_Action 0.100 0.000 4.920 1.070
Planning_Action 0.080 0.000 4.620 1.540
Procurement_Action 0.060 0.000 2.670 0.390
Procurement_Report 0.230 0.000 11.880 2.570
Prod_Scheduling_Action 0.150 0.040 9.210 9.650
Prod_Scheduling_Report 0.970 0.090 9.330 15.130
SalesMktg_Action 16.160 25.300 3.500 2.830
SalesMktg_Report 18.400 16.480 5.540 12.500
Stock_Report 24.940 1.390 19.170 22.780
Financial_Action 0.140 0.000 0.710 0.390
Cluster analysis (20 subjects, 9 quarters)
Léger, P.-M. et al. (unpublished). PLEASE DO NOT CITE . 21
Research results : Role transformation (2/2)
Quarter Inte
grat
ed M
arke
ting
Func
tiona
l mar
ketin
g
Inte
grat
ed p
lann
ing
Sche
dulin
g
Profi
t (st
anda
rdiz
ed)
1 2 1 1 0.472 2 2 -0.263 2 1 1 1.264 2 1 1 1.465 1 1 2 1.616 1 1 2 1.627 1 1 2 1.228 1 1 2 1.209 1 1 2 0.761 3 1 0.512 2 1 1 0.593 3 1 0.624 2 2 -1.185 2 1 1 -0.346 3 1 -1.047 1 2 1 -1.508 1 2 1 -1.499 2 1 1 0.56
1st p
ositi
on5t
h po
sitio
n
Léger, P.-M. et al. (unpublished). PLEASE DO NOT CITE .22
Psychophysiological data
ProComp Infinity from Thought Technology Ltd. 23
Psychophysiological data:Our start-up kit !
EEGSensor
EMGSensor
EKGSensor
EDA Sensor
TemperatureSensor
ProComp Infiniti
2 x 2 x 2 x 2 x 2 x289.00 $ / unit 289.00 $ / unit 210.00 $ / unit404.00 $ / unit 404.00 $ / unit
4043.00 $ / unit
24
Linking ERPsim and psychophysiological data
Performance data(virtual time, ERP
database)
Usage data(real time - ERP
database)
Psychophysiological data
(real time – laptop link to encoder)
Psychometric data and
observation data
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Virtual time line
Linking ERPsim and psychophysiological data (manual synchronisation)
26
Experimental platform Expected beta release date : Winter 2011
27
Demonstration
28
Research results : Cognitive absorption (1/3)
Léger,P.-M., Davis, F.D., Perret, J. et Dunaway, M.M., « Psychophysiological Measures of Cognitive Absorption Patterns », Working paper, ERPsim Lab, no 10-04(septembre 2010), HEC Montréal, 10 pages.
29
Research results : Cognitive absorption (2/3)
Survey Day 1: Only SAMSurvey Day 15: Only SAMSurvey Day 30 : SAM + CA
Q11 15 30
Discussion Q21 15 30
Discussion Q31 15 30
DiscussionGame 1
Q11 15 30
DiscussionGame 2
Q11 15 30
Discussion Q21 15 30
Discussion Q31 15 30
DiscussionGame 3
Q41 15 30
Discussion Q51 15 30
Discussion
Léger,P.-M., Davis, F.D., Perret, J. et Dunaway, M.M., « Psychophysiological Measures of Cognitive Absorption Patterns », Working paper, ERPsim Lab, no 10-04 (septembre 2010), HEC Montréal, 10 pages.
30
Cognitive absorption
Cognitive absorption
Curiosity
Control
Temporal dimension
Focus immersion
Heightenedenjoyment
The ability to register the passage of time while engaged in interaction
The experience of total engagement where other attentional demands are ignored
The pleasurable aspects of the interaction
The user’s perception of being in charge of the interaction
The extent to which the experience arouses an individual’s sensory and cognitive curiosity
(Agarwal and Karahanna, 2000)
31
SAM
Valence
Arousal
Dominance
(Lang, 1995)32
Research results : Cognitive absorption (3/3)
Léger,P.-M., Davis, F.D., Perret, J. et Dunaway, M.M., « Psychophysiological Measures of Cognitive Absorption Patterns », Working paper, ERPsim Lab, no 10-04 (septembre 2010), HEC Montréal, 10 pages.
Valence Arousal Dominance Temporal dissociation
Heighted Enjoyment Curiosity Focused
immersion Control Profit
Electrodermal response Corr. sig Corr. sig Corr. sig Corr. sig Corr. Sig Corr. sig Corr. sig Corr. sig Corr. sig
% EDA .248 ** .007 .119 -.135 .173 .202 .119 -.050 .174 *
Moving average
1 min .267 *** .061 .158 -.237 .154 .122 -.028 .071 .191 *
2 min .238 ** .033 .175 * -.158 .026 .005 -.131 .145 .151 ns
5 min .228 ** .040 .155 -.017 .024 -.055 -.170 .122 .096 ns
10 min .171 * .010 .075 -.032 .010 .011 -.151 .033 .113 ns
Moving standard deviation
1 min .088 .084 .016 .121 .008 .375 ** .255 -.244 .229 **
2 min .008 -.052 -.094 .134 -.030 .381 ** .314 * -.239 .177 *
5 min .035 -.056 -.018 .112 -.013 .271 .467 *** -.203 .222 **
10 min .031 .017 .129 .233 .134 .299 * .506 *** -.050 .265 ***
Correlation between EDA, psychometric measures and profits
SAM CA construct
33
Upcoming experiment at U. ArkansasOct – Nov 2010
Novice, Easy simulation
Chal
leng
es
Skills
Novice, Complex simulation
Expert, Easy simulation
Expert, Complex simulation
Cognitive absorption + SAM every 10 minutes
Skin conductanceEKG EEG
34
Upcoming experiment at U. Arkansas Oct – Nov 2010
35
ERPsim : A platform for multi-method experimental research in IS
Prof. Pierre-Majorique Léger, Ph.D.Director ERPsim Lab, HEC Montréal
Videoconference, University of Arkansas, Sept 17th 2010