information systems success awareness for professional long tail communities of practice

Lehrstuhl Informatik 5(Information Systems)

Prof. Dr. M. JarkeI5-Renzel-0716-1 This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International Public License.

Information Systems Success Awareness for Professional

Long Tail Communities of PracticeDipl.-Inform. Dominik Renzel

Doctoral Thesis DefenseChair of Computer Science 5 – Information Systems & Databases

RWTH Aachen University

Aachen, GermanyJuly 12, 2016

https://creativecommons.org/licenses/by-sa/4.0





http://dbis.rwth-aachen.de/cms/research/MiCIS

http://creativecommons.org/licenses/by-sa/3.0/


Prof. Dr. M. JarkeI5-Renzel-0716-2

Agenda

IntroductionMobSOS CIS Success Awareness Framework

– Methodology– Formalization– Technical Infrastructure

Longitudinal Studies & FindingsConclusion & Outlook





IntroductionIntroduction

Motivating Examples Terminology Research Problem & Questions





Small-Scale Example – Aphasics Group Chat Tool (DESRIST 2015)

Success?Quality? Impact?

Simple?Responsive?Reliable?Deployable?

Helps regain skills?Avoids isolation?Augments therapy?Unforeseen effects?

Success

Success Success

Introduction





How to measure & stay aware of success in/across highly distributed

& heterogeneous communities using mobile Web technologies on

a massive scale over time?

Large-Scale Heterogeneity & Dynamicity – ROLE Sandbox (EC-TEL 2015)

Introduction

Success? Success?Success?





Community of Practice, Long Tail & Community Information Systems

Community of Practice (CoP) [Weng98]

Technology

People

Rules

“A group of people who share a concern or a passion for

something they do and learn how to do it better as they

interact regularly.”

[Ande06]

Introduction

Learning requires reflection & awareness of CIS Success!





Related Work

Web Science

IS Success

IS Design

Research Framework [HMPR04]

Methodology [PTRC07,SHP*11]

Models [DeMc92,GSCh08,CKAb14]

Formalization [JMPo03,SBGe04]

Web Engineering [GJHV95,Fiel00] Web Analytics [Kaus10,CCSt12,Evan12]Introduction





Related Work

Web Science

IS Success

IS Design

Research Framework [HMPR04]

Methodology [PTRC07,SHP*11]

But: implicit success notion Models [DeMc92,GSCh08,CKAb14]

Formalization [JMPo03,SBGe04]

But: positivist paradigm,myopic financial focus

Web Engineering [GJHV95,Fiel00] Web Analytics [Kaus10,CCSt12,Evan12]

But: analytics infrastructure & strategy corporate secret

Introduction





General Idea

For us, success is…

Introduction

From main-stream, centralized & secret corporate business analytics to long-tail, scalable, distributed, open negotiation of CIS success





Research Questions & Contributions

RQ1: How can professional long tail communities of practice achieve a shared sense of CIS success awareness with the help of a media-supported and negotiation-based methodology?

RQ2: What formalization should communities use to model CIS success? How can communities formalize the assessment, negotiation, and validation of different notions of CIS success?

RQ3: How should technical infrastructure for community information systems be designed to achieve integrated support for CIS success awareness?

MobSOS CIS Success Awareness Framework

Introduction





MobSOS MethodologyMobSOS

Methodology

Community-Oriented Design Research Media-Supported CIS Success Negotiation

RQ1: How can professional long tail communities of practice achieve a shared sense of CIS success awareness with the help of a media-supported and negotiation-based methodology?





Design Science Research in Information Systems [HMPR04]

MobSOSMethodology





Community-Oriented Design Science Research (DESRIST 2015)

MobSOSMethodology





CIS Success Modeling & Negotiation as Media Transcription [JJKS08,Klam10]

MobSOSMethodology





MobSOS Formalization Framework CIS Success Model Formalization CIS Success Model Validation

RQ2: What formalization should communities use to model CIS success? How can communities formalize the assessment, negotiation, and validation of different notions of CIS success?

MobSOSFormalization

Framework





Formal CIS Success Models – Structural Model (DESRIST 2015)

MobSOSFormalization

FrameworkExplo

re

Validate

Adapt

IS-Impact Model, Formative Index [GSCh08,JMPo02]





Formal CIS Success Models – Measurement Model

Model:

Metrics:

MobSOSFormalization

Framework





CIS Success Model Validation – Ordinal Logistic Regression

Literature: multiple (linear) regression [DiWi01,PSRa07]

But: metrics violate assumptions (linearity, homoskedacity, normality, scales) Better: ordinal logistic regression [Agre01]

Interpretation: odds & probabilities Statistics:

– Model: fit– Effect Strengths– Significances

MobSOSFormalization

Framework





MobSOS Technical Infrastructure CIS Success-Aware Community Platform MobSOS Service Tool Kit

RQ3: How should technical infrastructure for community information systems be designed to achieve integrated support for CIS success awareness?

MobSOSTechnical

Infrastructure





Success-Aware CIS Platform (WISMA 2008, ICSC 2009)

MobSOSTechnical

Infrastructure





Usage Data Collection & Enrichment – MobSOS Pipeline (LAK 2013)

MobSOSTechnical

Infrastructure

Reverse Proxy & Pipeline Pattern: [Somm02,GHJV95]





Exploration by Visual Analytics – MobSOS QV (I-Know 2012)

Visualization Authoring Visual AnalyticsWidgets & Dashboards

MobSOSTechnical

Infrastructure





Evaluation Products for Awareness – MobSOS Success Model Service

MobSOSTechnical

Infrastructure





Longitudinal Studies & FindingsCase Studies

&Findings

2004 – 2015: SOCRATES – Aphasia Therapy (DFG SFB 427) 2008 – 2016: ROLE SDK – Self-Regulated Learning (ROLE) (6 more studies in thesis)





SOCRATESX – Aphasia Therapy(DESRIST 2015)

Case Studies&

Findings





SOCRATESX – Aphasia Therapy(DESRIST 2015)

Group chat tool incl. word completion

German aphasia patients (N>200) + therapists, linguists, computer scientists

Chat protocol logs & survey responses(1 yr.)

Compact success model? Mineable from protocol logs? Benefits?

Case Studies&

Findings





SOCRATESX – Optimizing Designwith System Quality Metrics

Case Studies&

Findings

High hit rate (HR) & low keystrokes until prediction (KUP)

5 candidates good size for word completion list





SOCRATESX – Measuring Individual & Community Impacts

Patients improve individual performance Community as whole improves performance Positive impacts traceably measurable

Case Studies&

Findings





SOCRATESX CIS Success Model (DESRIST 2015)

Case Studies&

Findings





CIS Success Models – General Target Characteristics

Sharply focused scope

Handful of most relevant factors

Balance universal vs. community-specific factors

Majority of quality & impact dimensions reflected

Actionable support for agency

Case Studies&

Findings





ROLE Sandbox – Analyzing Self-Regulated Learning (LAK 2013, ICWL 2015)

Background: planning, building & reflection of personal learning environments (PLE) essential activities for self-regulated learning (SRL) [Zimm02,NKR*14a].

Case Studies&

Findings





ROLE Sandbox – Analyzing Self-Regulated Learning (LAK 2013, ICWL 2015)

PLE management platform, learning spaces & learning widgets

Widget developers, learners, teachers, researchers (N>10.000)

Context-enriched PLE management logs (>3 yrs.)

Self-regulated learning activity traceable in PLE? Learner classification? Recommendations?

Case Studies&

Findings





Category-Based Recommendations for Self-Regulated Learners (ICWL 2015)

In SRL-Spaces: Intense use of recommendations Interesting evaluation contexts Improve recommendations by complete categorization

Case Studies&

Findings





Measuring Networking Impact of ROLE SDK with SNA Metrics

Case Studies&

Findings





Conclusion & Outlook

Conclusion&

Outlook





Major Contributions

Anti-positivist methodology for media-based CIS success negotiation as guidance for CIS design

Digital media representation & validation of fluid CIS success models capturing qualitative & quantitative aspects from multiple heterogeneous data sourcesRobust, scalable & extensible Open Source Software framework based on established patterns & cutting-edge mobile Web technologies

MobSOS CIS Success Awareness Framework





Possible Avenues for Future Work

Conclusion&

Outlook

Standard ACIS thesis evaluation framework

Social software for CIS success negotiation

Integrated early-phase RE & CIS success modeling

CIS success-driven recommender systems

Automated metric exploration by machine learning

Cross-community CIS success analytics

