community analytics – an information systems perspective

TeLLNet

Community Analytics –y yAn Information Systems Perspective

Ralf Klamma & ACIS GrouppRWTH Aachen University

Advanced Community Information Systems (ACIS)Advanced Community Information Systems (ACIS)[email protected]

CRIWG 2012 R f ld G S t b 17 2012Lehrstuhl Informatik 5(Information Systems)

Prof. Dr. M. JarkeI5-Klamma-0912-1 This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

CRIWG 2012, Raesfeld, Germany, September 17, 2012

Advanced Community Information Systems (ACIS)Systems (ACIS)

TeLLNet

Responsive Open

Community

Responsive Open

Community Community Visualization

d

Community Visualization

d

We

erin

gCommunity Information

Systems

Community Information

Systemsand

Simulationand

Simulation

ebAnangin

ee

Community Analytics

Community Analytics

Community Support

Community Support

lytics

Web

En

W

R i tLehrstuhl Informatik 5(Information Systems)

Prof. Dr. M. JarkeI5-Klamma-0912-2

RequirementsEngineering

AgendaAgenda

TeLLNetSy

stems

tics

look

nform

ation

unity

Ana

lyt

e Cas

es

ons &

Outl

mmun

ity In

Comm

u Us

Conc

lusi

Com

Lehrstuhl Informatik 5(Information Systems)


AbstractAbstract Information Systems serve the needs of organizations. With the

TeLLNety g

widespread availability of free Web-based tools and social networkingsites also communities with no institutional backing intensify the use ofth W b I thi t ti I ti t b l th tthe Web. In this presentation, I motivate by examples thatprofessional communities need community support beyond the commodity level Community analytics in such settings need a deepcommodity level. Community analytics in such settings need a deepunderstanding of interactions between community members and systems, members and resources as well as members among eachy gothers. Such a perspective is delivered by community informationsystems serving the needs of professional communities. The

i f l bi ti f tit ti d lit ti l timeaningful combination of quantitative and qualitative analyticsstrategies supports the understanding of community goals, communityprocesses and community reflection Case studies from ongoing EU



processes and community reflection. Case studies from ongoing EU research projects will support the argumentation.

TeLLNet

COMMUNITY INFORMATIONSYSTEMS



A Brief History of Community Information SystemsCommunity Information Systems

Organisational M iTeLLNet

Communities ofPractice

(Web 2.0)Business Processes

Memories(XML, HTML,

XTM)

Practice Business Processes

Semantic Web

Social Software

(XML, HTTP,

MetaData

Web(XML, RDF, Ontologien)

Groupware / E L i

Workflows(XML

RSS)

MediaTraces

E-Learning(XML, LOM, XML-RPC)

(XML, BPEL)

Digital MediaTechnology

Multimedia (XML, VRML, DC, MPEG)

Web Services(XML, WSDL, SOAP,UDDI)



Technology

Klamma: Social Software and Community Information Systems, 2010

Communities of PracticeCommunities of Practice Communities of practice (CoP) are groups of people

TeLLNetp ( ) g p p p

who share a concern or a passion for something they do and who interact regularly to learn how to do it g ybetter (Wenger, 1998)

Community Analytics SupportCommunity Analytics Support– How can CoPs record their complex complex media traces and

how they can deal with them?– Can CoPs continuously elicitate and implement requirements?

How much computer science support is needed? C C P l i f l di it l i l I t ti d k – Can CoPs learn meaningful digital social Interaction and make use of disturbances?

– Can CoPs maintain or even improve their agency (Learning Lehrstuhl Informatik 5(Information Systems)


Can CoPs maintain or even improve their agency (Learning, Researching, Working) in the Web 2.0?

TeLLNet

COMMUNITY ANALYTICS



Proposed Professional Development of the Community Analytics Fieldof the Community Analytics Field

Will happen Big Data by Digital Eco Systems (Quantitative Analysis) TeLLNet

pp g y g y ( y )– A plethora of targets (Small Birds)

– Professional Communities are distributed in a long tail– Professional Communities use a digital eco system

– An arsenal of weapons (Big Guns)– A growing number of community analytics methods– Combined methods from machine intelligence and knowledge representation

M t h D I l t ith it May not happen Deep Involvment with community(Qualitative Analysis)– Domain knowledge for sense making– Domain knowledge for sense making– Passion for community and sense of belonging– Community learns as a whole



→ Community Analytics for the Community by the Community

Interdisciplinary Multidimensional Model of CommunitiesModel of Communities

Collection of CoP Digital Traces in a MediaBaseTeLLNet – Post-Mortem Crawlers

– Real-time, mobile, protocol-based (MobSOS)(Automatic) metadata generation by Social Network Analysis – (Automatic) metadata generation by Social Network Analysis

Social Requirements Engineering with i* Framework for defining goals and dependencies in CoPfor defining goals and dependencies in CoP

Social SoftwareCross-Media Social Network Analysis on Wiki Blog Podcast

Network of ArtifactsContent Analysis on Microcontent, Blog entry, Message,

B t Th d C t C ti F db k (R ti )

Media Networks

Analysis on Wiki, Blog, Podcast, IM, Chat, Email, Newsgroup, Chat …

Web 2.0 Business

Burst, Thread, Comment, Conversation, Feedback (Rating)

Processes (i*)(Structural, Cross-media) Network of Members



Members(Social Network Analysis: Centrality,

Efficiency, Community Detection) Communities of practice

MediaBase: Cross Media / Cross Community SNACross Media / Cross Community SNA Post-Mortem Collection of ActorAttribute has

TeLLNet Social Software artifacts with parameterized PERL scripts

Blogs & Wikis

ActorAttribute has

isA

– Blogs & Wikis– Mails & Forums– Web pages

Member CommunityMedium(Social Software) Artifact

Database support by IBM DB2, eXist, Oracle, ...Web Interface based on Firefox Web Interface based on Firefox Plugin, Plone, Drupal, LAS, ...– www.learningfrontiers.eug– www.prolearn-academy.org

Strategies of visualizationLehrstuhl Informatik 5(Information Systems)


– Widget-based charts – Cross-media graphs

Klamma et al.: Pattern-Based Cross Media Social Network Analysis for Technology Enhanced Learning in Europe, EC-TEL 2006

Models of Community Success froman Information Systems Perspectivean Information Systems Perspective

TeLLNet

Reference Model: D&M IS Success Model (1992) – Based on >100 Empirical/Conceptual Studies p p– Validated by Independent Studies Updated

MobSOS Model: Integration of Future Web Concepts MobSOS Model: Integration of Future Web Concepts – Mobility– Real-Time



– Real-Time – Protocol-based (HTTP, XMPP, RESTful)

MobSOS Survey ModuleMobSOS Survey Module Testbed: MobSOS Survey

TeLLNetTestbed: MobSOS Survey Service– Survey ManagementSurvey Management– Survey Participation– XML/Relational DB SchemaXML/Relational DB Schema– Questionnaire XML Schema– Adaptive Templates– Adaptive Templates

Client: MobSOS SurveysS P ti i ti– Survey Participation

– Mobile ApplicationW b b d



– Web-based

MobSOS Success Model OverviewMobSOS Success Model Overview

TeLLNet



MobSOSTest beds Analytics & VisualizationTest beds, Analytics & Visualization

TeLLNet

Context-Aware Usage/Error StatisticsS i l N t k A l i Social Network Analysis Service Quality Analysis Visualization Set of MobSOS Widgets & Services interactive data mining visualization



Dominik Renzel, Ralf KlammaSemantic Monitoring and Analyzing Context-aware Collaborative Multimedia Services2009 IEEE International Conference on Semantic Computing, 14-16 September 2009 / Berkeley, CA, USA

visualization

Community Analytics in CoPCommunity Analytics in CoP User-to-Service Communication

TeLLNet • CoP-aware Usage Statistics• Identification of successful CoP services• Identification of CoP service usage patterns

User-to-User Communication User to User Communication • CoP-aware Social Network Analysis• Identification of influential CoP members• Identification of influential CoP members• Identification of CoP member interaction/learning patterns

+Lehrstuhl Informatik 5(Information Systems)


Supporting Community Practice with the MobSOS Success Modelwith the MobSOS Success Model

TeLLNet



Community SRE Processes–i* Strategic Rationalei* Strategic Rationale

TeLLNet



ROLE Requirements Bazaar –C it R i t P i iti tiCommunity-aware Requirements Prioritization

C it d d tC it d d tTeLLNet Community-dependentrequirements ranking lists

Community-dependentrequirements ranking lists

Factors influencingrequirements rankingFactors influencing

requirements ranking

User-controlled weightingUser-controlled weighting



of ranking factorsof ranking factors

TeLLNet

ROLE & TELMAP

CASE STUDIESROLE & TELMAP



Research Context: Responsive Open Learning Environments (ROLE)Learning Environments (ROLE)

TeLLNet

• Empower the learner to build their ROLE Vi i

Focus of key research objectives:

Empower the learner to build their own responsive learning environmentROLE Vision

• Awareness and reflection of own l i Responsiveness learning process Responsiveness

• Individually adapted composition of personal learning environment User-Centered



personal learning environment

Self Regulated LearningSelf-Regulated Learning

TeLLNet

learner profile informationis defined and revised

learner input regardinggoals, preferences, …

is defined and revised

learner finds and selectslearning resources

plancreating PLEevaluation and

self-evaluation

learner reflects and reacts on strategies, achievements, learning resources

learner works on selectedl i

learnreflectrecommendations

from peers or tutorsfeedback (from different sources)

g , ,and usefulness

learning resources

assessment andself-assessment

attaining skills using different learning events (8LEM)

it irecommen-dations

be aware of

learner should understand and control own learning process

ROLE infrastructure should provide adaptive guidance

monitoringbe aware of



Preparation forEnglish Language TestsEnglish Language Tests

Urch Forums (formerly TestMagic) User of clique

TeLLNet – Community on preparation for English language tests

– 120,000+ threads, 800,000+ posts, Thread 1 Thread 2

Non-clique User in threadClique-user missing in th d0,000 eads, 800,000 pos s,

100,000+ users over 10 years– Social Network Analysis, Machine

Learning and Natural Language Thread 3

thread

Learning and Natural Language Processing

What are the goals of learners?Intent Analysis (Phases 1 & 2) Time– Intent Analysis (Phases 1 & 2)

What are their expressions?– Sentiment Analysis (Phases 3 & 4)

Time

Refinement– 12881 cliques with avg. size 5 and

avg. occurrence of 14Lehrstuhl Informatik 5(Information Systems)


avg. occurrence of 14Petrushyna, Kravcik, Klamma: Learning Analytics for Communities of Lifelong Learners: a Forum Case. ICALT 2011

Self-Regulated Learning PhasesCan Be Observed in CommunitiesCan Be Observed in Communities

Phase 1 and 2 (low sentiment, questioner, lot of intents)Different users

TeLLNetPhase 1 and 2 (low sentiment, questioner, lot of intents)Phase 3 (increasing sentiment, conversationalist)Phase 4 (high sentiment, answering person)

1 week / step



40% of „footprints“ of cliques align with model for phases

Research Context: RoadmappingTechnology Enhanced LearningTechnology Enhanced Learning

TeLLNet

Mapping and roadmapping for TELMapping and roadmapping for TEL

Understanding the current TEL landscapeg p

Strong and weak signals for change at different levels

Different data sources

Different methods, e.g. Delphi, Community modeling, Text analysis Social Network Analysis etc



Text analysis, Social Network Analysis, etc.

TEL ProjectsTEL Projects

Project as a funded collaborative R&D effortTeLLNet

Project as a funded collaborative R&D effort Important role in the R&D value chain

Points of interest:Organizational – Organizational collaboration

– Progression of consortia

– Impact on the landscape



Data SetData SetProgr. Call # Projects (acronyms)

Call 2005 4 CITER, JEM, MACE, MELTTeLLNet

ECP

4 CITER, JEM, MACE, MELT

Call 2006 7 COSMOS, EdReNe, EUROGENE, eVip, Intergeo, KeyToNature, Organic.Edunet

Call 2007 3 ASPECT, iCOPER, EduTubePlusCall 2008 5 LiLa, Math-Bridge, mEducator, OpenScienceResources, OpenScoutIST-2002-2.3.1.12a 8 CONNECT, E-LEGI, ICLASS, KALEIDOSCOPE, LEACTIVEMATH, PROLEARN,

TELCERT, UNFOLD

FP6

,

IST-2004-2.4.10b 14 APOSDLE, ARGUNAUT, ATGENTIVE, COOPER, ECIRCUS, ELEKTRA, I-MAESTRO,

KP-LAB, L2C, LEAD, PALETTE, PROLIX, RE.MATH, TENCOMPETENCE

ARISE CALIBRATE ELU EMAPPS COM ICAMP LOGOS LT4EL MGBL UNITE IST-2004-2.4.13c 10 ARISE, CALIBRATE, ELU, EMAPPS.COM, ICAMP, LOGOS, LT4EL, MGBL, UNITE,

VEMUSICT-2007.4.1d 6 80DAYS, GRAPPLE, IDSPACE, LTFLL, MATURE, SCY

FP7 ICT-2007.4.3d 7 COSPATIAL, DYNALEARN, INTELLEO, ROLE, STELLAR, TARGET, XDELIA

ICT-2009.4.2b 13 ALICE, ARISTOTELE, ECUTE, GALA, IMREAL, ITEC, METAFORA, MIROR, MIRROR, NEXT-TELL, SIREN, TEL-MAP, TERENCE



Total: 77a … Technology-enhanced learning and access to cultural heritage”b … Technology-Enhanced Learning

c … Strengthening the Integration of the ICT research effort in an Enlarged Europe”d … Digital libraries and technology-enhanced learning”

TEL Projects as Social NetworksTEL Projects as Social Networks Projects x Organizations

TeLLNetj g

Project consortium progression Project consortium progression– Nodes: Projects

Ed O l f ti ROLEIMC, RWTH,

OU ZSI– Edges: Overlap of consortia(directed, weighted)

ROLE

TEL-MapOU, ZSI

Organizational collaborationN d O i i i– Nodes: Organiziations

– Edges: Collaboration in lti l j t

UniversityThe Open University

STELLAR, EUROGENE, ROLE, PROLEARN,

iCOPER, ASPECT



multiple projects(undirected, weighted) Leuven

KU Leuven

Consortium Progression NetworkConsortium Progression NetworkAt least 2 overlapping partnersAt l t 3 th ti b t j t t t d tTeLLNet At least 3 months time between project start dates

68 projects, 198 connectionsNode size proportional to weighted degreep p g g



Project Impact on the LandscapeProject Impact on the Landscape

TeLLNet

Successor projectsrelative to opportunity Cumulative fraction of successor



projects filled up with p's members

Derntl, Klamma: European TEL Projects Community. EC-TEL 2012.

Impact GraphImpact Grapht = 3 monthsAll programmes

TeLLNet k = 2

Node size proportional to impact

p grepresented, with

FP6 strongest

Best impact for money: PROLEARN, ICOPER,

GRAPPLE

All past networks of excellence among

top five ranks. Lehrstuhl Informatik 5(Information Systems)


Several running or recently completed projects

pThe two inaugural

NoEs on top

Expected Impact?Expected Impact? Correlation between weighted in-degree and impact

i i hTeLLNet

50

in progression graph Stronger incoming connections appears to lead to higher impact

404550

ICOPERSTELLAR

253035

In-D

egre

e

GRAPPLE

152025

Weig

hted

ASPECTLTFLL

05

10 MACE

Filter: Project start > 2005



00 0.1 0.2 0.3 0.4

Impact

Future GazingFuture Gazing

TeLLNet

505353

GALA

OpenScout

404550

e

ICOPERSTELLAR

OpenScout4646

253035

d In

-Deg

ree

GRAPPLE3232

ROLE

2929TEL MAPiTEC

152025

Weig

hted

ASPECTLTFLL

TEL-MAP2020

iTEC

05

10 MACE



00 0.1 0.2 0.3 0.4

Impact

In Degree “Expected Impact”In-Degree – “Expected Impact”

TeLLNet



Most Frequent CollaboratorsMost Frequent Collaborators

TeLLNet

1. PROLEARN (FP6): 16 pairs2 ICOPER (ECP): 10 pairs

4. GRAPPLE (FP7): 8 pairs, 5 STELLAR (FP7) ROLE (FP7)



2. ICOPER (ECP): 10 pairs3. OpenScout (ECP): 9 pairs

5. STELLAR (FP7), ROLE (FP7), PROLIX (FP6): 5 pairs

Projects Space @ LearningFrontiers euLearningFrontiers.eu

TeLLNet



TEL Mediabase Dashboardhttp://learningfrontiers.eu/?q=dashboard

TeLLNet



Derntl, Erdtmann, Klamma: An embeddable widget-based dashboard for visual analytics on scientific communities. I-KNOW 2012

AdvancedCommunity Information SystemsCommunity Information Systems

• SNA• LAS & ServicesTeLLNet

• Network Models• Advanced

SNA• Widgets

LAS & Services• ROLE Sandbox

ResponsiveOpen

ResponsiveOpen Community Community Models

• Network Analysis

• Actor Network Theorycs

AdvancedWeb & Multimedia Technologies• XMPPrin

g

Open Community

Environments

Open Community

Environments

yVisualization& Simulation

yVisualization& Simulation

• Communities ofPractice

• Game Theory• Community

DetectionAnaly

ti• HTML5• MPEG-7

• Web Services

RESTf lngine

er

• MediaBase• Requirements

Community Analytics

Community Analytics

Community Support

Community Support

Detection• Web Mining• Recommender

Systems• Multi Agent

Sim lation

Web

A• RESTful• LAS

• CloudComputingMobile W

eb E • MediaBase

• MobSOS• Requirements

Bazaar

Simulation• Mobile Computing Social Requirements Engineering

Agent and Goal Oriented i* ModelingLehrstuhl Informatik 5(Information Systems)


• Agent and Goal Oriented i* Modeling• Participatory Community Design

community analytics – an information systems perspective

Education