components and functions of crowdsourcing systems 1/wi2013... · components and functions of...
TRANSCRIPT
Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement
Lars Hetmank Dresden, 27.02.2013
Components and Functions of Crowdsourcing Systems
A Systematic Literature Review
Components and Functions of Crowdsourcing Systems Slide 2 / 13
Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop
Short Vita
• Dipl.-Wirt.-Inf. Lars Hetmank, TU Dresden • Research interests:
• Information and Knowledge Management • Semantic Web and Linked Data • Crowdsourcing
• Affiliations: • Manifesto for a Standard on Knowledge Exchange in
Social Knowledge Management Environments • Knowledge Research Center (KRC) e.V. Dresden
10.03.13
Components and Functions of Crowdsourcing Systems Slide 3 / 13
Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop
Motivation
Practical relevance: • development of a crowdsourcing system (CSS) and
its integration into existing IT infrastructure is risky, cost-intensive, and time-consuming,
• structured development process required Theoretical relevance: • various theoretical contribution in the area of
crowdsourcing • however, little has been investigated to define a
crowdsourcing system and its technical design aspects precisely
10.03.13
Components and Functions of Crowdsourcing Systems Slide 4 / 13
Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop
Method: Systematic Literature Review
10.03.13
Research Questions (RQs): RQ1: How and in which detail are CSSs defined in current research
literature? What design aspects do they cover? RQ2: What type of components and functions of a CSS can be
conceptualized?
Review Protocol
Plan Systematic Literature Review
Selected Studies
Relevant Studies
ConductSearch
ScreenPapers
ConceptMap
ExtractData
Result
Process
Components and Functions of Crowdsourcing Systems Slide 5 / 13
Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop
Search Strategy #1
• Population: peer-reviewed conference proceedings and journal papers in English since 2006; dissertations, newspaper articles, unpublished works or non-scientific articles were not considered
• Search Resources: ACM Digital Library, Ebscohost (Academic Search Complete and Business Source Complete), Emerald, IEEE Xplore Digital Library, Sage Journals, ScienceDirect, SpringerLink and Wiley
• Search Terms: crowdsourcing system, crowdsourcing application, and crowdsourcing platform (in singular and plural)
10.03.13
Components and Functions of Crowdsourcing Systems Slide 6 / 13
Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop
Search Strategy #2
• Inclusion criteria: • define or at least propose a description of what CSSs are
(RQ 1) • address design issues of CSSs (RQ 2) • or classify or give an overview of CSSs (RQ 2).
• Exclusion criteria: • use CSSs for evaluation purposes but do not address design
issues
10.03.13
Components and Functions of Crowdsourcing Systems Slide 7 / 13
Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop
Conduct Search Step 1: a) review the abstract, introduction, and conclusion
b) check inclusion and exclusion criteria c) label publication and research type à Relevant studies = 72
10.03.13
Step 2: a) read article, find and record the definitions of CSS b) set keyword for each article to identify components and functions of CSSs
Publica(on type Research type
Components and Functions of Crowdsourcing Systems Slide 8 / 13
Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop
Results: Crowdsourcing System Definitions (RQ1)
• 17 definitions or descriptions of the term crowdsourcing system, application, or platform
• vary in detail and none of them cover all of the four derived perspectives
10.03.13
“Crowdsourcing Platform is a trusted broker ensuring that providers successfully complete the task requests and that requestors pay for the charges. Crowdsourcing Platform issues authentication credentials for requestors and providers when they join the platform, stores details about skill-set, history of completed requests, handles charging and payments, and manages platform misuse. Crowdsourcing platform can execute crowdsourcing requests in a number of different modes, by advertising them on the marketplace, allowing providers to bid for them, or in the form of a competition, where requestor identifies criteria to be used for selection of the winning submission. Crowdsourcing platform may further allow requestors and providers to team-up.” (Vukovic, 2009)
Components and Functions of Crowdsourcing Systems Slide 9 / 13
Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop
Perspectives
10.03.13
Crowd-sourcing System
organiza
tional
perspec
tive process
perspective
human-ce
ntric
perspec
tive
technical
perspective
broker
requester
crowd
APIs
user interface
standardizationhistory tracking
authenticationworkflow support
payment mechanism
crowdsourcingprocess
social interaction
mediator, marketplace, interface, or trusted broker
software components, technical functions, or data objects
human brainpower and collective intelligence
algorithm
detailed actions performed to data objects or users
Perspec(ves of crowdsourcing systems (source: own illustra(on)
Components and Functions of Crowdsourcing Systems Slide 10 / 13
Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop
Results: Components and Functions (RQ 2)
10.03.13
Register user
Evaluate user
Form user group
Enable coordination
User management
Design task
Assign task
Task management
Evaluate contribution
Select contribution
Contribution management
Define workflow Manage workflow
Workflow management
Components and func(ons of crowdsourcing systems (source: own illustra(on)
Components and Functions of Crowdsourcing Systems Slide 11 / 13
Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop
Conclusion and Next Steps
• development of an integrated CSS definition • overview of reusable software components
and functions may be used as a checklist • dependencies between the elements are not
well represented • designing a semantic model to support
knowledge-intensive crowdsourcing activities
10.03.13
Components and Functions of Crowdsourcing Systems Slide 12 / 13
Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop
References
(1) Kitchenham, B.: Guidelines for performing Systematic Literature Reviews in Software Engineering. (2007).
(2) Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., Linkman, S.: Systematic literature reviews in software engineering – A systematic literature review. Information and Software Technology. 51, 7–15 (2009).
(3) Brereton, P., Kitchenham, B. a., Budgen, D., Turner, M., Khalil, M.: Lessons from applying the systematic literature review process within the software engineering domain. Journal of Systems and Software. 80, 571–583 (2007).
(4) Wieringa, R., Maiden, N., Mead, N., Rolland, C.: Requirements engineering paper classification and evaluation criteria: a proposal and a discussion. Requirements Engineering. 11, 102–107 (2006).
(5) Bortz, J., Döring, N.: Forschungsmethoden und Evaluation für Human- und Sozialwissenschaftler. Springer, Heidelberg (2009).
10.03.13
h>p://larshetmank.com/documents/wi2013_css_concept_map.pdf
Components and Functions of Crowdsourcing Systems Slide 13 / 13
Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop
Contact
Email: [email protected] Twitter: https://twitter.com/larshetmank
10.03.13