interim report on promoting citizen ......interim report on promoting citizen participation in...
TRANSCRIPT
Vrije Universiteit Brussel
D2.1a - Interim Report on Promoting Citizen Participation in CampaingsSeymoens, Tom; Hoelck, Katharina; Bleumers, Lizzy; Lievens, Bram
Publication date:2017
License:Other
Document Version:Final published version
Link to publication
Citation for published version (APA):Seymoens, T., Hoelck, K., Bleumers, L., & Lievens, B. (2017). D2.1a - Interim Report on Promoting CitizenParticipation in Campaings: Flamenco Project Deliverable 2.1a. Unknown.
General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.
• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portalTake down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.
Download date: 01. Jan. 2021
INTERIM REPORT ON PROMOTING
CITIZEN PARTICIPATION IN
CAMPAIGNS
FLAMENCO Project Deliverable 2.1a
VUB (SMIT)
Responsible SMIT (VUB)
Authors Tom Seymoens (imec-SMIT, VUB), Katharina Hölclk (imec-SMIT, VUB), Lizzy Bleumers
(imec-SMIT, VUB), Bram Lievens (imec-SMIT, VUB).
Promoting participation – interim report 2
TABLE OF CONTENTS
1 Introduction ...................................................................................................................... 4
2 Chapter 1: An Exploration of the Field: Citizen Science.............................................. 5
2.1 A Brief History and Definition of Citizen Science ..............................................................5
2.1.1 Citizen Science vs Related Concepts ...............................................................................6
2.1.2 Participatory Sensing .......................................................................................................8
2.2 Classifications of Citizen Science Projects ..........................................................................9
2.2.1 Classification of Bonney et al. (2009) .............................................................................9
2.2.2 Classification of Haklay (2013) .....................................................................................10
2.2.3 Classification of Wiggins and Crowston (2011) ...........................................................11
2.2.4 The Flamenco Platform as a Supporting Layer Across Typologies ..............................13
2.3 Lessons Learned for Flamenco ...........................................................................................15
3 Chapter 2: Initiating a Citizen Science Project: Reasons and Hurdles ..................... 17
3.1 Motivations for Choosing Citizen Science .........................................................................17
3.1.1 To contribute to science ............................................................................................17
3.1.2 To inform policy .........................................................................................................17
3.1.3 To educate ....................................................................................................................18
3.1.4 To improve buy-in ......................................................................................................18
3.1.5 To raise awareness and engage people ....................................................................18
3.1.6 To build partnerships and improve communication between different
stakeholders .................................................................................................................................18
3.1.7 To gain personal satisfaction ....................................................................................18
3.2 Barriers Inhibiting the Use of Citizen Science ..................................................................20
3.2.1 Data Quality and Biases ............................................................................................20
3.2.2 Requirement of Specialist Equipment / Knowledge .............................................20
3.2.3 Politics ..........................................................................................................................20
3.2.4 Unaware of audience ..................................................................................................20
3.2.5 Research design and implementation issues ..........................................................20
3.2.6 Participant Involvement and Identification ............................................................21
3.2.7 Time consuming and Resourcing issues .................................................................21
3.2.8 Peer review / Mistrust ................................................................................................21
3.2.9 Uncomfortable / unprepared to work with the public ...........................................21
3.3 Overcoming the Barriers for CS Project Initiation & Flamenco ....................................22
Promoting participation – interim report 3
4 Chapter 3: Participation in a Citizen Science Project: Motivations and Hurdles.... 24
4.1 Participation: Best Practices ...............................................................................................24
4.2 Factors Affecting Participation ..........................................................................................26
4.2.1 Motivations for Participation in Citizen Science Projects .............................................27
4.2.2 Organizational factors & Participation in Citizen Science Projects ..............................30
4.3 A Segmentation of Participants in Citizen Science Projects ............................................33
4.4 Factors that Benefit Participation & Flamenco ................................................................35
5 Gamification and Behavioral Change........................................................................... 38
5.1 Introduction: Gamification in the context of Behavioral Change ...................................38
5.2 Definition: Gamification and its elements .........................................................................40
5.3 Application: The potential of gamification for citizen science ........................................45
6 Conclusion ....................................................................................................................... 47
7 Bibliography .................................................................................................................... 51
1 INTRODUCTION
The Flamenco-project aspires to develop an open, reusable and reconfigurable citizen observatory
platform for Flanders. This platform should enable a myriad of different stakeholders to launch a
participatory sensing campaign on their topic of choice, each with its specific technological
prerequisites (e.g. the type of data gathered, its source, how it should be stored, etc.) and unique social
circumstances.
Part of the project1 addresses questions surrounding the latter, by taking on two broad objects of study.
Part of the research is reserved to an in-depth analysis of current practices, opportunities and
challenges surrounding environmental and mobility-related citizen science campaigns. This document
targets the second research area and focuses on concepts of motivation and engagement2. It commits
to an investigation, by means of an extensive literature review, into the key mechanisms that motivate
citizens to partake in participatory sensing campaigns.
Chapter one of this interim report starts with defining and differentiating citizen science (CS),
participatory sensing and other related concepts. Afterwards three complementary typologies of CS
projects are presented. In chapter two, we zoom in on the CS organizer’s point of view and question
the reasons for and against the application of the CS methodology. In chapter three we shift focus
toward the participant as we in succession digest the concept of participation, how their motivation
and the organization of the project influences contribution and a segmentation of participants. The
fourth and final chapter of this document is dedicated to behavioral change and gamification. First, an
introduction to general behavior change approaches is given. Gamification is framed as one of several
tools which can be used as part of strategies to steer behavior. Then, the concept is explored in further
depth. Important elements of gamification are outlined. Finally, the application of the concept within
the context of citizen science is discussed.
This report ends with an enumeration of lessons learned from this extensive literature review for
organizers of citizen science projects. As a follow-up we will examine how the distinguished concepts
will be applied in a framework for analyses of CS case studies. The latter will be comprehensively
presented in the next iteration of this report.3
1 Work package 2: from society to technology
2 This relates specifically to task 2.1 of the project (Determinants for motivation and engagement)
3 Integrated report on promoting citizen participation (D2.1b)
Promoting participation – interim report 5
2 CHAPTER 1: AN EXPLORATION OF THE FIELD: CITIZEN SCIENCE
2.1 A BRIEF HISTORY AND DEFINITION OF CITIZEN SCIENCE
The first use of the Citizen Science (CS hereafter) concept is attributed to the National Audobon
Society. Back in January 1989, Audobon set up a research program involving 225 citizen volunteers
for the monitoring of acid rain (Gharesifard, Wehn, & Van Der Zaag, 2017). CS gained more traction
in the mid-‘90s under the influence of two authors: Alan Irwin and Rick Bonney (UWE, 2013). Alan
Irwin applied the concept to articulate his sense of urgency to cease the depreciation of lay people’s
expertise and judgement, and rather to endorse more contextual forms of knowledge and
understanding (Irwin, 1995). The ambition is to end up with a more responsive science, by bringing a
greater understanding between the public and scientific sphere (Dickel & Franzen, 2016). Rick
Bonney, on the other hand, approaches CS as a method which enables the general public to contribute
to science through data gathering (Riesch & Potter, 2014; UWE, 2013).
Geoghegan et al. (2016) have established eight principles that surround the concept of CS.
Interestingly, they also allow non-scientific stakeholders from policy and practice communities to
influence or even initiate the different phases of a CS project. The eight principles are the following
(Geoghegan et al., 2016, p. 23):
1. Widening participation in science;
2. Recognizing benefits of participation to citizen;
3. Leading to measurable academic output and/or being used by citizens;
4. Harnessing emotional attachments to particular subjects;
5. Carrying out activities across a range of skills levels
6. Sharing data between experts (paid and voluntary);
7. Prioritizing science over engagement; and
8. Talking about ‘science’ only without separating citizen science and traditional science.
Public participation has been a part of the scientific tradition for a long time, through the extended use
of, amongst other things surveys, interviews and focus groups. CS can be seen as a natural
progression from this practice through its central characteristic of active engagement of the public
(Wiggins & Crowston, 2011).
Based on these previous attempts, we define CS in this report as the intentional and active
engagement, in a non-professional capacity of volunteers in the scientific process (Pocock, Tweddle,
Savage, Robinson, & Roy, 2017, p. 1). More specifically their contribution is possible in the stages of
research initiation and design, data collection, analysis, interpretation and dissemination of the results
(Bonney et al., 2009; Geoghegan et al., 2016; Pocock et al., 2017, p. 1; Rotman et al., 2012). CS,
defined in this manner, acknowledges the value of local and non-expert knowledge to inform public
Promoting participation – interim report 6
consultation and engagement in the scientific process, adhering to Irwin’s definition. Section 2.2.3
discloses that public science education is often a (non-)intended consequence of CS through the active
participation of the public. This is expected to have a positive influence on the relationship between
science and society (Martin, 2017), which brings us closer to Irwin’s ambition. The proposed
definition is also allegiant to Bonney’s definition, but expands it by allowing the contributor’s
involvement in the different phases of the scientific process.
CS brought forth the creation of a new type of researcher, the citizen scientist. Whereas the métier of
the scientist used to be restricted to individuals with an elaborate training and the associated
qualifications, citizen scientists are defined by Rotman et al. (2012, p. 1) as “individuals who typically
lack formal credentials and do not hold professional positions in scientific institutions or projects,
who participate in scientific endeavors related to their personal interests”. The segmentation of
Martin (2017), slightly nuances this definition as we will see in section 4.3 as people that work in
academia or have enjoyed tertiary education seem more likely to participate in citizen science projects.
CS has already been proven beneficial in a multitude of domains, such as biology, environmental
studies, astronomy, chemistry and mathematics (Rotman et al., 2012). Several societal and
technological evolutions have contributed to this potential of CS. From a technological point of view,
there has been the emergence of distributed technologies in the last fifteen years that enable the
conduction of “scientific inquiries on a larger scale than ever before by extending beyond
geographically collocated tasks and enabling distributed collaboration” (Beza et al., 2017; Rotman et
al., 2012, p. 1). This technological innovation allows for improved data flows and better feedback to
volunteers (Geoghegan et al., 2016). From a sociological point of view, increasing literacy and
education have supported the citizens’ growing capabilities and self-confidence for contributing in
scientific research (Mccrory, Veeckman, Coppens, & Claeys, 2017). Although further research is
needed, other contributing sociological evolutions might be distinguished. One might wonder, for
instance, how a shift in civic engagement from a dutiful citizen to an actualizing citizen as presented
by Bennett (2008) has an effect on a greater aspiration for participating in the scientific community.
CITIZEN SCIENCE VS RELATED CONCEPTS
A lot of different concepts are closely related to the one of citizen science, they are often used
interchangeably and as a consequence their meaning gets intertwined. In this section we provide some
clarity on the matter, based on the overview created by See et al. (2016). Their inquiry presented the
different terms that relate to the subject of citizen-derived geographical information. We only refer
here to the most important concepts.
Concept Definition
Promoting participation – interim report 7
Public Participation in Scientific Research In the analysis of Bonney et al. (2009), that is
discussed in-depth in the next paragraph, extensive use
is made of the concept “public participation in
scientific research” (PPSR). This concept denotes the
level of public involvement in the different research
steps. See et al. (2016) consider this to be equivalent to
the Citizen Science concept.
Science 2.0 Science 2.0 refers to the process where the increasingly
global, collaborative, interdisciplinary nature of the
scientific inquiry is supported by IT, the internet and
mobile devices. See et al. (2016) notice how the greater
potential of inclusion of citizens is only one component
in science 2.0
Crowdsourcing Crowdsourcing entails the act of addressing a
heterogeneous group of individuals through an open
call for voluntarily taking part in a specific task of
varying complexity. The term is more restrictively
applied to purely online activities, that entail a mutual
benefit for the participant (e.g. economic, social
recognition, self-esteem, development of individual
skills) and the crowdsourcer (the resources of the
participant). It differs itself in this sense from citizen
science as their calls for actions are not necessarily
open (e.g. restricted in geographical location, skills
required, time), nor is the mutual benefit implied as the
data collected might be used for other citizen science
purposes later (See et al., 2016).
Citizens’ Observatory Citizens’ observatory can be defined as a process of
joint observation and mutual information and resources
transfer. It thus “[…] promotes communication and the
sharing of technological solutions (e.g., sensors, mobile apps, web portals) and community participatory
governance methods among citizens” (Liu, Kobernus,
Broday, & Bartonova, 2014, p. 4). A citizen
observatory platform makes sure that resources such as
the data, sensors, issues, communication, guidelines
and reports can be shared in a two-way model between
participants and between participants and organizers.
Citizens’ observatory can be seen as an underlying, supporting infrastructure for citizen science projects,
that can in itself make use of participatory sensing data
collection techniques.
Participatory Sensing Participatory sensing has to be seen as a method of data
collection, and can thus be a component of
crowdsourcing or a contribution to an overall citizen
science project (which also takes into account the other
research tracks such as analysis). More specifically,
participatory sensing is data collection and knowledge
sharing by way of smart mobile devices, “[…]
deployed as part of an interactive participatory sensor
network” (See et al., 2016, p.7).
TABLE 1: RELATED CONCEPTS OF CS (BASED ON SEE ET AL. (2016)
Promoting participation – interim report 8
Through inaccurate use, the meaning of the different concepts can easily be vulgarized. The common
elements are the connotation they bear of a heightened (potential of) citizen involvement and its
mediation or facilitation by information technology. The degree of both varies in the different
approaches.
Participatory sensing, the last concept in the list, is of paramount importance for the Flamenco-project.
For this reason, we elaborate further on the concept in the next section.
PARTICIPATORY SENSING
Urban sensing can encompass both the use of sensors embedded in a city’s infrastructure, as well as
sensing that relies on the mobile devices that people carry with them as they move around in the city
(Chatzigiannakis, Mylonas, & Vitaletti, 2011). The latter interpretation of urban sensing is often
referred to as people-centric sensing (Lane, Eisenman, Musolesi, Campbell, & Miluzzo, 2008). Lane
and colleagues (2008) distinguish a range of approaches to people-centric sensing based on the extent
to which people, who are using the device, are aware of the sensing process and involvement in the
decision-making process related to the sensing system. Participatory sensing is situated at one end of
the spectrum, and opportunistic sensing at the other end.
Lane and colleagues describe participatory and opportunistic sensing as follows (2008).
Participatory sensing means people knowingly choose to meet requests made by the application on
the device they are carrying. Furthermore, they are involved in choices, for example, on what data is
being shared. Opportunistic sensing, on the other hand, refers to approaches where users of the
mobile device may not be aware of sensing applications, as these are running in the background and
do not require the user to take any action to meet application requests. Here, the focus is on having
sensing take place in a non-intrusive way so that people can use their device for their own purposes,
instead of being actively involved in sensing and associated decision-making. The definition of
participatory sensing highlights that the phenomenon is defined not solely from a technological
perspective – how sensing is implemented – but also from a social perspective – the way people are
engaged and participate in it.
Haklay (2013) considers participatory sensing as a relatively new expression of citizen science. Like
‘volunteered computing’, where people allow projects to use the computing power of their personal
devices (e.g. SETI@home) and ‘volunteered thinking’, where people voluntarily engage in scientific
analysis (e.g. FoldIt), Haklay groups participatory sensing under the category of citizen cyberscience,
a variant of citizen science relying on affordances of technology (such as the Internet, personal
computers and mobile devices, sensors, …) as a scientific tool.
Promoting participation – interim report 9
2.2 CLASSIFICATIONS OF CITIZEN SCIENCE PROJECTS
In the previous section, we outlined the concept of CS and how it distinguishes itself from related
concepts, such as participatory sensing. In what follows we aspire to take a closer look at
classifications of citizen science projects. As the Flamenco-platform aspires to host projects with
different scopes, this overview provides a basis for understanding what can be expected. Furthermore,
it allows for a better comparative evaluation of existing use cases, which in itself grants us a greater
comprehension of the different elements underpinning success and/or failure of the CS projects
(Pocock et al., 2017). We successively glance over the classifications of Bonney et al. (2009), Haklay
(2013) and Wiggins and Crowston (2011). In the final part of this section, we attempted to connect
the different classifications.
CLASSIFICATION OF BONNEY ET AL. (2009)
An often referred to classification of CS projects is provided by Bonney et al. (2009). They have
made the distinction based on the degree of public involvement in the different research activities,
which they broke down as follows:
1. Choosing or defining questions for study;
2. Gathering information and resources;
3. Developing explanations (hypotheses) about possible answers to questions;
4. Designing data collection methodologies (both experimental and observational);
5. Collecting data;
6. Analysing data;
7. Interpreting data and drawing conclusions;
8. Disseminating conclusions; and
9. Discussing results and asking new questions
They consequently came up with the following subcategories (Bonney et al., 2009, p.11; Mccrory et
al., 2017; Rotman et al., 2012):
a) Contributory projects: These types of projects are organized by researchers; the public
participates by gathering the required data. The use of the public is advantageous since the
necessary data needs to be collected over “wide geographic areas or over long spans of time”
(Bonney et al., 2009, p. 18). An example given by the authors is the FeederWatch-project, for
which the participants provided data on the birds in their backyard, the researchers then
analyze the data.
b) Collaborative projects: Similar to the contributory projects, the research is designed by
scientists, the public’s involvements however can also include the analysis, interpretation and
presentation of the data to others. The example given by Bonney et al. (2009) is that of water
quality monitoring projects, where the participants not only collect the samples, but also
perform analyses and present their findings towards local government agencies.
c) Co-created projects: Contrary to both collaborative and contributory, in co-created projects
it is the public itself that have questions that need to be answered. Together with scientists
Promoting participation – interim report 10
they design the rest of the research steps. As an example, the authors mention public health or
environmental restoration initiatives.
Step in Scientific Process Contributory Projects Collaborative Projects Co-Created Projects
Choose or define question(s) for study X
Gather information and resources X
Develop explanations (hypotheses) X
Design data collection methodologies (X) X
Collect samples and/or record data X X X
Analyze samples X X
Analyze data (X) X X
Interpret data and draw conclusions (X) X
Disseminate conclusions / Translate
results into action (X) (X) X
Discuss results and ask new questions X
TABLE 2: MODELS FOR PUBLIC PARTICIPATION IN SCIENTIFIC RESEARCH (BONNEY ET AL., 2009, P.17)
The most prevailing alliances between volunteers and scientists still never move beyond contribution.
The most common practice entails volunteers supporting the scientists with data collection and partial
data analysis. One can argue that this type of citizen science does not go beyond and is actually
similar to ‘traditional science’ with data stemming from crowdsourcing initiatives. The output is
delivered to scientists who continue the analyses and interpretation to apply it in their research
(Rotman et al., 2012). The barriers mentioned in section 3.2 explain the reasons for this. The
consortium should be aware of this and further investigate when public involvement is advantageous
for project initiators, and how this can be communicated to them. As such, opening up more functions
of the platform when required.
CLASSIFICATION OF HAKLAY (2013)
Analogous to Bonney et al. (2009), Haklay (2013)’s classification is based on the observation that
different degrees of participation and engagement are possible beyond mere data collection (See
Figure 1). At the most basic level, called crowdsourcing, participants deliver resources by gathering
measurements, samples and observations. At level 2, called distributed intelligence, they are
collectively involved in simple interpretations of the data that is gathered. At level 3, participatory
Promoting participation – interim report 11
science, the public is involved from the outset. They help define the problem or issue to be tackled
before conducting the scientific activities, and receive help from ‘experts’ in analysing the data.
Finally, at level 4, extreme citizen science, the public is involved throughout the entire scientific
process, possibly but not necessarily collaborating with or facilitated by professional scientists.
FIGURE 1. LEVELS OF PARTICIPATION IN CITIZEN SCIENCE PROJECTS. (ADAPTED FROM FIGURE 2 – LEVELS OF
PARTICIPATION AND ENGAGEMENT IN CITIZEN SCIENCE PROJECTS BY HAKLAY, 2013, FIGURE 2).
CLASSIFICATION OF WIGGINS AND CROWSTON (2011)
Wiggins & Crowston (2011, p.5-8) proposed a different typology of citizen science projects, which is
goal-oriented and identifies five mutually exclusive and exhaustive types. Where the classification of
Bonney et al. (2009) takes into account the influence the public can exert over the different research
steps, the typology of Wiggins & Crowston divides projects based on their intent.
For each type, they identified the inherent scientific, organizational and technology issues.
1. Action-oriented citizen science projects: Often a “bottom-up” and local initiative, set-up by
citizens and aiming to trigger an actual participant intervention in a specific concern.
Examples include wildlife concerns of a local population that then apply scientific methods to
investigate the matter and establish solutions.
a. Scientific issues: Since the projects are often initiated by stakeholders other than
scientists, their role is limited to consultants. As the aim often is to gather evidence
for an intervention, the data is less likely to be picked up by the scientific community.
Lev
els
of
par
tici
pat
ion i
n
citi
zen s
cien
ceLevel 4: Extreme citizen science
Level 3: Participatory science
Level 2: Distributed intelligence
Level 1: Crowdsourcing
Promoting participation – interim report 12
Moreover, the aggregation of data from different projects is hurdled by the use of
different scientific methods.
b. Organizational issues: Main challenge for these type of projects, and mainly due to
their grassroots origin, is attaining sustainability. They are oftentimes dependent on
donations and fundraising.
c. Technology issues: Due to the often-humble means of these type of projects,
involving technology is more a burden as they rely on volunteers to develop and
maintain the infrastructure.
2. Conservation-oriented citizen science projects: Often a top-down (researcher-initiated) or
middle-out (management-initiated) and regional initiative, it aims to support stewardship and
natural resource management goals. Public participation often demotes to mere data
collection, but their stewardship is often cultivated through educational material.
a. Scientific issues: When the projects are not under academic oversight, they are
typically still lead by professional researchers in governmental organisations. While
the main goal is to support resource-management decision-making, scientific validity
is valued highly.
b. Organizational Issues: Often dependable upon state funds or grants and part of
complex collaboration partnerships with regional actors and federal agencies.
c. Technology Issues: Most seem to be using web technologies for data entry and results
access.
3. Investigation-oriented citizen science projects: They have a top-down structure and they
are focused on previously well-defined scientific research goals, which require data collection.
Education is often a secondary, though inexplicit purpose of these types of projects, through
the provision of educational materials and tasks.
a. Scientific issues: Since these projects pursuit academic outcomes, valid scientific
results constitute their core concern. Among the methods used to ensure the quality of
the results are the application of uniform equipment, triangulation, etc. Yet, they are
less likely to screen the volunteers, since this subverts the ambition of education.
Uneven spatial distribution of the participants is another threat to a solid scientific
outcome.
b. Organizational issues: Management and sustainability challenges can occur due to the
large number of volunteers necessary.
c. Technology issues: Variates depending on the underlying funding resources and
project lifespan. Most seem to be using web technologies for data entry and results
access.
4. Virtual projects: These can basically be referred to as Virtual Investigation-oriented citizen
science projects, since they also focus on previously well-defined scientific research goals.
The differentiating characteristic lies in the sole use of ICT in all project activities, the
physical location of the volunteers is not meaningful. As an example Wiggins & Crowston
(2011) refer to Galaxy Zoo, where participants had to perform virtual image recognition and
classification tasks.
a. Scientific issues: Similar to investigation-oriented projects, their main interest lies in
producing valid scientific results. Due to their nature, the challenge lies in designing
valuable virtual tasks that also manage to engage the participants. Gamification
elements can be an added value in this respect.
b. Organizational issues: As virtual projects refer to top-down academic research
programs, they are also dependable on funding
Promoting participation – interim report 13
c. Technology issues: Complex IT platforms and technological tools are a necessity for
virtual projects, in that sense the help computer scientists are an asset to the project.
5. Education-oriented citizen science projects: a top-down initiative, for which education and
outreach are the primary goals. A further subdivision can be made in this respect by looking
into informal versus formal learning opportunities.
a. Scientific issues: Due to its focus on education, the project’s outcome does not
always translate to larger scientific research efforts. The target is to improve the
scientific inquiry skills of the participants. For this reason, the cost is relatively higher
than for other types of citizen science projects.
b. Organizational issue: A lot of the projects are part of a collaboration between
different partner organizations, and receive their revenue from funding sources. As a
result, it is often unclear for how long the project will be running.
c. Technology issues: The application of technology is variable, the projects’ websites
and tools have design constraints to appeal to their target student audience (be it
young children, families, teachers or students).
THE FLAMENCO PLATFORM AS A SUPPORTING LAYER ACROSS TYPOLOGIES
The three different typologies presented in the previous paragraphs can be bound together based on
the government of the project: top-down, middle-out or bottom-up.
• Action-oriented projects originate from the public and are thus strictly bottom-up. The role
of the scientific community is strictly guiding. The Flamenco platform should be able to
provide guiding mechanisms for the different research steps to support the achievement of
valuable results. In the classification of Bonney et al. (2009) these projects fall under the “co-
created” denominator. Haklay refers to these types of projects as “participatory science”, or
even “extreme citizen science”.
• Conservation-oriented projects originate from non-academic, often governmental
stakeholders. They are often still guided to some extent by researchers, but public stewardship
is cultivated. In the classification of Bonney et al. (2009) these projects could be contributory,
collaborative. It cannot be classified as “co-created” since the initiative is still top-down or
from a management level. They could be called “crowdsourcing” or “distributed intelligence”
according to Haklay.
• Investigation-, Virtual- and Education-oriented projects originate from the academic
community and are consequently top-down. In the classification of Bonney et al. (2009) these
projects can be found under contributory or collaborative. Haklay, in comparison, would opt
for the denominators “crowdsourcing” or “distributed intelligence”
It is interesting to map these classifications to the Flamenco case, where stakeholders who do not stem
from an academic field are encouraged to initiate their own citizen science project. Here, both
stakeholders and the public can be, but are not necessarily the same, and most importantly the
scientific support for the different research steps as distinguished above must be a technological
affordance.
Promoting participation – interim report 14
Initiator Process Participant Support Classification
of Bonney et
al.
Classification
of Wiggins &
Crowston
Classification
of Haklay
Public / Ad
hoc
organisation
Bottom-
Up /
middle-
out
Tailored
Public /
themselves
Technological
affordance,
Academic SH
Co-Created Action-
oriented
Participatory
science /
Extreme
citizen science
Non-
Academic
Stakeholder
Top-
Down
Tailored
Public
Technological
affordance,
Academic SH
Contributory,
Collaborative
Conservation-
oriented
Crowdsourcing
/ Distributed
intelligence
Academic
Stakeholder
Top-
Down
Tailored
Public
Technological
affordance,
Academic SH
Contributory,
Collaborative
Investigation-
oriented,
Virtual
projects,
Education-
oriented
Crowdsourcing
/ Distributed
intelligence
TABLE 3: MAPPING THE THREE CLASSIFICATIONS BASED ON THE TYPE OF INITIATOR
Flamenco
2.3 LESSONS LEARNED FOR FLAMENCO
We learned in this first chapter how citizen science can be defined as the intentional and active
engagement, in a non-professional capacity of volunteers in the states of research initiation, research
design, data collection, analysis, interpretation and the eventual dissemination of results. We further
elaborated upon the concept of participatory sensing and defined it as a method of data collection and
knowledge sharing, used for citizen science. It applies smart mobile devices and people knowingly
and intentionally choose to share certain data.
In the second part, we looked at classifications of citizen science projects. Depending on which of the
phases in a citizen science project is taken up by the contributors, they can be categorized by the
classification of Haklay (2013) and Bonney et al. (2009). Alternatively, the classification of Wiggins
and Crowton (2011) is goal-oriented.
As mentioned in the introduction, the central aim of the Flamenco project is to enable a myriad
number of (ICT-agnostic) stakeholders to configure a citizen observatory platform for their specific
application needs and domains. We can imagine that depending on the specific project, more or less
influence is entrusted on or embraced by non-professional scientists. In order to hold its promise as an
open and reusable platform, it should thus provide the means to give support in the different research
tracks. This is even more important, when the project is not initiated by an academic partner, but by a
societal stakeholder. As indicated in Table 3 above, the goals of Flamenco seem to be best aligned
with the first classification. As it there provides support as a citizen observatory for projects that can
be defined as action-oriented, initiated from the bottom-up by a specific segment of the general
segment or ad-hoc organization. The public is thus involved from the outset and have to themselves
express the problem definition, analyze and report the results. The expert that aid the initiator in each
step, should be substituted in the Flamenco platform by expert knowledge imbedded in the system and
guiding the different research tracks. Which results in the following requirements:
1. The platform should thus provide a way to determine who initiates the citizen science project,
and what their corresponding skill-level is.
2. Depending on the skill-level of the initiator, guidelines should be provided on how to conduct
a citizen science project, which steps should be included and how they should go about each
step. This might require the creation of a semi-automatic workflow composition supporting
the initiator when needed, based on the embodiment of expert knowledge in the system.
3. The platform should thus provide a way to determine who participates in the citizen science
project, and what the corresponding skill-level is.
Promoting participation – interim report 16
4. Depending on the skill-level and influence of the participant, guidelines should be provided
on how to help out in the citizen science campaign. Instructions could go from how to do a
proper data collection to how to correctly interpret the results.
5. Depending on the goal of the citizen science project, which can be indicated by the organizer,
different support mechanisms need to be set in place.
If the project succeeds in addressing variable needs and application domains it expands further upon
the core promise of citizen science as tool for a more tailored data collection through the distinct
expertise and capability of the targeted citizen (Conrad & Hilchey, 2011).
As mentioned, a core premise of participatory sensing campaigns is the notion of intent and conscious
participation and information sharing. A question that comes to mind is why should contributors do
that? What are the factors that trigger the intent of participation with volunteers? What constitutes
meaningful participation, how can the technology facilitate it, on the one hand, and how do
participants appropriate the sensing technology in accordance to the level of participation they seek?
These questions include but also extend beyond privacy-related issues. These are questions that we
will elaborate on in the third and fourth chapter of this document.
First, we will contemplate on the reasons of the initiators for instituting a participatory sensing
campaigns and what are the obstacles on their way that should be tackled by platforms like the one
provided by Flamenco.
Promoting participation – interim report 17
3 CHAPTER 2: INITIATING A CITIZEN SCIENCE PROJECT: REASONS AND
HURDLES
Two valuable question worth taking the time to answer when creating a generic citizen observatory
for an array of stakeholders are: Why would one prefer to undertake a Citizen Science project and
what are potential obstacles hindering its adoption? In a report on behalf of the UK Environmental
Observation Framework (UKEOF), Geoghegan et al. (2016) provide an elaborate overview,
answering these questions. Its main results, supplemented with the results from an additional literature
review can be found below. We will start this paragraph with an overview of the distinguished
motivations for stakeholders to get involved in CS. Subsequently, we take a closer look to the
challenges and barriers.
3.1 MOTIVATIONS FOR CHOOSING CITIZEN SCIENCE Geoghegan et al. (2016) discerned seven types of motivations for Citizen Science Projects. Their
findings are supplemented with insights from a broader literature study. Where possible, we link their
motivations up with the goal-identifying classification of Wiggins & Crowston (2011), as broadly
discussed in section 2.2.3.
TO CONTRIBUTE TO SCIENCE
The most common motivation for initiating a CS project, are the opportunities it brings to advance
scientific knowledge. Herewith we refer to the incorporation of the resources of volunteers, which
supports data collection and monitoring on a temporal, spatial, and a data-intensity scale that would
otherwise be too costly (Pocock et al., 2017). Or as put by Beza et al. (2017, p.2): “The accumulated
time dedicated to the crowdsourced research task, the number of contributions, and, in many cases,
the geographic spread of data entries often exceed the capacities of traditional research”. Moreover,
it enables to gather local knowledge and data that are siloed in other places. This is also
acknowledged by Burgess et al. (2017), who remark that citizen science has the ability to provide data
at a broad scale, while also allowing to zoom in on the local level.
This motivation can be aligned with the ambitions discerned for investigation-oriented citizen science
and the virtual citizen science projects.
TO INFORM POLICY
The volume of data intensity, analysis and interpretation enabled by CS, allows the stakeholders to
inform policy makers more swiftly about matters that preoccupy the local environments (bottom-up),
as well as to monitor activities that already have established a policy priority (top-down).
Promoting participation – interim report 18
Although informing policy can be seen as a potential side-effect of all different types of CS projects,
this motivation is very plain in the action-oriented and conservation-oriented citizen science projects.
TO EDUCATE
The mere partaking in academic activities educates their participants with regards to the processes that
underlie the scientific method. Moreover, they gather knowledge about the specific research topic at
hand (Pocock et al., 2017).
We can make the connection with the classification of Wiggins & Crowston (2011) and their
education-oriented projects that refers to top-down initiatives, for which education and outreach are
the primary goals.
TO IMPROVE BUY-IN
Central to the concept of CS is the fact that it enables a larger crowd to act as ‘the eyes and ears on the
ground’ (Geoghegan et al., 2016) for instances when mapping the spread of a specific disease. As a
consequence, this activated mass can also become more engaged and accepting towards the policy
decisions with regards to the specific topic.
This can be aligned with conservation-oriented CS projects, when the definition of conservation is not
limited to natural resource management, but also includes any prevention of deterioration.
TO RAISE AWARENESS AND ENGAGE PEOPLE
This motivation is in line with the previous one, as the authors claim that by raising awareness of for
instance environmental issues, the engaged citizen can get a feeling of shared responsibility, concern
and stewardship (Geoghegan et al., 2016; Williams, Hawthorn-Jackson, Orre-Gordon, & O’Sullivan,
2017). It also helps to hand the participants a bulk of knowledge so they can make better informed
decisions or even change their behaviour. CS can even inspire those participants to not only change
their own behaviour, but influence that of people in their environment as well. Here it can be seen as a
prerequisite for activism.
Similar to the previous motivation, there’s a relation with conservation-oriented citizen science
projects. There is also a definite connection possible with action-oriented projects, when the project is
a bottom-up initiative that triggers participant intervention.
TO BUILD PARTNERSHIPS AND IMPROVE COMMUNICATION BETWEEN DIFFERENT
STAKEHOLDERS
In this respect, one must think of relationships, data and information sharing between scientists,
amateur experts, local interest groups, the public etc. Besides promoting cooperation, it also builds
trust between the different communities. For some stakeholders, it can also be seen as a means of
reaching a targeted audience that might otherwise not have heard about the institution’s existence.
TO GAIN PERSONAL SATISFACTION
Promoting participation – interim report 19
A final motivation for conducting Citizen Science mentioned by the respondents of Geoghegan et al.
(2016) was that it gave them personal satisfaction. It made them witness a more direct impact on
society with their work, as did it allow them to increase their accountability towards the people that
often times, though indirectly paid for their work, the public.
Promoting participation – interim report 20
3.2 BARRIERS INHIBITING THE USE OF CITIZEN SCIENCE
The use of Citizen Science as a methodology to support the purposes listed above, is faced with a
number of barriers barriers and challenges that potentially inhibit the application of CS for different
projects. Geoghegan et al. (2016) have distinguished eight different elements. As with their research
on motivations, they broadened the applicability of the barriers beyond pure scientific stakeholders,
by interviewing others as well. We complemented their findings with the results from our own
literature study.
DATA QUALITY AND BIASES
The most common barrier refers to a mistrust towards the quality of the data gathered by citizen
scientists as well as their biases (Burgess et al., 2017). Poor data quality might be the result of the
equipment that need to be used by the citizen scientists, e.g. through the use of low-cost sensors. It is
also noted that the participants might not always be as motivated to engage successfully in a project.
Moreover, when very domain-specific data is needed, it is hard to outsource the tasks to laypeople. As
formulated by Rotman et al. (2012, p.2): “Scientists, educated, trained and placed within the
hierarchal academic world, are sometimes wary of letting others, who do not have the same
credentials as they do, into their labs and research”.
REQUIREMENT OF SPECIALIST EQUIPMENT / KNOWLEDGE
Some research projects require skills and capabilities (for example to deal with the specific
equipment) that surpass those of the majority of the public. Even with extensive training to enable
contributions by the public, some participants will not feel comfortable enough to partake (Mccrory et
al., 2017). Tasks on the other hand that are too easy or require regular input might result in the less
motivated and committed participants. At the same time, and due to an increase in citizen science
projects, potential participants might feel overburdened.
POLITICS
With politics, Geoghegan et al. (2016) refer to the question of accountability and liability when
decisions are based on data gathered from a Citizen Science project when no professional
interpretation of the results has been made.
UNAWARE OF AUDIENCE
Stakeholders that wanted to initiate CS projects, feel that they are not aware of the motivations and
interests of their target audience. This complicates a good communication and engagement strategy.
RESEARCH DESIGN AND IMPLEMENTATION ISSUES
For non-academic stakeholders, it might prove to be a challenge to create an appropriate research
design for the questions they have in mind. Included in this should be a good definition of their
Promoting participation – interim report 21
hypothesis, a proper collection of data, a good engagement strategy for the right participants and a
thorough analyses of the gathered material.
PARTICIPANT INVOLVEMENT AND IDENTIFICATION
An additional barrier is articulated by Rotman et al. (2012), as they acknowledge the difficulty to let
every participant feel like a full and legitimate member of the research project. They understand this
to be particularly challenging when the different contributors to the project stem from distant
communities with different values, goals and criteria for assessing the quality of the gathered data and
information.
TIME CONSUMING AND RESOURCING ISSUES
To assemble and activate a large base of participants is also very resource-consuming, both in time
and money. When talking about academics initiating the project, some of them fear that if the job can
be outsourced for free to the crowd, this can amount to a devaluation of the professional’s job. In the
research of Geoghegan et al. (2016) this was identified by all respondents as a key barrier.
PEER REVIEW / MISTRUST
The perception and fear exists that peer-reviewers who decide on publication of academic research
papers do not trust results from CS, due to the perceived data quality as mentioned above. This
explains the hesitation to conduct CS, as it is believed to slim the chances of being published. This
can only be solved if the field is becomes more generally accepted as a scientific approach in all ranks
of different institutions. To counter this, projects should take into account a sufficient amount of
participant training, quality assurance and metrics for evaluation comparable to those in traditional
science (Burgess et al., 2017).
UNCOMFORTABLE / UNPREPARED TO WORK WITH THE PUBLIC
Some academics might just feel it to be unconfident or distressing to work together with the public on
their projects.
Promoting participation – interim report 22
3.3 LESSONS LEARNED FOR FLAMENCO
The entire document provides an overview of which factors can play an important role to achieve
volunteer participation. To that end, an overview was provided in this second chapter of what
motivates and discourages stakeholders to opt for a CS approach. The different motivations of the
initiators are relevant for taking along, especially in light of Flamenco, which tries to provide an
adaptable platform. Additionally, it is commendable to find a good fit between the organizer’s
intentions and the ambitions of the participants, which will be discussed in the next chapter.
As mentioned, the biggest motivator is the (potential) benefit CS brings to the table, as in data
collection on a grand scale: time-, location-, and data-wise. The Flamenco platform should thus be
technologically able to support such intense data collection from different projects at the same time.
In relation to this, it is difficult for different initiators to realize when CS projects relevant to their
interests already exist, and it is not necessary to initiate one themselves. There are three reasons why
project initiators are not aware of the availability of applicable data: when it simply does not exist, if
they have not found about it, or if they are misinformed about the data (Burgess et al., 2017).
Flamenco should thus provide mechanisms so data over the projects could be shared with those
interested, if the participants and initiators have agreed upon opening up the data. The same
mechanisms should be in place for the sharing of relevant documentation on the topic between
initiators. When a project’s main motivation for conducting CS is to inform policy from a bottom-up
outset, a natural requirement is that a report with the results should be easy to generate. Projects that
also want to educate need to have part of the platform dedicated to the provision of material that can
educate the participants about the topic at hand. If the initiators want to improve buy-in or raise
awareness and engage people timely feedback seems of uttermost importance. In this way, they get
the feeling that their actions matter and can see in what direction the results are pointing to. A second
requirement that stems from this is that the platform or the organizers should enable social contact
between the participants, and between the participants and organizers. This improves the engagement
and buy-in as they can be part of a community. Another motivation mentioned above is the
opportunity to build bridges between different stakeholders in the field, possibly with the same
concern. It should thus be made feasible to connect with different and relevant stakeholders through
the platform. This can also support the previous requirement to relieve organizers from redundant
effort.
The main obstacles for applying meaningful CS for societal, non-academic stakeholders demonstrate
the need again for support mechanisms for the organizers and volunteers in the different steps of the
research process. Be it for ensuring a high-level (and quality) data collection, the provision of user-
friendly equipment and interfaces, the research design and implementation and finally the
interpretation and dissemination of results. When citizens initiate all the research steps, they should be
Promoting participation – interim report 23
able to get in touch with the necessary knowledge, be it through personal contact or the availability of
timely information. Flamenco has to ensure that the process, the data and results, generated through
its platform, are up to standard and accepted.
Other barriers identified are the fear of unsuccessful activation of a large public, how this can be
resource-consuming. Moreover, they wonder how to make every volunteer feel as an important
contributor. This relates to the perceived difficulty of getting to know one’s target audience
motivations and interests. Next to applying the right means for participant recruitment, it is paramount
to understand why participants want to partake and what can trigger a continued participation, they
should be able to communicate this to the organizers, so proper feedback can be provided.
In this respect, we will take a closer look at the audience in the next chapter. This should assist us to
understand what motivates them and what other factors can influence contribution.
4 CHAPTER 3: PARTICIPATION IN A CITIZEN SCIENCE PROJECT: MOTIVATIONS
AND HURDLES
The previous chapter provided us with an overview of the different opportunities and pitfalls that
citizen science brings to the committed stakeholder. The current one moves away from the organizer’s
point of view and scans the participation side. We consecutively touch the subject of participation, a
segmentation of citizen science participants, their motivation for partaking and how this changes over
time. Our aim of this chapter is to gather the existing knowledge about citizen science participants and
come up with recommendations that stakeholders conducting a citizen science project can take into
account.
4.1 PARTICIPATION: BEST PRACTICES
Reed (2008) provides us an elaborate literature review on participation. He defines the concept as
“[…] a process where individuals, groups and organizations choose to take an active role in making
decisions that affect them” (Reed, 2008, p. 2419). Three elements stick out in this definition: the
voluntary character, active engagement and the influence on their own lives. The latter makes us
expect that when there is a direct reciprocal benefit for contribution, participation has a higher chance
of materializing. It is with regards to this that Mccrory et al., (2017) mention how participation and
citizen engagement can be made more attractive, but not forced, as it always has to remain a voluntary
endeavor. Reed (2008) further refines his scope and focuses on stakeholder participation, with which
he concentrates on those who directly or indirectly have a stake in the matter. With regards to citizen
science, he notes the following interesting benefit of participation: “Stakeholder participation, […],
can empower stakeholders through the co-generation of knowledge with researchers and increasing
participants’ capacity to use this knowledge” (Reed, 2008, p. 2421). Here he thus again mentions how
the development of skill and awareness is a preferable outcome of the process.
Central to the same article lies the formulation of the best practice for stakeholder participation, which
is boiled down to the following eight key features (Reed, 2008):
1. Stakeholder participation needs to be underpinned by a philosophy that emphasizes
empowerment, equity, trust and learning.
a. Participants need to be empowered through their involvement by making sure that
they really have an influence on the process and that they have the capabilities to
actually partake. The latter means handing them the necessary skills for a meaningful
engagement.
b. Inequalities and trust issues between groups need to be tackled, in particular between
the participants and organizers. Only in this manner the actions of volunteers can be
Promoting participation – interim report 25
sensed as fair and valid by all groups. This also means that there should exist a two-
way learning between the participants, and between participants and organizers.
2. Where relevant, stakeholder participation should be considered as early as possible and
throughout the process
3. Relevant stakeholders need to be analyzed and represented systematically
4. Clear objectives for the participatory process need to be agreed among stakeholders at the
outset
5. Methods should be selected and tailored to the decision-making context, considering the
objectives, type of participants and appropriate level of engagement
6. Highly skilled facilitation is essential
7. Local and scientific knowledge should be integrated
8. Participation needs to be institutionalized
With regard to Flamenco, where stakeholders are not only to be seen as participants but also potential
initiators of campaigns, this implies the following:
• Hand over the necessary guidelines during each stage of the participation to enhance the
contributor’s sense of self-efficacy.
• Participants should not have the feeling of being merely instrumental to the process, their
motivation for partaking should be understood at the start of each project and supported.
• From the start of the project, the objectives and tasks should be clearly communicated.
With respect to understanding the reasons for participation, in the rest of this chapter we will examine
the different factors that encourage contribution.
4.2 FACTORS AFFECTING PARTICIPATION
One of the main challenges when initiating a CS-project is the attraction of a valuable number of
participants. It is for this reason that we zoom in the concept of participation and its contributing
elements in this section. We present the most common models that explain how participation in CS
can be enhanced (both in number of contributors and meaningfulness of the participation).
Geoghegan et al. (2016) refer to previous work of its contributing authors West and Pateman, who
created an overview of the different factors that affect the decision to partake in citizen science
projects (See Figure 2).
The created framework splits the road to a meaningful participation in three main elements: the
decision to participate, initial participation and sustained participation. Four influencing factors are
then distinguished that influence these steps: the project’s organization, the motivation, the awareness
of the CS opportunity and personal circumstances and demographics.
An online survey was organized by imec-SMIT-VUB, in light of the Smart City Meter initiative and
was completed by 742 respondents that can be classified as innovators or early adopters (see diffusion
of innovations by Rogers). Part of this survey related to respondent engagement in CS projects. Of the
742 respondents 10,2% already participated in a CS project, and only one fifth (20%) knew what it
entails. Keeping in mind that the respondents have a high innovative profile, it is clear that CS is not
yet an established concept nor that the participation to it is a common practice. Creating awareness of
the opportunity is thus a first main barrier to cross. In the rest of this chapter, we consecutively look at
the other factors that are important influencing factors for participation.
FIGURE 2: MODEL OF INFLUENCES ON PARTICIPATION IN CITIZEN SCIENCE
(GEOGHEGAN ET AL., 2016, P.36 – FROM WEST & PATEMAN (2015), BASED ON
PENNER
Promoting participation – interim report 27
MOTIVATIONS FOR PARTICIPATION IN CITIZEN SCIENCE PROJECTS
In this section, we provide an inclusive understanding of the different types of motivation that cause
attraction or retention of volunteers, who have to contribute their skills, time and effort to the
scientific cause (Nov, Arazy, & Anderson, 2011).
Nov et al., (2011, p.2) distinguish six different types of motivation to partake in the specific context of
citizen science and operationalized them as follows:
1. Collective motives: the importance attributed to the project’s goals
2. Norm-oriented motives: expectations regarding the reactions of important others, such as
friends, families or colleagues
3. Reward motives: benefits such as gaining reputation or making new friends, which was split
up in the following subcategories:
a. Community reputation benefits
b. Social interaction benefits
4. Collective identification: identification with the group, and following its norms
5. Hedonistic/Intrinsic motivation: the enjoyment associated with participation in the project
They then fit these motivations inside the framework of the theory of reasoned action and the
technology acceptance model. The motivations are the antecedents leading to the intent to continue or
increase participation. (See Fig. 3)
The literature review distinguished the two most important and recurring motives that lead to
participation as intrinsic motives, more specifically the enjoyment of participating, and the collective
motives, thus identification with the project’s goals. Nov et al. (2011) acknowledge how these two
motives seem to return and hence seem to be independent from specific participation contexts. The
FIGURE 3: MOTIVATIONS AFFECTING PARTICIPATION
(NOV ET AL. 2011, P.2)
Promoting participation – interim report 28
extensive literature review on motivation to contribute to different research studies by Jennett et al.
(2016) align with these findings. Four of the seven most recurring motives they discovered were
interest in the research topic (-intrinsic), contributing to original research and science (-collective),
sharing the same goals and values as the project (-collective identification), enjoying the research task
(-intrinsic). They thus connect with the collective and intrinsic motives.
From these findings, Nov et al. (2011) derive more concrete design recommendations for the
organizers of citizen science projects: First of all, the project’s goal and achievements should be
timely and regularly communicated towards the potential pool of participants. Whereas, the pure
enjoyment of partaking in a project can be attained to by involving some gamification elements for
the participants. In the next chapter of this deliverable we take a look of what that might entail in
more detail.
Secondary motives that also seemed to have an influence, though in a lesser manner, were the
identification with the group of fellow citizen scientists and organizers. These were also identified by
Jennett et al. (2016) as the felt need to help others and feeling part of a team, as well as receiving
recognition of and appropriate feedback on their contribution. Leading Nov et al. (2011) to advocate
for the establishment of a strong community of volunteers who “[…] share beliefs, interact regularly,
possibly using social media outlets, and work collectively towards a common goal” (Nov et al., 2011,
p.5).
Referring to Curtis (2015), Jennett et al. (2016) mention how the chance on interaction heightens
when projects are more complex. Looking at how co-operation is a powerful motivator to sustain
participation, this can be aligned with another interesting suggestion to create a dynamic contribution
environment, with this they refer to enabling the participants to gradually take on more demanding
tasks and take steps up the “ladder of the responsibilities”(Nov et al., 2011). They note how “being
able to contribute to a project in a number of different ways (e.g. as a team leader, or moderator) can
motivate an individual to sustain participation, particularly if this contribution is felt to be of
importance and is valued” (Curtis, 2015; Jennett et al., 2016, p. 4).
Innate to the expansion of research tasks and responsibilities is the promise of education. A little
underexposed in the framework provided by Nov et al. (2011) is learning and understanding as a
motive, though they can be conceived under intrinsic motivation. Alender (2016) considers both
learning about the topic as learning new skills as catalysts for contribution. She further notes how
especially learning from others and sharing this experience are significant predictors of a gratifying
experience, denoting the creation of an atmosphere of social learning.
Related to this is the MLC model of Jennett et al. (2016) which gives an explanation of how
motivations, learning and creativity are related (See Figure 4). They assert that through an active
Promoting participation – interim report 29
participation in both the micro-and macro-levels of a project, the participants gain new knowledge
and identify themselves stronger as volunteers. This new-found identity creates a feeling of
contribution and belonging to the project community as well as a greater sense of self-confidence.
This expands even further as time progresses and their desire to share is nourished the more the
community welcomes their input. A minority of high-involved volunteers will eventually reach a
confidence threshold that enables them to share their own creative ideas, e.g. on a forum. If these are
appreciated and taken up by the community the process maintains itself (Jennett et al., 2016, p. 15).
Questions that need to be answered at the onset of every citizen science project to enable learning are
according to Jennett et al. (2016, p. 18):
• How difficult is it to start contributing?
• Is there progression in the difficulty of the tasks?
• Do the volunteers receive feedback about their contribution?
4.2.1.1 CHANGING MOTIVATIONS
Past exploration into the matter of motivation for contributing to citizen science projects has
delineated how the motives change over time. Research by Rotman et al. (2012) determined how
initially participants are persuaded with intrinsic, egoistic motives, indicating knowledge or
experience expansion and/or the fulfilment of an enjoyable activity. After a while, the participants
enter what Rotman et al. (2012) call a period of reflection and reassessment. In this phase, continued
participation is decided upon by revaluation of their satisfaction with the participation up to now
(intrinsic factors), or they continue for altruistic reasons (where the goal is to increase someone else’s
welfare (Land-Zandstra, van Beusekom, Koppeschaar, & van den Broek, 2016)) or out of collective
interest (where the goal is to increase a group’s welfare (Land-Zandstra et al., 2016)) (Rotman et al.,
2012). In a figure, provided by Rotman et al. (2012) this looks like the following (Figure 5):
FIGURE 4: MLC MODEL OF MOTIVATION, LEARNING AND CREATIVITY IN CITIZEN
CYBERSCIENCE (JENNETT ET AL., 2016)
Promoting participation – interim report 30
There is value in these observations for the design of a fitting communication strategy. We will
elaborate on this in the next section, where we are looking into organizational factors and how they
contribute to a greater participation.
ORGANIZATIONAL FACTORS & PARTICIPATION IN CITIZEN SCIENCE PROJECTS
Characteristics of the configuration and coordination of the project have an influence on a (sustained)
contribution and on the volunteers’ motivation. The ones primarily mentioned in literature are the
design of the proper communication and feedback mechanisms. This goes from receiving
acknowledgements of the data collection, notification of reception, affirmation that it is being used
and eventually reviewing the results and its (hopefully positive) impact on the initial cause. These
mechanisms refer also to the reception of clear instructions and guidance from professionals and peers
(Geoghegan et al., 2016).
Geoghegan et al. (2016) and Debacq (2016) provide us with the communication aspects that need to
be set in place in the recruitment phase and throughout the rest of the project. Recruitment has to be
initiated making use of the different media outlets, tailored to the desired audience, research context
and culture. For example, one can try to garner press attention, buy advertisements, make use of direct
mailings or use social media to gain traction. If the organizers envision the consultation of lay people,
the language used must remain simple to create a low threshold for participation. Keeping in mind the
previous section on motivations, the message should include the impact the participation of the citizen
scientist can have on the research at hand.
To allow for a sustained participation, there is reason to belief the development of a sense of
community for everyone involved holds certain benefits. Dickinson et al. (2012) note how for short
participation of large crowds “easy, fun and social” are three key words, for longer projects that
FIGURE 5: A PROCESS MODEL OF VOLUNTEERS AND
SCIENTISTS INVOLVEMENT IN CITIZEN SCIENCE
PROJECTS (ROTMAN ET AL., 2012)
Promoting participation – interim report 31
require more effort however, a closer interaction is necessary. This can be done by creating specific
new media tools for the project with requirements stemming from the tailored population. These tools
could range from project-specific applications, towards a platform where all stakeholders can bi-
directionally communicate with each other. Continual feedback about the status of the project, the
quality and the impact of the results, personal and direct contact between groups and the provision of
educational material are three main means that need to be cherished in this respect as well. A side-
effect of communication and handing over educational information is that it can contribute to a higher
quality of the data collected. Participation can also endure if the value of the volunteers’ contribution
is emphasized in all different types of communication (Land-Zandstra et al., 2016).
If we turn this around, the barriers for an effective communication as listed by Debacq (2016) and
adapted to for our purpose are the following:
• Project organizers that are unfriendly / intimidating and use specific jargon
• Lack of attention for the initial interests of the citizen scientists for the project
• The different parts and the timing of the scientific process are not clearly communicated, and
thus not understood by the volunteers
• Absence of regular feedback
• Communication means are not adapted to the targeted audience
The organization of citizen science projects need to consider the perceived control of the volunteers
over the factors that enable data sharing. Herewith, Gharesifard & Wehn (2016) mean that not only
the volunteers need the necessary skills, they moreover need to believe that they are able to contribute.
In their conversation with participants, the organizing stakeholders need to elucidate that it is not
necessary to understand the scientific method of data collection or its supporting technologies, nor
that the volunteers should hold a great understanding of the research topic at hand. Besides the
communication aspect, this also means that the tools used for data collection need to be accurate, of
high quality and user-friendly and that the participants have enough time to complete their tasks. We
have learned from Aoki, Woodruff, & Willett (2017) that another essential aspect of the organization
is a clear view on the project’s goals from the start, as this has an effect on which participants to invite,
from which region and with what type of sensors. As an example, they note how massive data
collection campaigns are not always reaching a critical data density. Volunteers are often also
interested to gain knowledge about how the research can affect their own situation, be it their well-
being or that of their neighborhood. If the data collection is to sparse to get valuable results for their
specific situation, this can leave them disillusioned about their contribution. Aoki et al. conclude as
follows: “[…] resources might be better spent developing low-cost, high-accuracy, fixed sensors, and
building dense sensing networks that can meet the needs of both individuals and scientists” (Aoki et
al., 2017, p. 3147).
Promoting participation – interim report 32
To epitomize, Geoghegan et al. (2016) include a table with the most important organisational
(logistical) and dispositional (personal motivations) factors that encourage or discourages
participation.
TABLE 4: SUMMARY OF WHAT ENCOURAGES AND DISCOURAGES PARTICIPATION (GEOGHEGAN ET AL.
4.3 A SEGMENTATION OF PARTICIPANTS IN CITIZEN SCIENCE PROJECTS In the previous paragraphs, we zoomed in on the motivations of stakeholders and participants for
engaging in CS, and what barriers might hurdle their enthusiasm. In this section, we will briefly
glance over the participants of CS projects. A better understanding will help to find out what
motivates them to partake.
In the typology of citizen science projects by Wiggins & Crowston (2011), as presented above, it was
clear that education purposes were often an important goal in each of the categories. Making it into a
priority to get ‘lay-people’ involved and knowledgeable about the scientific processes underpinning
research. According to Martin (2017) this way of thinking holds “[…] the implicit assumption that
“non-scientist” volunteers come to citizen science projects with relatively low levels of scientific
knowledge or understanding, or (sometimes) little support or interest in science” (Martin, 2017, p. 3).
In his study, he rightfully notices, as we have seen in the previous section, how one of the primary
motivations of the participants to engage in citizen science projects is a desire to contribute to science,
which already gives the feeling that a segment of participants are science enthusiasts. Martin, (2017)
made a segmentation of his population, based on their prior Engagement in Science (EiS), a parameter
that was measured using the following the questions, answered on a 7-point Likert scale:
• How interested are you in science, generally? (interest)
• In general, how often do you actively seek out scientific information? (seek)
• In general, how much do you trust scientific research? (trust)
• In general, how easy do you find it to understand scientific information? (understand)
He then went on to look if there was a correlation between EiS and willingness to participate in
scientific research and in what role they would see themselves partaking. He eventually could identify
two main groups: working scientists (respondents that already worked in higher education) and non-
scientists. The latter group was eventually divided into 5 subgroups (From low to very high EiS).
From this segmentation, he then drew the following the conclusions:
• The respondents with a very high prior engagement in science, appeared to be the most
interested and the most likely to participate in scientific research. Their education level was
also the highest of the different groups. As seen in the barriers for citizen science, the
perception exists that involving the public can bring forth bad, unreliable data. This new
insight tempers this belief as the people, who are most likely to participate in CS, seem to be
highly educated.
• Besides this group, involving the high and moderate EIS group seems also attainable. The
main barrier towards their inclusion is their self-perceived lack of knowledge. As mentioned
by Martin (2017, p. 19) they feel “less able to understand science and are less confident in
their abilities to assist” than the very high EIS group. She sees a solution in projects that
market themselves as opportunities to increase volunteers’ knowledge.
• Engaging the Low-EIS group is the most challenging as it might be hard to find a fit between
their interest and the objectives of the CS project. Their lower education levels can be a
Promoting participation – interim report 34
barrier for inclusion, even more so since CS projects often don’t engage with the audience on
their terms, which “demonstrates a lack of understanding about the disconnection some
groups have with the “elite” world of science”(Martin, 2017, p. 20) .
It seems that in order to install a great CS with the highest potential of inclusion of groups with a
different socio-demographic background, establishing a relationship of trust between participants and
organizers seems to be vital. We can conclude that different engagement strategies are necessary for
ensuring the inclusion of different types of participants. Providing an easy entry point for people with
no scientific background, avoiding the use of scientific slang and contextualize the process in the
everyday life of its contributors. On the other hand, people who want and are able to learn more about
the science in itself should be provided with that opportunity.
4.4 LESSONS LEARNED FOR FLAMENCO
Within Flamenco the different needs of the Advisory Committee of Users (ACU) members, interested
in starting a CS project, have been listed. These go from different data needs, the specific constraints
(e.g. time constraints, population constraints), as well as the needs in the deployment phase (e.g.
communication needs), the execution phase (from privacy concerns to mechanisms that counter the
risks of dropout) as well as the specific output possibilities.4 A subpart of those needs calls out for
mechanisms that supports the involvement of participants from the start until the end of a project. In
this section, we will shortly summarize the literature study of this chapter and elucidate how this can
be useful for Flamenco.
The questions that this chapter tried to answer was a. how can we make participation more attractive
and b. how can we keep the volunteers interested over the run of the project?
A couple of elements were distinguished that influence the intent of participation at the volunteers’
side.: their motivations, awareness of the opportunity, organizational factors, personal circumstances
and demographics.
The two main motives for contributing were the mere enjoyment and identification with the projects’
goals. This fits with the main objective of Flamenco. The project wants to provide the necessary
supporting tools and mechanisms to create bottom-up CS-based campaigns. This allows the projects
to be driven by users’ needs and interests, this would thus potentially make it easier to engage
participants.
Moreover, it seemed that organizational factors can have a grand influence on these and on the
eventual road to participation as well. Which is a positive message for the Flamenco project. The
main lessons learned are the following:
• Awareness of the project should be created using as many communication sources as possible,
tailored to the public and its specific context.
o Awareness towards the final project goal: what will the campaign enable, what
problem does it tackle. Less the scientific purpose, but the societal goal is something
that needs to be addressed.
• Contribution should be reciprocal as in, participants who are interested should also feel that
they get something out of it. In this respect, it is paramount to know what motivates the
specific participation at the start of the project, so they can be supported if possible. As a
consequence, participants feel valid contributors and not just instrumental data collectors.
Promoting participation – interim report 36
• Contribution to a project that is interesting to the participants is one of the main motivators.
Thus, the objective of the project should be communicated clearly, and regularly towards the
(potential) pool of participants. It should not promise things that are not part of the project as
this will cause disillusionment.
o What is expected of the participants, as well as the timeline of the project should be
disclosed at the start
• (Potential) Participants have different literacy levels, and not all feel that they are capable of
contribution. If the goal is to include as many people as possible, a big proportion should be
convinced that they do have the right skills, or that they can pick them up along the way
o Hand over the necessary guidelines during each stage of the participation to enhance
the contributor’s sense of self-efficacy. This can vary from just providing guidelines
up till a built-in flow mechanism that guides the user automatically throughout a
process.
o Working together and feeling part of a community is well received by contributors.
Without overburdening them, establishment of a strong community of volunteers who
“[…] share beliefs, interact regularly, possibly using social media outlets, and work
collectively towards a common goal” (Nov et al., 2011, p. 5) is important. Especially
so since learning from each other and being able to share the knowledge is a
gratifying experience for the volunteers.
o Keep the language used in communication simple and friendly
o Enough time should be given to the participants to complete their tasks
o Establish a relationship of trust between participants and organisers
• If tasks are too easy and people do not feel challenged, participation will stop eventually. For
this reason, it has been proposed to create a dynamic contribution environment, with this they
refer to enabling the participants to gradually take on more demanding tasks and take steps up
the “ladder of the responsibilities” (Nov et al., 2011). This educational aspect, comes down to
three things: Don’t make it too difficult at the beginning, make the tasks more challenging
over the course of the project and give enough feedback so participants can actually learn.
• Enough communication from organizers towards participants is important to feel approval:
o Sending acknowledgements and give value of the data collections
o Notifications of reception
o Affirmations that they are being used
o Information about the status of the project
o Giving the participants an opportunity to review the results and witness the impact on
the initial cause
• The tools used need to be accurate, of high-quality and user-friendly for the participants.
Proper and clear instructions on how to use them should be provided as well.
o Sensors should be easy to couple with the created platform
• Volunteers are interested to apply the knowledge and data on their own situation or that of
their neighborhood.
o Hand over the proper information for how they can do that, or if it is not possible (e.g.
too low data-density in their neighborhood) explain this from the beginning.
4 A full list is described in Deliverable 6.3
Promoting participation – interim report 37
• The pure enjoyment of partaking in a project can be attained to by involving some
gamification elements for the participant, more on that in the next and final chapter.
Promoting participation – interim report 38
5 GAMIFICATION AND BEHAVIORAL CHANGE The following chapter is dedicated to the use of a mechanism to playfully steer behavior referred to as
“gamification”. First, an introduction to general behavior change approaches is given. Gamification is
framed as one of several tools which can be used as part of strategies to steer behavior. Then, the
concept is explored in further depth. Important elements of gamification are outlined. Finally, the
application of the concept within the context of citizen science is discussed.
5.1 INTRODUCTION: GAMIFICATION IN THE CONTEXT OF BEHAVIORAL
CHANGE Gamification approaches can be situated within the strand of behavioral change research. Contrary to
traditional economic approaches, behavioral change approaches as common within behavioral
economics recognize that humans are not rational, thus being closer to psychological approaches (see
e.g. Dawnay and Shah 2005). Indeed, behavioral change approaches originate from the social sciences
or psychology and mainly differentiate themselves through their specific focus (Darnton, 2008).
Nevertheless, they have in common to acknowledge that humans are emotional, influenced by their
context and other individuals. As a result, it is possible to observe a huge gap between most people’s
(usually rational) intentions and their actual actions (see also Blake, 1999). Giving an example, while
around new year a lot of good intentions are formulated, e.g. to smoke less or to do more sports with
presumably positive effects on health or money spent, these intentions are often not followed through.
Behavioral change approaches try to overcome this intention-action gap by designing an environment
that pushes the right choices.
As Soman (2015) outlines, there are four main strategies for behavioral change, whose actual form
might vary depending on whether they are used in the context of a market or policy implementation.
A first option to change behavior is to restrict existing choices. In a policy context that would mean a
regulation of supply, while in a business context this measure would translate into product
unavailability. A second strategy to influence behavior is to give individuals incentives. As part of a
policy, these might be subsidies (positive incentive) or the imposition of taxes (negative incentive). In
a market environment, the promotion or discount of products might be preferable options.
Furthermore, it is possible to use the strategy of information or persuasion to elicit behavioral
change. While in a policy context this would entail the disclosure of specific information, this strategy
translates in the free market into advertising. The final and fourth option is nudging. This is probably
the most misunderstood option of behavioral change, since it is often equated with behavior change
per se. Yet, nudging describes both in a policy and market context the influence of human’s choice
Promoting participation – interim report 39
architecture, i.e. it entails a change of the context of the decision to implicitly steer behavior in the
desired direction.
As an example, the goal of a behavioral change intervention might be to increase the consumption of
sustainably caught fish by policy makers to prevent the over-fishing of the oceans. In this case,
strategy one would translate in an abolition of non-sustainably caught fish. The second strategy would
entail a subsidization fish which was caught considering matters of sustainability. Strategy three
would require disclosing information about the ocean as an ecosystem, the fish industry, and
sustainable fishing practices to the consumer. Finally, the fourth strategy could entail giving fish
which was caught sustainably a more prominent shelf place in supermarkets.
While in the given example thus all strategies would theoretically be an option, depending on the
context of the research some strategies might be more interesting, and might be therefore worth
exploring in practice, i.e. in the form of experiment to determine the most effective strategy. Giving
the example, suggested by Kamenica (2012): Monetary rewards might not always be suitable, e.g. in
the case of inherently interesting tasks or prosocial behavior, or it might be difficult to provide the
right amount of reward and options. Also, ethical and practical reasons have to be considered, for
example it might not make sense to forbid sugar altogether to change people’s behavior regarding
sugar consumptions since this would lead to consumer boycott.
When a strategy is chosen, additionally several tools can be combined with the strategy to implement
it most effectively. Such tools could include for example the introduction of decision points (i.e.
constant reminders), categorization (i.e. organizing information or income), and, very popularly,
gamification, which entails the use of game techniques to influence behavior.
Hence, gamification is not a behavioral change strategy per se, but a tool that can be used to execute
certain behavioral change strategies, usually in the context of information or (monetary) incentives.
Promoting participation – interim report 40
5.2 DEFINITION: GAMIFICATION AND ITS ELEMENTS Ass mentioned gamification is likely the most frequently used tool as part of behavior change
strategies. A popular definition of gamification is provided by Deterding et al. (2011). Accordingly,
gamification entails the “use of elements of games in non-game contexts” and is concerned with
gamefulness, gameful interaction, and gameful design (p. 10). Specifically, the authors (2011, p.13)
refer to gamification as:
1. the use (rather than the extension) of
2. design (rather than game-based technology or other game- related practices)
3. elements (rather than full-fledged games)
4. characteristic for games (rather than play or playfulness)
5. in non-game contexts (regardless of specific usage intentions, contexts, or media of
implementation).
Gamified applications are thus inherently different from whole games (including serious games) or
more playful design, in the sense of toys.
Yet, also gamifications aims to appeal to the hedonist in users. It is assumed that users seek positive
affective reactions such as joy, mirth and joviality, while avoiding unpleasant ones. Gamification
makes it possible that non-hedonic, i.e. utilitarian potentially non-enjoyable tasks, which however
might be socially or environmentally desirable, can be made more enjoyable. In other words, intrinsic
motivation, i.e. the wish to perform activities for the sake of it are promoted for tasks which would
usually require extrinsic motivations, i.e. motivation from outside to achieve certain valued outcomes
(Ryan & Deci, 2000).
Gamification is therefore more complex than “adding points and badges on top of a system” (Thiel,
Reisinger, Röderer, & Fröhlich, 2016, p. 35). It is rather the process of balancing utilitarian and
hedonic benefits (Hamari and Koivisto, 2015).
In this context, design thinking is highly important when creating gamification in order to get the
participants to participate and to keep engaged and playing. Detering et al. (2011) provide a
comprehensive overview about the levels on which such a game design can take place. The authors
differentiate between design patterns, mechanics, general principles, models, and methods. Giving an
example, design patterns such as a smooth player journey, creating balance between challenge and
skills, and the designing of the set-up as a full experience are essential.5 Game-mechanic such as
5 http://gamifyforthewin.com
Promoting participation – interim report 41
competition, mastery, scarcity and discovery can then be applied to make the game attractive (e.g.,
Chou, 2013).
Level Description Example
Game interface
design patterns
Common, successful interaction design components and
design solutions for a known problem in a context,
including prototypical implementations
Badge, leaderboard, level
Game design
patterns and
mechanics
Commonly reoccurring parts of the design of a game that
concern gameplay
Time constraint, limited
resources, turns
Game design
principles and
heuristics
Evaluative guidelines to approach a design problem or
analyze a given design solution
Enduring play, clear goals,
variety of game styles
Game models Conceptual models of the components of games or game
experience
MDA; challenge, fantasy,
curiosity; game design
atoms; CEGE
Game design
methods
Game design-specific practices and processes Playtesting, playcentric
design, value conscious
game design
Table 5: Levels of game design according to abstraction (Detering et al., 2011, p. 12)
However, as the authors note, the description of game elements varies significantly in literature.
While there seems to exist a general agreement of basing frameworks on the concept of intrinsic and
extrinsic motivation, depending on the theoretical focus of the framework, the elements listed might
differ (see also Seaborn & Fels, 2015).
Aparicio, Vela, Sanchez, & Montes (2012) chose Self-Determination Theory (SDT) as introduced by
Ryan & Deci ( 2000) in their framework to refer to elements relevant for gamification. According to
SDT, “human beings can be proactive and engaged or, alternatively, passive and alienated, largely
as a function of the social conditions in which they develop and function” (p. 68). Three factors have
been hereby identified which can foster intrinsic motivation, self-regulation, and well-being:
competence, autonomy, and relatedness. Aparcio et al. (2012) connect these three factors in the
context of gamification with a list of game elements. Accordingly, autonomy is realized through
profiles, avatars, macros, configurable interface, alternative activities, privacy control, and
notification control. The need to feel competent can be addressed via positive feedback, optimal
challenge, progressive information, intuitive controls, points, levels, leaderboards. Finally, relatedness
can be created by considering the creation of groups, messages, blogs, connection to social networks
and the provision of chats.
Promoting participation – interim report 42
Blohm and Leimeister (2013) start from the perspective of game design and aim to create so called
“gamified service bundles”. Starting from a specific usage objective, these are translated into game
design elements which are then compiled in the gamified services, which should active certain user
motivates and thus stimulate consumption. The motives include hereby not only self-determination,
but also intellectual curiosity, achievement, social recognition, social exchange and cognitive
stimulation. The authors list hereby two kinds of game design element: game mechanics and game
dynamics. The mechanics are the actual building blocks for the design of the game, while the
dynamics describe their effect (see Table 6
Game-design elements Motives
Mechanics Dynamics
Documentation of behavior Exploration Intellectual curiosity
Scoring systems, badges, trophies Collection Achievement
Rankings Competition Social recognition
Ranks, levels, reputation points Acquisition of status
Group tasks Collaboration Social exchange
Time pressure, tasks, quests Challenge Cognitive stimulation
Avatars, virtual worlds, virtual trade Development/organization Self-determination
TABLE 6: BLOHM AND LEIMEISTER (2013)
Finally, Nicholson (2012) proposes a user-centric framework for “meaningful gamification”. He bases
his framework on a variety of theories, namely Organismic Integration theory (a sub-theory of Deci &
Ryan’s (2000) SDT), Situational Relevance (e.g. Wilson, 1973) Situated Motivational Affordance,
(Deterding, 2011), Universal Design for Learning (Rose and Meyer, 2002), and User-centered Design
(Norman, 1988). Organismic Integration theory suggest that meaningful game elements are
intrinsically motivating, situational relevance describe the users decision about meaningfulness,
situated motivation affordance the need to match user profile and context with the game, universal
design helps to design the most beneficial experience for different user, while user-centric design
brings them all together by putting the user in the center (Seaborn & Fels, 2015). The author
emphasizes the need to put “the needs and goals of the users over the needs of the organization” in
order to create a long-term benefit emphasizing the need to create a personalized gaming experience
for the individual user based on their life instead of external rewards (Nicholson, 2012).
In the end, as Seaborn & Fels (2015) emphasize, most gamification-specific frameworks were
developed in isolation and “there is, as yet, no evidence of their completeness; these frameworks
need to be applied in order to determine their applicability and convergence.”
Promoting participation – interim report 43
Hence the overview of Thiel et al. (2016) proves to be especially useful. The authors provide an
comprehensive overview about the most commonly used elements in gamification approaches. The
authors list as common elements for gamification achievement, points, status, expression, feedback,
personalization, challenge, competition, and time constraint (see Table7).
Classifier Elements Description
Achievement e.g., badges A mechanism to show the user his or her
progress and achievements within the system
Points Users can earn virtual points that in some
cases can be used to redeem physical
artefacts.
Status e.g., levels In contrast to points in leaderboards, the
underlying mechanic that aims to motivate is
the strive for recognition by others and
findings one’s place in a community.
Expression e.g., spaces for open creativity
Feedback e.g., notifications Where used, these spaces for open
creativity/creation are usually the main
component of the system.
Personalization e.g., profiles, avatars The system provides the user with additional
information, hints or gives encouraging
statements.
Challenge e.g., missions, quests The system offers a space that contains
information about the specific user or can be
modified by the user.
Competition e.g., leaderboards, highscore lists The system or other users ask the user to
perform a certain activity under predefined
conditions.
Time constraint e.g., due dates, countdowns Competition does not necessarily connect to
rivalry, but can also be neutral comparison.
TABLE 7: OVERVIEW OF GAME ELEMENTS (THIEL ET AL., 2016)
In this context Seaborn & Fels (2015) provide a list of distinct definition of elements that are often
used, namely points, badges, leaderboards, progression, status, level, rewards, and roles.
• Points: Numerical units indicating progress e.g. score.
• Badges: Visual icons signifying achievement e.g. trophies
• Leaderboards: Display of ranks in comparison e.g. rankings
• Progression: Milestones indicating progress e.g. levelling
• Status: Textual monikers indicating progress e.g. title
Promoting participation – interim report 44
• Levels: Increasingly difficult environments e.g. stage
• Rewards: Tangible, desirable items e.g. prizes
• Roles: Role-playing elements of a character e.g. class
Summing up, there are several approaches towards game elements within gamification approaches,
depending on the underlying framework used by the prevailing authors. Thiel et al (2016) provide a
good summary of the important elements independent of the theoretical foundation chosen, while
Seaborn & Fels (2015) provide a list of distinct terminology for gamification approaches.
Promoting participation – interim report 45
5.3 APPLICATION: THE POTENTIAL OF GAMIFICATION FOR CITIZEN SCIENCE As Thiel et al. (2016) emphasize, gamification can contribute to participation since it can provide a
means for engaging and educating individuals. Indeed, as Geoghegan, et al (2016) outline, with the
increasing use of online platforms for citizen science platforms, gamification is being used by an
increasing amount of projects to stimulate participation.
Gamification is popular in the context of topics as diverse as education (e.g. Foster et al., 2012),
health (e.g. Rose et al., 2013) , sustainability (e.g. Liu et al., 2011), research (e.g. Rap & Marcengo
2012), crowdsourcing ( e.g. Mason et al., 2012) or marketing (e.g. Downes-Le Guin et al, 2012). The
most common cause is hereby data collection, which would be usually resourceful, difficult and/or
highly cost intensive. Giving an example, the Geograph.org.uk engages citizens to map remote areas
in the United Kingdom. Participants competed via the numbers of uploaded images. Likewise, the
Citizens’ Network for the Observation of Marine BiodivERsity (COMBER) project in Greece reached
out to local diving clubs to monitor biodiversity in a marine environment (Hakley, n.d.).
Gamification literature in general has some weakness, for example in terms of coherence of
definitions, the use of theoretical foundations, lacking theoretical or empirical investigation, and
missing long-term studies (Seaborn & Fels, 2015). However, as Murphy (2015) outlines, the
application for gamification for citizen science especially can still be regarded as needing significant
research effort.
Prestopnik and Crowston (2012a) provide an important contribution to this strand of research by
differentiating generally between “task gamification” and “game taskification”. The first concept
refers to adding game elements to a needed task to foster the completion by making it more enjoyable.
As an example, the authors describe the tool Happy Moths which was created by scientists for
taxonomy purposes . Users have to identify moths according to pictures, one of them being “the
happy moth” which is used to evaluate their performance. However, users also achieve scores for
their long-term involvement, leader boards, and high scores to promote competition. This also help to
evaluate the quality of the users’ work (see also Prestopnik &Crowston 2012b).
On the contrary, game taskification refers to building a game with tasks elements. The authors use the
example of Forgotten Island. The game features a whole story-driven gaming world, namely an island
that the user has to build and explore. The classification of insects is hereby only a means to collect
game money for equipment and game items for the game.
As Prestopnik and Crowston (2012a, 2012b) outline, task gamification is easier to realize in the
context of citizen science since it requires fewer resources and shorter development time, yet, game
taskification has immense potential for user involvement (see also Prestopnik and Crowston 2012a).
Promoting participation – interim report 46
Crowston & Prestopnik (2013) showed that in both cases the competitive features of the game and the
knowledge that the game was helping scientists highly motivated them to participate, even above the
required benchmark. This finding is also echoed in the work of Cooper et al. (2010) and Curtis (2015),
who show that indeed sometimes participants’ motivation stems from the competitive features, while
for others the contribution to science is on the forefront. Also West et al. (2015) confirmed these
altruistic motives. As Bowser et al. (2014) showed, this might be indeed related to the age of the
participants. While Millenials were attracted in their study to game elements such as earning badges
and peer competition motivating, and elaborations offered in response to open-ended survey questions,
older users wanted to contribute to science and the public good.
However, as Massung et al. (2013) have shown gamification might also demotivate participants to
participate if participants feel they are not successful in competing. As Bowser et al. (2013) add, also
the target group for the citizen project can be decisive. The authors revealed that participants who
identified as amateur scientists did not embraced highly game-like interfaces. Hence, whether
gamification is successful depends highly on the participants’ personal profile.
Once these issues are solved, Eveleigh, Jennett, Lynn, and Cox (2013) propose a range of guidelines
how to successfully valorize gamification for citizen science.
(1) Scoring mechanisms: Scoring mechanism should be detailed enough to show continuous
personal progression next to being a means of comparison with competitors.
(2) Personalized feedback: Feedback should help participants to assess the quality of their
contribution, rather than its quantity, to ensure that a learning process is taking place.
(3) Game elements: Not all participants will like the same features at all times, hence it is
important to stimulate interest by providing, e.g. team challenges on a regular basis.
(4) Narrative: Participants should be able to follow a personalized approach in the project. This
approach might be, e.g. location, time, or interest based and can be stimulate by personalized
narratives.
The main take away for the Flamenco consortium is that although gamification is a useful instrument,
it cannot be simply implemented in every context and expected to be effective. The mechanics that
are used need to be tweaked and tailored towards the project goal, the target audience and their
demographics and self-understanding, as well as the specifics of the activity that needs to be
performed. Within Flamenco, it should further be investigated which elements (e.g. scoring and
feedback mechanisms) can support citizen science projects and should thus be provided as an option
to project initiators to include in their campaigns.
Promoting participation – interim report 47
6 CONCLUSION
This deliverable provides some insights on the key mechanisms that motivate citizens to partake in
participatory sensing campaigns. We consecutively looked at the concepts of citizen science and
participatory sensing, the reasons and barriers for choosing a CS approach, the factors that influence
participation and gamification elements that can be used as a strategy to steer behavior and
contribution. At the end of most of the chapters, we already provided an overview of what can be
learned for Flamenco. In Table 8, we summarize the different insights
N° Requirement
Initiators
1 The platform should provide a way to determine who initiates the CS project and what their
corresponding skill level is.
2 Depending on the skill level of the initiator, guidelines need to be provided on how to conduct
a CS project, the steps that need to be included and how they should go about each step. This
might require the creation of a semi-automatic workflow composition supporting the initiator
when needed.
3 Flamenco should clearly communicate its purpose, benefit and value to potential societal
stakeholders.
Participants
4 The platform should provide a way to determine who participates in the CS project, and what
their corresponding skill level is.
5 Depending on the skill-level and influence of the participant, guidelines should be provided
on how to help out in the citizen science campaign. Instructions could go from how to do a
proper data collection to how to correctly interpret the results.
6 Contribution should be reciprocal as in, participants who are interested should also feel that
they get something out of it. In this respect, it is paramount to know what motivates the
specific participation at the start of the project, so these can be supported if possible. As a
consequence, participants feel valid contributors and not just instrumental data collectors.
7 Create a dynamic contribution environment, with this they refer to enabling the participants to
gradually take on more demanding tasks and take steps up the “ladder of the responsibilities”.
Promoting participation – interim report 48
Don’t make it too difficult at the beginning, make the tasks more challenging over the course
of the project and give enough feedback so participants can actually learn.
8 Give the participant information on how the project and its results impacts him/herself and his
environment.
9 The project could provide a matchmaking possibility for support between those participants
who appear to have had a scientific training and those who are new to the process.
Communication and recruitment
10 Awareness of the project should be created using as many communication sources as
possible, tailored to the public and its specific context.
11 The objective of the project, and especially the underlying societal goal, should be
communicated clearly, and regularly towards the (potential) pool of participants. It should not
promise things that are not part of the project as this will cause disillusionment.
12 (Potential) Participants have different literacy levels, and not all feel that they are capable of
contribution. If the goal is to include as many people as possible, a big proportion should be
convinced that they do have the right skills, or that they can pick them up along the way.
13 Working together and feeling part of a community is well received by contributors. Without
overburdening them, establishment of a strong community of volunteers, e.g. through the use
of social media or an in-platform communication environment is mandatory.
14 Keep the language used simple and friendly.
15 Send acknowledgements and give value to the data collection, notifications of reception,
affirmations that their contribution is being used, information about the status of the project.
Give the participants an opportunity to review the results and witness the impact on the initial
cause.
Technical factors
16 The Flamenco platform should thus be technologically able to support such intense data
collection from different projects at the same time.
17 Flamenco should thus provide mechanisms so data over the projects could be shared with
those interested, if the participants and initiators have agreed upon opening up the data.
Promoting participation – interim report 49
18 Different stakeholders initiating different CS projects should be easily able to contact each
other.
19 Flamenco should provide mechanisms for the sharing of relevant documentation on the topic
between initiators
20 The tools used need to be accurate, of high-quality and user-friendly for the participants.
Proper and clear instructions on how to use them should be provided as well, enhancing the
trustworthiness of the overall project.
21 Sensors and other devices used by the participants should be easy to couple with the platform.
Gamification
22 The application of gamification elements can contribute to a higher and sustainable
participation in CS projects. But be aware that its applicability depends on demographics and
self-understanding of the group. It should thus be personalized and consider participants
wishes and needs.
23 Be aware that gamification can also demotivate participants if they are not successful in
competing.
24 Scoring mechanisms: Scoring mechanism should be detailed enough to show continuous
personal progression next to being a means of comparison with competitors.
25 Personalized feedback: Feedback should help participants to assess the quality of their
contribution, rather than its quantity, to ensure that a learning process is taking place.
26 Game elements: Not all participants will like the same features at all times, hence it is
important to stimulate interest by providing, e.g. team challenges on a regular basis.
27 Narrative: Participants should be able to follow a personalized approach in the project. This
approach might be, e.g. location, time, or interest based and can be stimulate by personalized
narratives.
General / Other
28 Enough time should be given to the participants to complete their tasks.
29 Report with results of a CS project should be easily generated.
Promoting participation – interim report 50
30 Part of the platform should give place to documentation about the topic at hand, so
participants can learn more if wanted.
31 Aim for a relationship of trust between participants and organizers.
As a follow-up we will examine how the distinguished concepts will be applied in a framework for
analyses of these and other CS case studies. The latter will be comprehensively presented in the next
iteration of this deliverable (D2.1b: Integrated report on promoting citizen participation).
Promoting participation – interim report 51
7 BIBLIOGRAPHY
Alender, B. (2016). Understanding volunteer motivations to participate in citizen science projects: A
Deeper look at water quality monitoring. Journal of Science Communication, 15(3), 1–19.
Aoki, P., Woodruff, A., & Willett, W. (2017). Environmental Protection and Agency : Motivations ,
Capacity , and Goals in Participatory Sensing, 3138–3150.
Aparicio, A. F., Vela, F. L. G., Sánchez, J. L. G., & Montes, J. L. I. (2012, October). Analysis and
application of gamification. In Proceedings of the 13th International Conference on Interacción
Persona-Ordenador (p. 17). ACM.
Bennett, W. L. (2008). Changing Citizenship in the Digital Age. In Civic Life Online: Learning how
Digit (pp. 1–24). http://doi.org/10.1162/dmal.9780262524827.001
Beza, E., Steinke, J., van Etten, J., Reidsma, P., Fadda, C., Mittra, S., … Kooistra, L. (2017). What
are the prospects for citizen science in agriculture? Evidence from three continents on motivation and
mobile telephone use of resource-poor farmers. Plos One, 12(5), e0175700.
http://doi.org/10.1371/journal.pone.0175700
Blake, J. (1999). Overcoming the 'value-action gap' in environmental policy: Tensions between
national policy and local experience. Local Environment: The International Journal of Justice and
Sustainability, 4 (3): 257-278.
Blohm, I., & Leimeister, J. M. (2013). Gamification: Design of IT-based enhancing services for
motivational support and behavioral change. Business & Information Systems Engineering
(BISE), 5(4), 275-278.
Bonney, R., Ballard, H., Jordan, R., McCallie, E., Phillips, T., Shirk, J., & Wilderman, C. C. (2009).
Public Participation in Scientific Research: Defining the Field and Assessing Its Potential for
Informal Science Education. CAISE Public Participation in Scientific Research Inquiry Group
Participants. Washington. Retrieved from http://www.ukeof.org.uk/resources/citizen-science-
resources/MotivationsforCSREPORTFINALMay2016.pdf
Bowser, A., Hansen, D., Preece, J., He, Y., Boston, C., & Hammock, J. (2014). Gamifying citizen
science. Proceedings of the Companion Publication of the 17th ACM Conference on Computer
Supported Cooperative Work & Social Computing - CSCW Companion ’14, (February), 137–140.
Promoting participation – interim report 52
Burgess, H. K., DeBey, L. B., Froehlich, H. E., Schmidt, N., Theobald, E. J., Ettinger, A. K., …
Parrish, J. K. (2017). The science of citizen science: Exploring barriers to use as a primary research
tool. Biological Conservation, 208, 113–120. http://doi.org/10.1016/j.biocon.2016.05.014
Chatzigiannakis, I., Mylonas, G., & Vitaletti, A. (2011). Urban pervasive applications : Challenges ,
scenarios and. Computer Science Review, 5(1), 103–118. http://doi.org/10.1016/j.cosrev.2010.09.003
Chou, Y: (2013). Octalysis: Design Breakdown of the Speed Camera Lottery. Gamification.co [Web
blog post]. Retrieved from http://www.gamification.co/2013/04/25/gamification-breakdown-of-the-
speed-camera-lottery/.
Conrad, C. C., & Hilchey, K. G. (2011). A review of citizen science and community-based
environmental monitoring: Issues and opportunities. Environmental Monitoring and Assessment,
176(1–4), 273–291. http://doi.org/10.1007/s10661-010-1582-5
Cooper, C. B., and J. A. Smith. 2010. Gender Patterns in Bird-related Recreation in the USA and UK.
Ecology and Society 15:13.
Crowston, K., & Prestopnik, N. R. (2013). Motivation and data quality in a citizen science game: A
design science evaluation. In System Sciences (HICSS), 2013 46th Hawaii International Conference
on (pp. 450–459). IEEE. Retrieved from
http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6479888
Curtis, V. (2015). Online citizen science projects : an exploration of motivation , contribution and
participation. The Open University.
Darnton, A. (2008). Practical Guide: An overview of behaviour change models and their uses. GSR
Behaviour Change Knowledge Review. London. Government Social Research Unit.
Dawnay, E. & Shah, H. (2005). Behavioural economics: seven principles for policy makers.
London: New Economics Foundation.
Debacq, C. (2016). Motivatie en Communicatie bij Burgerwetenschapsprojecten omtrent
Luchtkwaliteit - Obstakels en Opportuniteiten. Vrije Universiteit Brussel.
Deterding, S. (2011, May). Situated motivational affordances of game elements: A conceptual model.
In Gamification: Using game design elements in non-gaming contexts, a workshop at CHI.
Deterding S., Dixon, D., Khaled, R., & Nacke, L. (2011). From game design elements to gamefulness:
defining "gamification". In Proceedings of the 15th International Academic MindTrek Conference:
Envisioning Future Media Environments (MindTrek '11). ACM, New York, NY, USA.
Promoting participation – interim report 53
Dickel, S., & Franzen, M. (2016). The “Problem of Extension” revisited: new modes of digital
participation in science. Journal of Science Communication, 15(1).
Dickinson, J. L., Shirk, J., Bonter, D., Bonney, R., Crain, R. L., Martin, J., … Purcell, K. (2012). The
current state of citizen science as a tool for ecological research and public engagement In a nutshell :
http://doi.org/10.1890/110236
Downes-Le Guin, T., Baker, R., Mechling, J., Ruyle, E., 2012. Myths and realities of respondent
engagement in online surveys. Int. J. Mark. Res. 54, 613–633.
Eveleigh, A., Jennett, C., Lynn, S., & Cox, A. L. (2013, October). “I want to be a Captain! I want to
be a Captain!”: Gamification in the Old Weather Citizen Science Project. In Proceedings of the first
international conference on gameful design, research, and applications (pp. 79-82). ACM.
Foster, J.A., Sheridan, P.K., Irish, R., Frost, G.S., 2012. Gamification as a strategy for promoting
deeper investigation in a reverse engineering activity. In: Proceed- ings of the 2012 American Society
for Engineering Education Conference, pp. AC 2012–AC 5456.
Geoghegan, H., Dyke, A., Pateman, R., West, S., & Everett, G. (2016). Understanding Motivations
for Citizen Science. Final Report on behalf of the UK Environmental Observation Framework
(UKEOF), (May), 124. Retrieved from http://www.ukeof.org.uk/resources/citizen-science-
resources/MotivationsforCSREPORTFINALMay2016.pdf
Gharesifard, M., & Wehn, U. (2016). To share or not to share: Drivers and barriers for sharing data
via online amateur weather networks. Journal of Hydrology, 535, 181–190.
http://doi.org/10.1016/j.jhydrol.2016.01.036
Gharesifard, M., Wehn, U., & Van Der Zaag, P. (2017). Towards benchmarking citizen observatories:
Features and functioning of online amateur weather networks. Journal of Environmental Management,
1–13. http://doi.org/10.1016/j.jenvman.2017.02.003
Haklay, M. (2013). Citizen Science and Volunteered Geographic Information: Overview and
Typology of Participation. In D. Sui, S. Elwood, & M. Goodchild (Eds.), Crowdsourcing Geographic
Knowledge (pp. 105–122). Dordrecht: Springer Netherlands. http://doi.org/10.1007/978-94-007-4587-
2
Haklay, M. (n.d.). Citizen Science and Policy : A European Perspective, 4.
Irwin, A. (1995). Citizen Science: A Study of People, Expertise and Sustainable Development.
Routledge.
Promoting participation – interim report 54
Hamari, J., & Koivisto, J. (2015). Why do people use gamification services?. International Journal of
Information Management, 35(4), 419-431.
Jennett, C., Kloetzer, L., Schneider, D., Iacovides, I., Cox, A. L., Gold, M., … Talsi, Y. (2016).
Motivations, learning and creativity in online citizen science. Journal of Science Communication,
15(3), 1–23.
Kamenica, E. (2012). Behavioral Economics and Psychology of Incentives. Annual Review of
Economics, 4, 427-452.
Land-Zandstra, A. M., van Beusekom, M. M., Koppeschaar, C. E., & van den Broek, J. M. (2016).
Motivation and learning impact of Dutch flu-trackers. Journal of Science Communication, 15(1), 1–
26.
Lane, N. D., Eisenman, S. B., Musolesi, M., Campbell, A. T., & Miluzzo, E. (2008). Urban Sensing
Systems : Opportunistic or Participatory? In Ninth Workshop on Mobile Computing Systems and
Applications (HotMobile 2008) (pp. 11–16). Napa Valley.
Liu, Y., Alexandrova, T., Nakajima, T., 2011. Gamifying intelligent environments. In: Proceedings of
the 2011 International ACM Workshop on Ubiquitous Meta User Interfaces. Presented at Ubi-MUI'11.
ACM, pp. 7–12.
Liu, H., Kobernus, M., Broday, D., & Bartonova, A. (2014). A conceptual approach to a citizens ’
observatory – supporting community-based environmental governance. Environmental Healh,
13(107), 1–13.
Mason, A.D., Michalakidis, G., Krause, P.J., 2012. Tiger nation: empowering citizen scientists. In:
Proceedings of the 2012 6th IEEE International Conference on Digital Ecosystems Technologies.
Presented at DEST 2012. IEEE, Campione d’Italia, pp. 1–5. 10.1109/DEST.2012.6227943.
Massung, E., Coyle, D., Cater, K., Jay, M., Preist, C., 2013. Using crowdsourcing to support pro-
environmental community activism. In: Proceedings of the 2013 ACM SIGCHI Conference on
Human Factors in Computing Systems. Presented at CHI'13. ACM, Paris, France, pp. 371–380.
Martin, V. Y. (2017). Citizen Science as a Means for Increasing Public Engagement in Science.
Science Communication, 107554701769616. http://doi.org/10.1177/1075547017696165
Murphy, R. J. (2015). Facilitating Citizen Science Through Gamification(Doctoral dissertation, PhD
thesis, Memorial University of Newfoundland).
Mccrory, G., Veeckman, C., Coppens, P., & Claeys, L. (2017). D6.1 : Engagement strategy for
hackAIR community involvement. Brussel.
Promoting participation – interim report 55
Nicholson, S. (2012). A user-centered theoretical framework for meaningful gamification. Games+
Learning+ Society, 8(1), 223-230.
Nov, O., Arazy, O., & Anderson, D. (2011). Dusting for science: motivation and participation of
digital citizen science volunteers. Proceedings of the 2011 iConference: Inspiration, Integrity and
Intrepidity, iConference 2011, 68–74.
Norman, D. (1988). The psychology of everyday things. New York: Doubleday.
Pocock, M. J. O., Tweddle, J. C., Savage, J., Robinson, L. D., & Roy, H. E. (2017). The diversity and
evolution of ecological and environmental citizen science. PLoS ONE, 12(4), 1–17.
http://doi.org/10.1371/journal.pone.0172579
Prestopnik, N., & Crowston, K. (2012a). Exploring collective intelligence games with design science:
A citizen science design case. In ACM Group Conference. Retrieved from
http://crowston.syr.edu/system/files/designing%20citizen%20science%20games.pdf
Prestopnik, N., & Crowston, K. (2012b). Purposeful gaming & socio-computational systems: a citizen
science design case. In Proceedings of the 17th ACM international conference on Supporting group
work (pp. 75–84). ACM. Retrieved from http://dl.acm.org/citation.cfm?id=2389188
Rapp, A., Marcengo, A., Console, L., Simeoni, R., 2012. Playing in the wild: enhancing user
engagement in field evaluation methods. In: Proceedings of the 16th International Academic
MindTrek Conference. Presented at MindTrek 2012. ACM, Tampere, Finland, pp. 227–228.
Reed, M. S. (2008). Stakeholder participation for environmental management : A literature review.
Biological Conservation, 141(10), 2417–2431. http://doi.org/10.1016/j.biocon.2008.07.014
Riesch, H., & Potter, C. (2014). Citizen science as seen by scientists: Methodological, epistemological
and ethical dimensions. Public Understanding of Science, 23(1), 107–120.
http://doi.org/10.1177/0963662513497324
Rotman, D., Preece, J., Hammock, J., Procita, K., Hansen, D., Parr, C., … Jacobs, D. (2012).
Dynamic changes in motivation in collaborative citizen-science projects. Proceedings of the ACM
2012 Conference on Computer Supported Cooperative Work - CSCW ’12, 217.
http://doi.org/10.1145/2145204.2145238
Rose, K.J., Koenig, M., Wiesbauer, F., 2013. Evaluating success for behavioral change in diabetes via
mHealth and gamification: MySugr's keys to retention and patient engagement. Diabetes Technol.
Ther. 15, A114.
Promoting participation – interim report 56
Rose, D. H., & Meyer, A. (2002). Teaching every student in the digital age: Universal design for
learning. Association for Supervision and Curriculum Development, 1703 N. Beauregard St.,
Alexandria, VA.
Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic
motivation, social development, and well-being. American psychologist, 55(1), 68.
See, L., Mooney, P., Foody, G., Bastin, L., Comber, A., Estima, J., … Rutzinger, M. (2016).
Crowdsourcing, Citizen Science or Volnteered Geographic Information? The Current State of
Crowdsourced Geographic Information. International Journal of Geo-Information, 5(5).
http://doi.org/10.3390/ijgi5050055
Soman, D. (2015). The last mile: Creating social and economic value from behavioural insights.
Toronto: Rotman-utp Publishing.
Seaborn, K., & Fels, D. I. (2015). Gamification in theory and action: A survey. International Journal
of Human-Computer Studies, 74, 14-31.
Thiel, S. K., Reisinger, M., Röderer, K., & Fröhlich, P. (2016). Playing (with) Democracy: A Review
of Gamified Participation Approaches. JeDEM-eJournal of eDemocracy and Open Government, 8(3),
32-60.
UWE. (2013). Science for Environment Policy In- depth Report: Environmental Citizen Science.
Retrieved from http://ec.europa.eu/science-environment-policy%0AAcknowledgements
West, S. E. 2015. Understanding participant and practitioner outcomes of environmental education.
Environmental Education Research 21:45-60.
Wiggins, A., & Crowston, K. (2011). From conservation to crowdsourcing: A typology of citizen
science. Proceedings of the Annual Hawaii International Conference on System Sciences.
http://doi.org/10.1109/HICSS.2011.207
Wilson, P. (1973). Situational relevance. Information storage and retrieval, 9(8), 457-471.
Williams, C. R., Hawthorn-Jackson, D., Orre-Gordon, S., & O’Sullivan, S. (2017). Some cautions in
the use of citizen science: a case study of urban insect collection. Transactions of the Royal Society of
South Australia, 0(0), 1–13. http://doi.org/10.1080/03721426.2016.1268564