game on qualitative researchers: using gamification to increase partipant engagement, data quality...

What to

expect

We believe gamification can be applied in 3 different phases of the

research process; (1) during data collection, (2) during analysis and

interpretation and (3) during reporting and presentation of the

results. In this paper, we present an approach to gamification

in online qualitative research. There is already ample research

with respect to using gamification in quantitative research; however,

a comprehensive approach for online qualitative research is

lacking so far.

In this paper we will focus on using gamification during data

collection and will briefly demonstrate how we apply

gamification in the last 2 phases. At InSites Consulting, we

identified 4 levels in an online community at which

gamification can be applied to increase data quality,

participant engagement and impact on the client side. From a

question level to a community level, gamification helps, not only to

increase participant engagement, but also to increase data quality.

Introduction

In the last decade, we saw a tremendous shift in

engagement with brands. Consumers are

looking for brands and companies who

engage with them and offer them an

experience. Brands and companies which do not

interact or offer experiences to their consumers

tend to fade away, while brands that are inclusive

tend to flourish. This has already been showcased

plenty of times in the field of marketing and

marketers have reacted, by changing their

marketing plans by including more engaging

marketing actions (Van Belleghem, 2010; 2012).

However, the lower consumer engagement

problem has also hit the market research

industry. Response rates only go down while

straight lining and quickly replying to a question

become more and more prevalent. In addition, we

are now also facing a demanding and tech-

savvy new generation we want to do

research with.

Gen Y is used to constant stimulation and having a say

in everything. In order to engage with them, brands

need to fulfill these expectations (Van den Bergh and

Behrer, 2011 for an elaborate discussion on the

characteristics of Generation Y). Finally, the last

decade has been marked by disruptive changes in

technology, with the rise of social media and

mobile technology evolving at an exponential

rate. Everybody is now connected to everybody

24/7. People have become used to an on-demand

lifestyle where everything is available at their fingertips

anytime.

Clearly, market research needs to adopt its

offering to participants and, by extension,

clients, in order to keep engagement high. As a

solution for the problem of waning engagement, some

researchers have suggested adopting gamification in

market research. The reason behind this is that

games are very effective at engaging people

with a certain topic or task. Playing games is

popular with everybody, with gaming being popular in

different age groups and both for men and women.

36% of gamers are over 36 and 45% of gamers are

female (ESA, 2013). However, we believe

gamification is only part of the solution, as it is

not sufficient to solve the engagement problem.

To structurally solve the engagement problem, market

research needs to empower participants, create a

two-way dialogue with them and offer them a

stimulating (visually appealing & cognitively

stimulating) and 24/7 environment where they can

freely speak their mind. Given the broad scope of

this, we will focus in this paper on increasing

engagement via gamification.

The problem of decreasing engagement has been

troubling researchers in the industry for a while now

and money is not the issue, as increasing monetary

incentives has not been able to solve this problem.

The real problem lies within market research

itself and what we offer participants. We know

these people like to be in an interactive environment,

expect anything at their fingertips 24/7 and want to

feel empowered. However, our offer towards them

consists mainly of surveys which force them to

follow a certain question flow. These questions

are repetitive, not cognitively stimulating and can be

extremely long. With focus groups, people feel more

empowered and the questions are normally more

cognitively stimulating. However, here also,

participants don’t have the opportunity to

interact with the brand or feel like they have a

big impact on the brand. In addition, it is not in

line with the 24/7 availability to which they have

grown accustomed.

Definition

Gamification is not the same as turning research into

a game. We at InSites Consulting use the following

definition from Deterding et al. (2011)

“Gamification is the use of game design

elements in non-game contexts.”

The goal of gamification is to improve the data

quality by applying game elements in research,

thus motivating participants to participate longer and

provide richer answers.

Game elements or game mechanics, as explained

by Donato and Link (2013) are actions, tactics or

mechanisms used to create an engaging and

compelling experience for consumers. For

example, by including levels or using challenges,

participants can be stimulated to engage more with

the research at hand. These game mechanics

work because they tap into game dynamics,

which refers to the motivations people have

to engage with a game: (1) achievement, (2)

social and (3) immersion (Yee, 2006).

Achievement relates to wanting to become better

at something, analyzing and solving problems and

outperforming others. Social motivations to engage

with a game relate to meeting and interacting

with new people, making friends, building

relationships and contributing to team

efforts. Finally, immersion relates to discovering

new things, taking on new roles and a form of

escapism.

Gamification is not a monolithic concept; it rather is a

spectrum with different shades of gamification.

On one hand, we identify gamification of research.

Jon Pulleston is a pioneer in this type of gamification,

with a plethora of research into the gamification of

surveys. On the other extreme of the spectrum we

identify “researchification” of games. Betty

Adamou drew the blueprints for this type of

gamification. Here, the research is embedded in a

game environment, immersing participants in a

game which also includes research questions

(Adamou, 2011). Although these represent different

shades of gamification, both methods help to create

more engagement and collect better data. Before

applying gamification to your research, there are two

important caveats which need to be taken into account.

Firstly, applying gamification can be a costly

endeavor. It costs time and money to implement

it and it may not be worthwhile to invest a lot in

gamifying a short survey, as the return would not

outweigh the investment made.

Secondly, gamification needs to support the

research and promote the desired behavior (e.g.

answer creatively, give a spontaneous reaction, etc.). It

does not have the purpose to entertain people. By

tapping into game motivations, the research experience

improves for the participant and better data is obtained.

An improved experience for the participant is a

means, but not an end goal of gamification. It is

important to ask yourself whether the game mechanics

you want to introduce will enhance your data or not.

Andy Barker, Engage Research and Lisa Hunt, Heinz

(2012) were among thefirstt o explore gamification in

offline qualitative research. They operationalized

gamification by setting challenges and using game-like

exercises such as making a sentence about soup by

asking each member to say one word that followed on

that stated by the person before them.

Participants were more enthusiastic and engaged

while participating. Answers were similar to those

collected on the Heinz Facebook page. On the downside,

Barker noted that the games are not introspective in nature

and mentioned that the games disrupted the group

dynamics. This case highlights the importance of

applying gamification strategically to enhance data

quality. As Barker and Hunt explored gamification more

broadly and put more emphasis on making the research

more game-like (instead of using game motivations to

enhance the quality of the data rather than the experience of

the research) their results show increased engagement, but

not better data. Another example of applying gamification

without focusing on the research questions comes from our

own experiences. At InSites Consulting, we used to have

leader boards on the homepage of a community that

displayed the ranking of the participants. The person

with the most posts was on top and the person with

the least posts at the bottom.

They concluded that gamification has pros and cons

What we noticed, however, was that gamification was

not improving the research because of two

problems. Firstly, the leader board encouraged the

more vocal people to become even more vocal while

the more timid participants felt discouraged and became

more silent. Secondly, the leader board was not rewarding

the behavior we wanted to encourage. People posted

more replies, but these posts were not richer or adding

much to the topic. In this case, the gamification

rewarded posting a lot and not posting richer

answers. If you want to benefit from gamification and

collect better data, gamification should not only create a

more engaging research experience, but should stimulate

desired behaviour you want to improve (based on the

research objectives). This could be richer answers

that provide more context, more emotional depth

or more creativity.

They concluded that gamification has pros and cons

Current status

Typically, gamification in market research has been studied in surveys. However, looking at the concept of

gamification we would like to argue that there are equally many - if not more - reasons to apply

gamification in qualitative research as well.

1 Firstly, from an investment perspective, it makes sense to also focus on

qualitative research. Donato and Link mention that the cost of developing

gamification mechanics in surveys might not be worthwhile given the non-

length of a short survey. However, typically qualitative research tends to be

longer in nature than quantitative research, making the investment more

worthwhile.

2 Secondly, in qualitative research we use projective techniques and

motivational techniques that are akin to gamification. For

example, participants are often challenged to come up with pro and con

arguments. We ask participants to take on a different role and think from

this perspective; we send them on missions (e.g. shopper missions) and

challenge them cognitively by asking analogies. This shows that

gamification is ideally suited to be applied in qualitative research.

At InSites Consulting, we believe MROCs - or Consumer Consulting Boards

as we call them (De Wulf & De Ruyck ed., 2013) - are powerful qualitative

research tools which help solve a myriad of business questions.

MROCs are composed of a small group of people, brought together on a

platform to answer and discuss questions from a moderator for a longer period

of time (three weeks or longer). The discussions are organized in different

topics and rooms. As this is the main qualitative research method at

InSites Consulting, we investigated the fit between gamification and

MROCs.

1 Firstly, given that MROCs are used for qualitative research and that we just explained how

qualitative research already is a type a gamification, MROCs are well suited for

gamification.

2 Secondly, participants in an MROC need to be engaged for three weeks or longer. As

engagement needs to be built up and maintained, an MROC is more suited than

a 30-minute survey. During three weeks, people can be immersed in the research topic

and are taken on a research journey, providing more context for gamification and different

gamification mechanics can be applied, resulting in more impact.

3 Thirdly, the online environment of an MROC makes that some game mechanics can be

automated and hence made more scalable.

4 Finally, while in surveys a social layer can be added, the social dimension in MROCs

is stronger and more natural. This helps to stimulate the social game dynamics in

people.

Looking at MROCs, there are four reasons why this method is

Especially suited for applying gamification.

Gamification in MROCs

As there is a high need to research gamification in

qualitative research and there is a high fit between

MROCs and gamification, we decided to invest in

this topic. In order to tap into the different

underlying game motivations (see earlier), we

decided to apply gamification on four levels.

On the lowest level, the question level, we

apply gamification by rephrasing the

questions. For example, to understand what

Generation Y considers to be the cool places in

their city, we posted a challenge. Instead of asking

them to list the cool venues and the reason why

these places are cool, we challenged the

participants to prove their city was the coolest to the

client (MTV). We told them that, by the end of a

given week and based on their answers, MTV

would select the coolest city. This spurred elaborate

reactions from our Gen Y participants who were

fiercely defending their city.

Figure 1. Examples of badges

One level higher, on the individual level, we

introduce badges and levels. By answering to

topics posted by the moderator, participants gain

points. Depending on the quality of the post,

participants gain more or less points. Additionally,

participants collect badges (see Figure 1) by

performing certain actions such as posting

very creative replies on a specific topic,

keeping specific topics alive and interactive…

Every day, the number of contributions would be

counted and the country with most posts would

score one point (see Figure 2). The effect of this

was a significant increase in posts by

everybody and people encouraging each other

to answer to the questions and to take part in

the game.

Figure 2. Football derby between The Netherlands and Belgium

The third level is the group level. This level is

not always applicable, as it requires having at least

two different groups in the research. On this level,

gamification can be applied by setting a

challenge for all groups and then compare the

result between them. For example, in a research

we did with R&D people at Unilever, we split the

entire R&D team in several smaller groups. These

groups could then compete to show who knows the

consumer best. The winning team would then be the

first to get access to new and exclusive content,

namely new products in the pipeline. Another

example of applying gamification on a group level

comes from a community we did for Initial. We had

participants from both the Netherlands and Belgium

on the community, for three weeks. Between both

countries there has always been friendly rivalry. So

during the community we organized a derby between

both countries.

Finally, gamification can also be applied on a community level,

where the complete group of participants is involved. This is where

we set challenges for the community such as completing a certain task by

the end of a given week or reaching a given amount of posts in a given

timespan. Another example would be the ideation tool which we use to

find new insights. Here, the community searches and thinks together to

come up with new ideas and solutions. Participants can upload ideas as

an answer to a need or problem. Others can rate these ideas and

comment on them to further improve them. This way the community

works together on a need or problem that is close to their heart

and is personally relevant.

Measuring the results

1 Firstly, gamifying a community makes participants think harder. This entails several

results. We receive seven times more on-topic arguments compared to a non-gamified

community (De Ruyck, Knoops, Schillewaert, Coenen & Rogrigues, 2011). In addition,

participants provide us with more context when answering, enabling us to better

understand and frame their answers. Another consequence is that participants give

emotional richer answers which allow us to understand them more completely. Additionally,

in a gamified community, we get more creative answers.

2 Secondly, a gamified community allows us to make people think differently. When

people approach a topic from another perspective, we can again better understand the

topic and come up with different insights. For example, for Chiquita we had to research

the potential of a fruit smoothie with both people who eat healthily and people who eat less

healthily and who don’t eat fruit regularly. In a first phase, we explored the perception and

attitudes towards eating healthily and eating fruit for both groups. In the second

phase, we decided to apply gamification by having the groups switch roles in an activation

deprivation exercise. The group of healthy eaters had to decrease their fruit consumption for

one week and the group of unhealthy eaters had to eat a certain amount of fruit every day, for

one week. Both groups then had to report back to us, providing us in the end with a complete

picture on why people eat fruit and why not.

Looking at MROCs, there are four reasons why this method is

Gamification beyond data collection

As mentioned in the introduction, the market research world needs to

adopt its offering to both participants and clients. So far, we focused on

applying gamification during data collection to increase participant engagement

to collect better and more data. However, we believe gamification can also

be applied beyond data collection, further increasing participant

engagement and also client engagement. Gamification can be applied

during the analysis phase and the reporting phase as well.

interpretation game which exists of three rounds during

which a selected group of participants (n=10) can collect

points. The different rounds are analogous to the analysis

process in qualitative research, where we go from observation to

interpretation and to interpretation of the interpretation. In a first

round, participants receive answers from other participants and

are asked to interpret these. In a second round, they receive the

same answers, but this time are asked to think deeper and

provide more details.

Analysis

At InSites Consulting we apply gamification during the analysis phase as well, following the principles of

crowd interpretation (Surowiecki, 2004; Verhaege, 2011). Crowd interpretation helps us understand the data

even better, providing us with initial insights. Imagine for a second that you are doing a research project with

people who have a very rare condition. Even though you might be an experienced and well-trained

qualitative researcher, if you do not have that rare condition yourself, you will always have some knowledge

gaps. This is where crowd interpretation comes into play. By asking our participants who suffer from the

condition to interpret certain observations, they can add a different perspective that we - as researcher - are

unable to spot. In order to make this even more engaging, we developed the crowd

After this, all the interpretations are

collected and sent back to the original

poster of the answer. This person reads how

others interpret his/her answer and then scores

each interpretation on how good it is. Based on

this, a ranking is made with the best participant

researchers. The crowd interpretation game

has two benefits, one on participant level

and one on researcher level. For participants,

the crowd interpretation game is rewarding as

they are more engaged in the game and

because making them part of the analysis makes

them feel more empowered. On a researcher

level, having participants interpret their

answers means we get richer data and are

able to gather 20% to 40% more insights

(Verhaege, 2011).

During the reporting phase, there is another opportunity to apply

gamification; this time to increase engagement on the client side,

thus creating internal leverage for the research project. The goal of

gamification during this phase is to create positive

disruption at the client side by exposing their knowledge

gaps. Before presenting the report, different stakeholders at the

client side test their knowledge of the topic. They are then given

their score. The lower the score (or the bigger the knowledge gap),

the more eager people are to read and reflect on the report. The

example of the project with the R&D people at Unilever highlights

the benefits (De Ruyck, Knoops, Schillewaert, Coenen & Rogrigues,

2011). The R&D people’s goal was to learn as much as possible

about their consumer. To show they knew their consumer, they

had to play a game about consumers where they could

collect points for each correct answer. There were different

quizzes and after each quiz the scores of the different teams were

advertised in different places throughout the R&D offices.

Reporting

To make sure the designers would use the insights,

we created an interactive app that immersed the

designers in the results, allowing them to interact

with the insights. In addition and to further

stimulate engagement with the results, the

designers had to come up with ideas based on the

results. Every uploaded idea contributed to

the personal score of the designers and the

different scores were displayed on the home

page. Figure 3. The Heineken ODE community platform

The game increased engagement significantly and the report was not only used more

frequently, but also reflected upon more. This meant that the R&D people spent more

time thinking about the results and their implications. Another example would be the

Heineken Open Design Explorations case (De Ruyck, De Boeck, Eising, Troch & Van Hoff, 2012;

see Figure 3. The Heineken ODE community platform). In this case we collected data and insights

about what going out means to young and trendy people. Heineken’s goal was to take these insights

and use them to build the club of the future with upcoming designers from different fields.

and differently (the ability to get a 360°

perspective on consumers’ perceptions and

attitudes), resulting in better and more data. The

prerequisite for this being that the gamification is

focused on increasing engagement with the

research questions (instead of providing a general

nice environment). In addition, to further increase

the market research offering, we have shown that

gamification can also be applied beyond data

collection, namely during the analysis and

reporting phase.

Finally, because of the nature of MROCs,

we are able to develop a scalable model to

apply gamification that keeps a healthy

balance between the investment made and

the benefits gained.

Conclusion

As a result of an outdated offering and

changed consumer reality, the market

research industry is faced with a declining

engagement. Part of the solution to increase

engagement is to apply gamification throughout the

research process. Other methods to increase

engagement include: creating a more engaging

environment that is visually appealing and

stimulates exploration, empowering people and

starting a two-way dialogue as well as offering

flexible solutions that bring convenience to

participants. This paper presents a first account

of the effects of gamification in qualitative

research and more specifically in MROCs. In

addition, it also provides a framework on how to

apply gamification beyond data collection. We have

shown that applying gamification to an MROC can

make people think harder (providing more context,

more emotional and creative answers and seven

times more on-topic arguments)

References

Adamou, B. (2011). The Future of Research Through Gaming. CASRO Journal.

Barker, A. & Hunt, L. (2012). Game on to keep consumers engaged. Marketing

Week.

De Ruyck, T., De Boeck, F., Eising, H., Troch, T., & Van Hoff, C. (2012). Designing the

club of tomorrow. ESOMAR General Congress.

De Ruyck, T., Knoops, S., Schillewaert, N., Coenen, G. & Rogrigues, S. (2011).

Engage, Inspire, Act: 3 Step Stones towards Developing more Impactful

Products. ESOMAR General Congress.

Deterding, S., Sicart, M., Nacke, L., O’Hara, K. & Dixon, D. (2011). Gamification.

using game-design elements in non-gaming contexts. CHI '11 Extended Abstracts

on Human Factors in Computing Systems. 2425-2428.

De Wulf, K. & De Ruyck, T. (2013). The Consumer Consulting Board: Consumers

shaping your business. Belgium: InSites Consulting.

.

Donato, P. & Link, M. W. (2013). The gamification of marketing research.

Marketing News. 38-42.Enterntainment Software Association (2013). Essential facts

about the computer and video game industry. (source:

http://www.theesa.com/facts/pdfs/ESA_EF_2013.pdf)

Van Belleghem, S. (2010). The Conversation Manager. Belgium: Lannoo.

Van Belleghem, S. (2012). The Conversation Company: Boost Your Business

Through Culture, People and Social Media. London: Kogan Page.

Van den Bergh, J. & Behrer, M. (2011). How Cool Brands Stay Hot: Branding to

Generation Y. London: Kogan Page.

Verhaege, A. et al. (2011). Crowd Interpretation: are participants the researchers

of the future? ESOMAR, General Congress.

Yee, N. (2006). Motivations of Play in MMORPGs: results from a Factor Analytic

Approach. CyberPsychology & Behavior, 9(6): 772-775.

Tom De Ruyck Head of Consumer Consulting Boards

InSites Consulting

Sébastien Van Laere Research Consultant

InSites Consulting

Anouk Willems Research Innovation Manager

InSites Consulting

www.insites-consulting.com

Thank you!

@InSites

marketing@insites-consulting.com

www.facebook.com/insitesconsulting

www.slideshare.net/InSitesConsulting