research now conducting successful online focus groups november 2007 london paris hamburg frankfurt...
TRANSCRIPT
Research Now Conducting Successful Online Focus Groups
November 2007
London
Paris
Hamburg
Frankfurt
Athens
Toronto
New York
San Francisco
Chicago
Sydney
Melbourne
• Introduction
• Recruitment
• Focus Group examples
• Case Study
• Conclusions
Contents
Introduction
Background – Research Now
• Research Now is a leading international provider of online fieldwork to the market research industry, and our focus is on quality across all elements of our business
• We are fieldwork only – we are therefore not in competition with market research agencies – we will therefore focus this presentation on the enabling side of online focus groups today
• We conduct an average of 550 online projects globally per month
• However we have only conducted a handful of online focus groups for clients, and our own pilot studies and have gathered some valuable learning.
• We have also conducted qual intercept and bulletin board techniques to a greater extent, these are not covered today.
Specific Aims Today
Aims of this session….
• Today we are going to walk through the ‘enabling’ of an online focus group
• Less theoretical, you are the methodologists and will make the ultimate decision on appropriate application, though we will make some suggestions
• More practical – what are the nuts and bolts? Stages? How do you effectively execute an online focus group (including case study)?
• Then we will revisit the question….
Do online focus groups provide a good context for engaging with
Participants online?
Online Qualitative
Context
Background – Online Qualitative
• Some of the earliest online qualitative methods in the UK occurred around 1998• NOP with Bulletin Boards• Virtual Surveys with Moderated Email Groups (MEGS)
• One of the pioneers of online qualitative groups was Michael Herbert of Michael Herbert Associates, using a modified ‘chat room’ format – CyberQual
• Developed his own qualitative recruitment panel (offline and online interaction)
• However recruitment was almost exclusively conducted by phone.• Only very internet savvy / early technology adopters were able to
participate• Broadband required, and penetration very low!
• Different dial up speeds were an issue, meaning that Participants with greater speed access had an advantage in putting their views across
• Lack of familiarity with the internet meant this method was very niche and only appropriate for technology savvy audiences (very niche consumer groups or IT Managers)
• Acceptance of online research as viable was very low, and it has only been in the last 2-3 years that clients are comfortable with online
Background – The Online Audience
• Around 10% quantitative fieldwork is now conducted online in the UK – what about qualitative?
• Over two-thirds of the population are online, and over 40% have broadband access
• Researchers are increasingly looking to the internet to power their qualitative solutions
• Consumers are more and more internet-savvy and are more used to engaging socially online thanks to the proliferation of online communities and networking sites such as Facebook, Friends Reunited, MySpace and YouTube
Do online focus groups, therefore, provide a good context for engaging with
Participants online?
Background –Why an online qualitative solution?
- Traditionally, running a qualitative project with face-to-face
focus groups can be stressful and time-consuming:
- Need to travel to a variety of cities and venues
- Need to organise for clients to attend and view the group
- Need to find a mutual date / time that Participants, the
moderator and the client (s) can make
- Need to interpret information and produce summaries of
research findings from multiple groups quicklyConducting focus groups online can address
most, if not all of these issues
Background – Online Qual and The International
Perspective
- Face-to-face groups are a logistical
challenge for researchers looking to
conduct multi-country research
- Online focus groups allow you to talk to
Participants from multiple
geographical locations for one
single group or conduct multiple
groups in a single day
- You can also invite clients to log-in
and tune in to discussions wherever
they are in the world
Reach Participants across the world from one single touchpoint
Overview of the Online
Focus Group Process
Steps in the focus group process
• Constructs are very similar to offline qualitative. As you probably anticipate it follows the same steps with some notable differences….
• Recruitment: Can be offline and traditional for an online group, alternatively online access panels or the online environment can be suitable• Of course, a broad geographical dispersion is possible using
online focus groups – for very low incidence samples this maybe the only context that you can engage in a qualitative exercise – for example, apple iphone purchasers
• The Focus Group : Online Groups mirror all of the facets of a traditional focus group:• Moderator with discussion guide• Stimulus• Participants• Two way mirror / client area• Yellow post it’s from client to researcher
• Analysis: Transcripts available, and ‘recording’ of focus groups
Online vs Offline – Summary of The Benefits
Offline
Online
Group Feature
x No travel required
Simultaneous client viewing
x Include Participants from different cities / countries in the same group
x Conduct multiple groups across multiple locations in one day
? Recruit niche Participant sample in a matter of hours
Use your own moderator
? Seamlessly play audio / video clips and web content
x Engage with the ipod generation in their own “environment”
x Instant transcripts
Recruitment
Recruitment
- Traditional offline recruitment methods still act as a very powerful tool for
engaging participants for online focus groups
- Beyond the practical benefits of offline recruitment, panel recruitment (online)
can also add value to your research by:
- Providing the more elusive Participants quickly and easily – e.g. find
male owners of Audi TT’s in Manchester and London, or female users of a
brand of deodorant under 30 in a matter of hours rather than days
- Engaging with a recruitment source that are already used to giving their
opinions in the online modality. Certainly they are more likely to be found
IMing, Blogging and interacting socially online than in the local shopping mall
- One of the major strengths of online panel is it’s broad geographical
dispersion. This means that for focus groups we can bring people together
where previously it would have been impossible
It is possible to fuse qual and quant data collected through panel to
create very powerful insight into behaviour
Online Focus Groups
Types of Focus Groups
Fall into two broad types….
More Basic ‘Chat Room’ style online focus groups
•Moderator and Participant with basic identification
•Standardised transcript output
‘Rolls Royce’ Applications (becoming the norm).
• Usually include the ability to have a discussion guide pre-loaded into the group
• A client area• The ability to show stimulus and potentially video• Emoticons and sounds to simulate emotion, for example applause,
gasps etc.
•Standardised transcript output
Example – Participants view
1. Participant portal
Participant sees
moderator
questions
displayed on
screen and can
type responses
Participant can
also see other
participants
comments in an
open screen
Example – Moderators View
2. Moderator portal
Moderator types in
live questions or
selects a question
to ask from the
pre-loaded
discussion guide
Example – Client Viewpoint
Client participation via a
split screen –
Can view Participant
discussion as well as
communicate with
colleagues
Because the moderator is
able to communicate in
both rooms, instant
client-moderator
communications are
achievable
3. Client portal
Example - Stimulus
Moderator can
display visuals
to the group
Example- Stimulus
Moderator can
show group
websites or
other live online
content within
the focus group
environment
TRANSCRIPTS:
Instantly
available
Case Study
Case Study – Background
Research Now conducted 2 pilot groups in August 2007 amongst our own
panelists to get some diagnostic feedback on the ‘panel experience’, and
to fully test the process of conducting a focus group
The Pilot was designed to give us valuable feedback in three main areas:
1. Recruitment – what is the most effective way of recruiting to an online focus
group?
2. To test the effectiveness and technical stability of our online focus group package
3. To examine Participant engagement once panelists were focus group participants
Case Study - Schedule
Pilot Group 1 Pilot Group 2
Recruitment commenced Friday 3rd Sunday 12th
Participants confirmed Monday 6th Monday 13th
Reminder sent Tuesday 7th Tuesday 14th
Date Wednesday 8th Wednesday 15th
Time of Group 6.30pm UK time 6.30pm UK time
Total no. RN panellists contacted 500 410
Total no. RN panellists recruited 12 20
Total no. group participants 2 7
Length of group 1.5 hours
Incentive structure £20.00, if you do not qualify for the survey you will receive a free entry into our quarterly prize draw for £250
Recruitment Criteria Recruits to be UK panellists – mix of ages / gender between 18-24 –
all regular survey takers
Valued Opinions Panellists – random selection – qualitative research
Project managers Rahul Krishna, Lucy Swain and Dominic Jarville (RN)
Moderator Research Now (John Caldwell)
Group Subject The Valued Opinions Panel Experience
Case Study - Recruitment
Case Study – Recruitment
For both pilot groups, recruitment was conducted in 2 stages:
The first stage was a general screening survey, which was sent to 400-500 selected
RSN panelists to establish eligibility and interest……(more in a minute)
The second stage was used to select and invite eligible panelists to the specific
group events
Case Study – Recruitment Stage 1
ScreenerWe have an innovative new survey available for you to take. As a valued member of our panel we would like to invite you to an online session next week, where you can discuss your experience on our panel and the new technologies we are introducing to improve this experience.
If you complete the survey and undertake a one hour online session with ourselves you will receive £20.00, if you do not qualify for the survey you will receive a free entry into our quarterly prize draw for £250. These next few questions will help us determine what group to put you in. Please be advised that none of this information will be passed to any third parties.
Are you willing to take part in this Valued Opinions study taking place on Wednesday, August 8th…? Yes No
Case Study – Recruitment Stage 1
Do you or any member of your family work in any of the following industries? Please select THOSE THAT APPLY
Advertising (screened out) Public RelationsMarketing and sales Civil ServiceIT Professional (screened out) Market research (screened out) Catering/Restaurant None of the above
Are you… Please select one answer belowMaleFemale
Which of the following ranges includes your age? Please select one answer belowUnder 18 45-5418-24 55-6525-34 66+35-44 Prefer not to say
Which of the following bests describes the region of the UK you live in? PLEASE SELECT ONE ANSWER BELOW
North East (Tyne Tees) East of England (Anglia TV)North West (Granada TV) South East (Meridian TV)Yorkshire & The Humber (Yorkshire) South West (West country TV)East Midlands (Central TV) London (Carlton TV/LWT)West Midlands (Central TV) Wales (Wales/HTV)Scotland (Grampian TV/STV/Border) None of these
Case Study – Recruitment Stage 1
Your answers so far indicate your profile fits the needs of this research study. We are undertaking this study twice on the Wednesday, August 8th.
There is a limit to the number of people taking part in each session to ensure that your opinion and views are properly heard. Please indicate which of the following time you would prefer to help us determine which chat sessions are available to you
-3:30 pm – 4:30 pm Wednesday, August 8th-6:30 pm – 7: 30 pm Wednesday, August 8th
Please enter you mobile phone number…
This will only be used to send an SMS to remind you of your session time.
Case Study – Recruitment Stage 1
For both Pilots, around three quarters of those screened where eligible
and interested in taking part in this group
As with offline groups, it was important to ensure that the screener was just a
screener, and not a mini survey in its own right. We advise that you should only use
10-12 simple, closed-end questions for a recruitment screener
In our experience, the screener often needs adapting (many come from face-to-face
recruitment templates) or they get very vague screening briefs e,g “Mums with kids
under 2”.
In both cases, the screener needs some adaptation for an online audience and
clarification with the client in terms of any quotas or demographic splits required .
Case Study – Recruitment Stage 2
The second stage was to select and invite a selection of eligible Participants to
the group. Having a pool of eligible Participants to choose from allowed us to
balance the group demographically
Stage 2 focus group invitees were given more detailed information on what to
expect in the group and how to access the focus group room. Key information
communicated at this point included:
•Dates and times for the focus group
•A request to arrive a little in advance of the start time to allow time for
everyone to get into the group
•Details on the incentive offered (and rules about how the Participant
would qualify for the incentive)
•Information on how to log on to the group
•Note on limited space – designed to encourage the Participant to
turn up
•Information on who to contact should there be any technical problems
Case Study – Recruitment Stage 2 – Over-recruitment
• We over-recruited by 160% per group (with some adjustments for under and
over responsive demographic sub segments – e.g 15-24 males are typically the
worst responders!)
• In terms of timing, we would advise that screeners are sent out prior to a
weekend for a group scheduled to happen mid week
• For both of these focus groups we scheduled both pilots mid week, with the
stage 1 screener happening before the previous weekend, and stage 2
recruitment on the Monday, with a reminder on the Tuesday for a group on the
Wednesday
• For both case studies, no-one dropped out between the initial recruitment to
the reminder stage
Case Study – Incentive and Timing of Focus Group
• We pitched our incentive at £20 for 90 minutes and this seems to have worked
with our panellists
• It is important to get the timing of the focus group right so that the group is
scheduled at a time that is convenient to panellists.
• In particular, if the Participant demographic is full-time workers, it may be
more appropriate to conduct a group in the evening. Conversely, if the target
is students or an older demographic, a time during the day may well be more
appropriate.
• This also needs to be co-ordinated with a time that the client and moderator is
available and calculated to avoid any major national or sporting events that
may tempt Participants away from the group
• A good practice that we adopted for the first pilot was to ask the panellists we
screened what their preferred time was. Interestingly, feedback did vary by
gender demographic, highlighting that this is an important consideration
when agreeing times for the group
Case Study – Timing of Focus Group
Preferred TimeFemal
eMale Total
3:30 pm – 4:30 pm Wednesday, August 8th 5 14 19
6:30 pm – 7: 30 pm Wednesday, August 8th 13 13 26
Grand Total 18 27 45
Case Study – Recruitment
FINDINGS
• Attendance for focus group pilot 1 was poor (only 2 people turned up from a
recruited pool of 12)
• Attendance for focus group pilot 2 - 7 took part in the group, indicating that
recruitment rate for the second group had improved on the first.
Case Study – Recruitment
FINDINGSWhen examining reasons for poor attendance in pilot 1, two factors
seemed to have had an impact on success rates:
1. The group demographic was unreliable – we targeted young 18-24 year olds
– despite lining up 12 people to take part in the group at their preferred time
and reminding them by SMS the day before, it seems that this group were
unreliable. We think that perhaps this age group is less committed to taking
part in groups and may be swayed by last-minute changes of plan. By contrast,
when we mixed up the age groups for pilot 2, we saw much better attendance
rates.
2. The wording in the recruitment invitation about over-recruitment: “There is
limited space availability, so only the first to arrive will be able to
participate” seems to have put some Participants off turning up for the
group.
We amended the invitation for the second pilot, and response seems to
have been better.
Recruitment Recommendations (1)
1. Be clear in your instructions – make sure password and link information that you
send is correct!
2. Don’t put off Participants from turning up to the group by indicating that they
may be rejected (i.e ‘please ensure you turn up at least 20 minutes before the start.
Places will be given on a first come first serve)
3. Make sure that Participants are told that clients may be watching the group – this
is a MRS code of conduct requirement for all focus groups
4. Try and set the scene and give as much detail as possible about what the
experience will be like to manage expectations
Recruitment Recommendations (2)
5. Vary your recruitment strategy to suit your target demographic e.g over-recruit
young people by up to 3x
6. Time the group so that is appropriate for your target audience
7. Consider making the process more personal with a quick phone call reminder
rather than an e-mail / SMS to engender more loyalty to turn up to the group
8. Ensure technical support is available at the moment the group is about to start –
this is the most crucial moment and the moment you are most likely to lose
Participants!
Case Study – Technical
Case Study – Technical FINDINGS
Troubleshooting
• We also set up a helpdesk for participants at: [email protected]
for contacting us more than 30 minutes prior to the groups. We also provided
support via e–mail 30 minutes before the group and during the group. This
worked well in practice, but we would recommend a phone number, available
to panelists in case they have any online difficulties and need to chat to
someone
.
Setting up for the group
• We set up the room for the group a couple of days before the group was due to
take place. This enabled the moderator to go in and familiarise themselves
with the group settings and pre-load the discussion guide. In a live client
situation, we’d suggest taking advantage of this as a way of reassuring clients
and giving moderators some pre-group training
Case Study – Technical FINDINGS
Starting the groups
• We recommended that the moderator and a “hostess” are present in the focus
group room prior to the start time to welcome in Participants and clients who
turned up early.
• In both groups we piloted, we had people turn up ahead of schedule, and it
was good to have someone there to welcome and encourage them into the
group and keep them talking while others arrived.
• It is advisable to have some general topics lined up to engage Participants who
have turned up early or on time while you wait for the stragglers to log in.
• The hostess also provided a few “rules” of engagement for the group and a
general encouragement to speak up and express opinions. Topics covered
included:
1. Introducing the moderator
2. Telling Participants where the emoticons were
3. Re-assuring Participants that typos didn’t matter
Technical Recommendations
1. Make it as easy as possible for Participants to log in – use existing names and
passwords if you are recruiting from a panel source where possible
2. Set up a helpline for technical issues (preferably a phone and e-mail option in
case the Participant is experiencing internet connection issues)
3. We would encourage the moderator to visit a practice room ahead of the live
group
4. Ensure there is a “hostess” on-hand to welcome participants into the group,
explain the rules and turn away any “extra” Participants or late-comers
5. Have a project manager on-hand to trouble-shoot and call up non-showing
Participants
Case Study – Participant
Engagement
Case Study – Participant Engagement FINDINGS
Moderating
•The software we use allows moderators to get ahead of the game by pre-loading
questions, links to visuals and websites and Participant polls that could be activated
during the course of the group. This means that the moderator does not have to
cause any delays to the group by typing out each question and can be more reactive
to what people are saying.
•Whilst moderating online is different to a face-to-face environment, the key
principles are still the same. As with offline groups, the moderator needs to check
that everyone is fully engaged and responding (it is easier for a Participant to “leave
the room” online that it is offline!) and encourage everyone to give their feedback.
•The moderator therefore needs to keep track of who has said what (having the
transcript in front of you definitely helps here) and allow sufficient time for everyone
to respond (this can take a bit longer online to allow time for typing). However, the
advantage is that Participants can respond at once.
•As with offline groups, the moderator needs to keep an eye on time (in both of our
pilots, time went really quickly and we had to really keep an eye on the pace) and
update Participants on where they are at / how much longer there is to go etc.
Case Study – Participant Engagement FINDINGS
Participant Engagement
• In practice, we found that Participants remained well-engaged in the groups
and in most cases, were happy to go beyond the original time set for the group
(we had trouble getting rid of some of them!).
• You can tell when someone logs out of the group (either self-imposed or
through a connection problem) make sure that you have someone on hand to
diagnose connection problems.
• Some Participants can feel intimidated by a face-to-face group full of strangers
and can feel pressurized to respond “in the right way” to the moderator’s
questions. Online, there is more of a sense of anonymity, which can make
some Participants feel less inhibited about expressing their viewpoints. .
Here is an example of a transcript output from our pilot groups….
Case Study – Participant Engagement FINDINGS
…………Moderator: Have you ever done any Market research In-person or over the Telephone with an interviewer?
Resp 1: No
Resp 4: I’m always a bit scared of phone interview....not entirely sure why
Resp 3: In person a few times
Resp 5: I have not been interviewed over the phone but have done it when I worked. I think it would good to do one.
Moderator: Do you feel that you are able to give more complete or direct responses online compared to other ways of conducting Market research?
Resp 4: I think some people like the fact that they can take their time over a survey, and that they aren’t being judged by their response, unlike in a telephone situation… where people might think they are
Resp 5: I think it’s good to do surveys online because you have more time to think about what you want to say
Resp 3: yes, I actually prefer doing them online
Case Study – Participant Engagement FINDINGS
Moderator: Would you like to take part in these types of online research methods in the future?
Resp 4: blogs?
Resp 1: I like this type of Focus group. It's more interesting and more interactive.
Resp 4: Focus group is interesting…I think you can squeeze more out of people in this sort of social situation
Resp 5: I have never done a focus group before but it is good to talk to other people about research
Moderator: Do you ever feel that you would like to talk directly with the Market Research companies that commission the surveys?
Resp 1: Not really. Online surveys are quite isolated…yes...and ask them why it takes them 10 minutes to tell me I'm not what they want.
Resp 4: Sometimes, I get curious why they ask some of the questions they ask, and I feel like I want to ask them.
Resp 3: Yes, I would
……………………………………….
Participant Engagement Recommendations
1 Where possible, pre-load the discussion guide – this gives you more time to
manage and engage with the responses of the group.
2. As with offline, make sure everybody is engaged!
3. At the start of the group, it can be a bit chaotic! However it does settle down in to
a nice pattern once everyone gets to know each other
4. Keep an eye on the time – it really does go quickly.
5. The moderator should have an opportunity to familiarise themselves with the
software they are using prior to the start of the group
6. Think about how you will want to analyse output from the groups and build in
time if the transcript needs translating or analysing in non-word document format
Overall Considerations in
Applying this method
Types of Application..
Qual Only
• Business Travellers / Time poor individuals who are geographically dispersed.
• Young and Technology Savvy audiences that are already used to conversing in this
way through social networking sites.
• As part of customised communities developed by yourselves and your clients.
Qual and Quant
• Arguably the most powerful application for online focus groups and other online qual
techniques…
• As part of a diary study, where you want to convene participants and glean
richness of insight around their behaviour from the previous week
• Concept development, where qual gives you the depth of understanding and
information, whilst quant gives you the breadth of information and context to
deliver very rounded findings and conclusions
Conclusions
Conclusions
Yes they do, and it is possible for you to use the flexibility of the internet to create some quite cutting edge approaches, leveraging:…
Broad Geographical recruitment – National and International.
Possibility to ‘reconvene’ groups quite cost effectively using online focus
groups
Possible to link in quantitative metrics captured alongside the qualitative
exercise
Are online focus groups any better than traditional offline focus groups?
NO! They are not a substitute, and it is difficult to make the comparison as
they are different. Clearly non verbal communication is of vital importance and something that cannot be captured online well / yet.
However there are some undeniable benefits of using online to capture
insight, and in some instances it is the only available option within cost / time parameters to gather insight into consumer and B2B sub segments.
Do online focus groups provide a good context for engaging with
Participants online?
Any Questions?
Chris DubreuilClient Development Director
Research Now PlcElizabeth House
39, York Road, London SE1 7NQ
T : + 44 (0) 207 7921 2400M : + 44 (0) 7974 170011
www.researchnow.co.uk