the 2012 confirmit market research software...

The 2012 Confirmit Market Research Software Survey

Full report

Ninth annual survey by meaning ltd

Report written by Tim Macer and Sheila Wilson, meaning ltd

independent advice on technology for research

Confirmit 2012 Annual MR Software Survey: Full report | 2 of 63

Contents

1 Background to the survey 7 1.1 The Sample 7 1.2 Demographics 8

2 Part 1: New questions 9 3 Survey length 10

3.1 Survey length: the shift to shorter interviews 11 3.2 Methods adopted to shorten surveys 12 3.3 Natural survey length 13 3.4 Natural survey length: distribution of responses 15

4 The future for mobile apps 17 4.1 Preferences for apps or browsers for mobile surveys 18 4.2 Predicting the future for apps and browsers 19

5 Panel rewards and quality control 20 5.1 Prevalence of incentivised panel surveys 21 5.2 Distribution of incentivised panel use 22 5.3 Incentives used 23 5.4 Survey quality measures applied 25 5.5 Survey quality measures at a detailed level 26 5.6 Use of independent panel verification services 27

6 Communities 28 6.1 Who runs communities? 29 6.2 Change in uptake since 2009 30 6.3 Number of communities operated 31 6.4 Predicted growth for communities 32 6.5 Software used to administer communities 33 6.6 Required capabilities in community software tools 34 6.7 Satisfaction with community software capabilities 35

Part 2: Trends 36

7 Research activities 37 7.1 Research mix – 6-year trend 38 7.2 Company revenues from research in 2012 39


7.3 Volume of work 40 7.4 Research modes offered – mainstream modes 42 7.5 Research modes offered – minority modes 43 7.6 Changes in the amount of work 44 7.7 Reported smartphone usage in web surveys 45

8 Sources of online sample 46 8.1 Proportion of online sample in use - trend 47 8.2 Proportion of online sample in use in 2012 48 8.3 Online sample in use - trend 49

9 Mixed mode 50 9.1 Importance of mixed mode 51 9.2 Use of integrated platforms 52 9.3 Level of mixed mode support required 53

10 Analysis and results reporting 54 10.1 Distribution methods in use 55 10.2 Distribution methods in use, by company size 56 10.3 Change in demand for high tech delivery 57 10.4 Importance of producing cross tabs in volume 58

11 Changing software 59 11.1 Planning on changing software in 2012 60 11.2 Planning on changing software - trend 61 11.3 Software type to change 62 11.4 Reasons for changing software 63


List of tables Table 1 Sample composition: company size by region 8

Table 2 Sample composition: responsibility and seniority 8

Table 3 Descriptive statistics for the estimates of the natural interview length for surveys 16

Table 4 Incentives used frequently – by global region and company size 24

Table 5 Incentives used frequently or occasionally – by global region and company size 24

Table 6 Quality control measures by global region and company size 26

Table 7 Proportion of quantitative research by mode 41

List of charts Figure 1 Changes reported in survey length 11

Figure 2 Methods adopted to shorten surveys 12

Figure 3 Natural limit for interview length for telephone, online and mobile surveys 14

Figure 4 Distribution of the stated natural limit of different interview modes, in minutes 15

Figure 5 Preference for app or browser when delivering mobile surveys 18

Figure 6 Predictions for which method will dominate in future 19

Figure 7 Extent to which panel participants are incentivised 21

Figure 8 Distribution of the prevalence of incentivised panel as a sample source 22

Figure 9 Incentives used – frequently or occasionally 23

Figure 10 Chart title 25

Figure 11 Adoption of independent panel verification services (among panel owners) 27

Figure 12 Communities operated in 2012 by region and company size 29

Figure 13 Number of communities operated in 2009 and 2012 31

Figure 14 Predicted growth for communities 32

Figure 15 Software used to administer communities 33

Figure 16 Required capabilities in community software tools 34

Figure 17 Satisfaction with community software capabilities: ranked from good to poor 35

Figure 18 Proportion of company revenues that derive from research activities, annual trend38

Figure 19 Proportion of company revenues that derive from research activities, by region and company size 39

Figure 20 Volume of work by mode 40

Figure 21 Percentage of market research firms using the main modes of research 42

Figure 22 Percentage of market research firms using the minor modes of research 43

Figure 23 Anticipated changes in the amount of work over next three years 44


Figure 24 Percentage of web survey participants taking surveys on smartphones and other small format mobile devices: 2011 and 2012 45

Figure 25 Proportion of online sample in use 47

Figure 26 Proportion of online sample in use, by region and company size 48

Figure 27 Online sample sources in use 49

Figure 28 Importance of mixed mode 51

Figure 29 Use of integrated platform versus switching between different platforms 52

Figure 30 Level of mixed mode 53

Figure 31 Distribution methods in use 55

Figure 32 Distribution methods in use, by company size 56

Figure 33 Anticipated change in demand for high tech delivery methods 57

Figure 34 Importance of being able to produce volumes of printed cross tabs 58

Figure 35 Planning on changing software, by region and company size 60

Figure 36 Planning on changing software, by year 61

Figure 37 Software type wishing to change 62

Figure 38 Reasons for wishing to change software 63


Acknowledgements

We are greatly indebted to Confirmit for the assistance they have provided in supporting the survey financially, and also technically, for providing us with the use of their software for this survey, and for their significant contribution they have made to the content and design of the survey.

We are also most grateful for the support provided by CASRO (www.casro.com), JMRA (http://www.jmra-net.or.jp/index-e.html) and Quirk’s Publishing (www.quirks.com) in publicising this survey among their own members and subscribers, and inviting participation. This has been especially helpful in increasing the coverage of this survey among groups that had proved harder to reach in the past.

We also thank Ascribe (www.goascribe.com) for their generosity in coding the open-ended questions in this survey, and Lloyd Irving of Magnify Solutions for programming the survey for us.

Tim Macer and Sheila Wilson, meaning ltd London, 2013


1 Background to the survey

Meaning ltd has been carrying out this annual software survey since 2004, making this the ninth successive year of the survey. The survey provides a unique set of information and insights into the interplay of technology and methodology within the market research industry. It provides a snapshot of current usage and attitudes and predictions from practitioners, and identifies trends from a number of tracking questions that are asked repeatedly each year.

In addition to identifying general trends, each year, the survey also focuses on several areas of topical interest. For the 2012 survey, these topics are:

• Survey length (and the impact of mobile surveys on this)

• Attitudes to apps versus browsers for mobile surveys

• Survey quality initiatives and the use of incentives in panels

• Research communities: uptake and software requirements

This year (as in several previous years), the survey software company Confirmit has kindly sponsored the survey and provided financial and technical assistance that support its administration. This is acknowledged in the survey’s full title, which is the 2012 Confirmit Market Research Software Survey by meaning.

1.1 The Sample

The 2012 survey comprises a sample of 250 market research companies globally, and only of individuals who are responsible for, influential in or aware of technology decisions within each company. The sample is managed to ensure representation of three global regions: North America, Europe and Asia Pacific according to the relative amount of research carried in these regions from data published by ESOMAR.

The survey consists of a self-completion interview on the Web, comprising around sixty questions and timed to last approximately fifteen minutes. As it is difficult to identify the target individual within a research company, sample is obtained from a variety of sources:

§ Participants who agreed to be re-contacted from the previous year’s survey

§ Sample compiled by meaning ltd

§ Sample provided by the survey’s sponsor, Confirmit

§ Sample from 2010 and 2011 surveys

§ Emailed invitations by CASRO (Council of American Survey Research Organizations) and by JMRA (Japan Market Research Association) to an open survey link.

§ Several links inviting participation from mailshots and on the homepage of the research publication Quirk’s Marketing Research Review to an open survey link.


All emailed invitations are de-duplicated across all sources prior to invitations being sent, and de-duplication is again applied to completed interviews for the small number of cases where more than one response has been received from the same company within that country.

Responses from open survey links accounted for 6% of the sample. These were additionally asked to provide company information, and this was used to verify their eligibility for the survey – i.e. that they were a research company.

The nature of the sampling method, and its eligibility criteria (i.e. senior IT decision makers within research organisations) makes it impossible to provide a meaningful estimate of response rate, because invitations are sent to individuals without prior knowledge of their eligibility, and those who are not eligible are unlikely to respond to the survey invitation.

1.2 Demographics

This report is based on 250 participants in unique research companies within each country and covers 37 countries. The composition of the demographics referred to in this report is as follows:

Size Total N America Europe Asia Pacific

Small 145 58% 55 56% 72 64% 18 45% Medium 63 25% 26 27% 23 21% 14 35% Large 42 17% 17 17% 17 15% 8 20% Total 250 100% 98 100% 112 100% 40 100%

Table 1 Sample composition: company size by region

Company size is defined by the declared revenue of the organisation. A company is considered ‘small’ if its revenue is below $5 million or fewer than 50 employees, and large if it exceeds $25 million or 500 employees. ‘Small’ is intended to reveal the different needs of companies unlikely to have specialist in-house technical staff. The smaller proportion of companies in the ‘large’ (or over $25m turnover) category is only a reflection of the pyramid that exists of company size, with a smaller number of large companies globally.

The survey is only asked of senior individuals who are involved in IT decision-making within their respective company. The level of responsibility and role of the participants is shown below:

Role N % Level of responsibility N %

Research 88 35% Board Member 67 27% IT 25 10% Primary IT decision maker 106 42% Data processing 32 13% IT Decision influencer 77 31% Business or Operational 105 42%

Table 2 Sample composition: responsibility and seniority


2 Part 1: New questions

Each year, the survey also focuses on several areas of topical interest. For the 2012 survey, four related topics have been explored.

Topics in this section

§ Survey length § Mobile survey apps versus browsers § Panel rewards and survey quality measures § Communities


3 Survey length

Questions

1. Are surveys getting any shorter?

2. What, if anything, is being done to shorten surveys?

3. What is the natural length of a survey on the phone, web or mobile?

Key findings

Only a minority report that surveys are getting any shorter…

§ Despite the fact that survey length is well understood to be a significant factor in poor response and poor engagement by respondents, it is only a third of companies that are reporting any overall reduction in survey length

§ For the majority, it’s business as usual, including some who say length continues to increase.

…but many companies are at least taking action

§ Half of the companies surveyed have a company policy that seeks to limit survey length.

§ A minority of firms are using other measures such as adaptive surveys and semi-structured questions to replace large grids or batteries of questions

Seven or eight minutes is the limit for a mobile survey

§ There was broad agreement across the whole sample on the natural limit for different types of survey – 20 minutes for telephone and online, but just 8.3 minutes for mobile.

§ A small minority consider surveys of one or two hours duration are OK.

§ The median, which is less affected by these outliers, gives optimum interview lengths of 18 minutes for telephone surveys, 15 minutes for web and 7 minutes for mobile.

But you may get a bit longer in Asia Pacific

§ Companies in Asia Pacific cited slightly longer, times for all modes, especially mobile.


3.1 Survey length: the shift to shorter interviews

à There is a weak trend towards shorter surveys

à A third say they are getting shorter while 1 in 8 say they are still getting longer

à 50% see no change

à Small firms and Asia Pacific lead the way on shorter surveys

The question was introduced with this preamble: “Survey methodologists have identified excessive survey length as an important factor in diminished participation in surveys by the public.” The situation that emerges is very mixed – the largest group see no change; beyond that, more than twice as many report a decrease in length as report an increase.

Q: To what extent have you observed any decrease in the length of the surveys that your company administers?

N = 250

Figure 1 Changes reported in survey length

10%!

8%!

7%!

20%!

12%!

5%!

7%!

26%!

30%!

25%!

20%!

26%!

24%!

31%!

50%!

46%!

53%!

50%!

48%!

54%!

48%!

13%!

13%!

13%!

10%!

12%!

16%!

12%!

2%!

3%!

2%!

0%!

2%!

2%!

2%!

Total!

North America!

Europe!

Asia Pacific!

Small!

Medium!

Large!

Most of our surveys are now shorter! Some are now shorter!

No overall change! Surveys are getting longer!

Not sure!


3.2 Methods adopted to shorten surveys

à 50% of firms have a policy or best practice guide to limit survey length

à The adaptive survey is cited as a popular method to reduce survey length

à Breaking projects into smaller surveys or using semi-structured questions are less popular methods

Our interpretation of these data is that firms are not doing as much as they could to address the issue of survey length. Professional codes of ethics or conduct all state the importance of collecting no more data than is necessary and many also speak of respondent burden, so we would expect virtually all companies to have policies that reflect this. It is possible that they do, and those participating in this survey are unaware of these. However, the other more technical methods of limiting survey length are uncommon, with the exception of adaptive surveys – using intelligent routing to ensure no single participant is faced with an excessive number of questions.

Q: Which of these methods, if any, does your company use, to limit survey length?

N = 250

Figure 2 Methods adopted to shorten surveys

50%!

42%!

24%!

21%!

13%!

By adopting policy or best practice guidance!

Adaptive surveys!

Break up projects into several smaller surveys!

Substitute semi-structured/open questions for question batteries!

Other measures!


3.3 Natural survey length

à The mean average suggested is 20 minutes for phone and web; 8.3 minutes for mobile

à Slightly longer surveys tolerated in Asia Pacific

à Small firms are the most cautious about survey length

This question was asked as an open numeric question. Most responses were rounded to multiples of 5 minutes.

Overall, there is very little difference in the mean average survey length proposed by participants. Behind this, there is of course, a wider variation. The maximum survey length proposed was a mind-numbing 120 minutes for telephone and web, and an optimistic 34 minutes for mobile.

It is our general experience that the actual interview lengths on a great many commercial market research projects for telephone and online, far exceed the optimum lengths stated here.


Q. What do you consider is the natural limit for these kinds of surveys, in interview length… telephone, online and mobile?

N = 250

Figure 3 Natural limit for interview length for telephone, online and mobile surveys

8.3!

8.2!

8.0!

9.5!

19.9!

19.8!

19.4!

21.4!

19.9!

20.2!

19.2!

21.3!

Total!

N America!

Europe!

Asia Pacific!

7.9!

8.6!

9.4!

18.1!

21.2!

23.9!

18.8!

21.4!

21.4!

Small!

Med!

Large!

Telephone survey!

Online survey!

Mobile survey!


3.4 Natural survey length: distribution of responses

à Median lengths probably give a better estimate: 18 mins for phone, 15 for web and 7 for mobile

à 49% think 5 minutes is the natural limit for mobile

à Agreement is strong, with the majority of responses for all modes fairly close to the mean

The difference in the shape of these three curves are interesting. Phone and online have a much longer tail than mobile because there are several outliers where atypically large lengths were given. This has the effect of making the mean a bit higher than the media, and shows there is a minority that think surveys can be very much longer than the majority do.

Q. What do you consider is the natural limit for these kinds of surveys, in interview length… telephone, online and mobile?

N = 250

Figure 4 Distribution of the stated natural limit of different interview modes, in minutes

Telephone and online can be considered to follow a positively skewed curve (i.e. the centre of the curve is to the right of where it might be expected to be on the chart, if it were a normal distribution). On the other hand, the distribution for mobile is asymmetric – it is only one half of a curve, with the peak on the extreme left of the chart.

0%!

10%!

20%!

30%!

40%!

50%!

<5 mins!

10! 15! 20! 25! 30! 35! 40! 45! 50! 55! 60! Don't know!

Telephone! Online! Mobile!


The values are more tightly clustered than a normal distribution, with 87% of the responses for telephone lying within one standard deviation (SD) of the mean; 89% for online and 74% for mobile (in a normal distribution 68% of values would be expected to fall within one SD of the mean and 98% within 2 SDs.) This can also be observed in the high Kurtosis scores. These and other descriptive statistics for these distributions can be seen in Table 3, below.

Telephone Online Mobile

Mean 19.91 19.85 8.34 Median 18 15 7 Std dev 12.36 12.75 5.09 Skew 3.27 3.56 1.81 Kurtosis 19.09 20.46 4.80 Std err 0.782 0.806 0.322

Table 3 Descriptive statistics for the estimates of the natural interview length for surveys


4 The future for mobile apps

Questions

1. Which is better for mobile surveys today: app or browser?

2. Is this likely to change in the future?

Key findings

Mobile browser surveys have the edge – for the moment

§ Participants in this survey are fairly divided on whether the app or browser is the most viable way to administer mobile surveys, with most considering either equally valid.

§ Among those favouring one or the other, it is the browser that has the edge today.

Apps are likely to catch up, or even surpass the browser in the future

§ The gap between apps and browsers closes when respondents were asked for their future prognosis on which will dominate.

§ Among some sections, apps are predicted to edge ahead of browsers

Mobile research is now considered viable by the vast majority

§ There was broad agreement, especially in Asia Pacific, that mobile research is now a viable method – only one in ten consider that mobile as a whole is not viable.


4.1 Preferences for apps or browsers for mobile surveys

à The browser is slightly more popular than the app

à Apps hold more appeal in Asia Pacific and among large firms, but even here they don’t eclipse the browser

à Apps hold the least appeal in North America

à One in ten still think that mobile research isn’t viable yet – but far fewer in Asia Pacific

Bets are fairly well hedged on whether using an app or the browser will get better results, with one third of the sample thinking both have equal merit – a view we share. But among those favouring one or the other, it is the browser that has more credibility at present.

Q: Which method do you consider is most viable at this time for administering mobile surveys - mobile app or mobile browser?

N = 250

Figure 5 Preference for app or browser when delivering mobile surveys

18%!

11%!

21%!

25%!

19%!

13%!

24%!

39%!

43%!

37%!

35%!

39%!

40%!

38%!

34%!

35%!

33%!

38%!

32%!

43%!

29%!

9%!

11%!

9%!

3%!

10%!

5%!

10%!

Total!

N America!

Europe!

Asia Pac!

Small!

Medium!

Large!

App! Browser! Both equally viable! Neither viable at this time!


4.2 Predicting the future for apps and browsers

à A reversal is predicted in several sectors: Europe, Asia Pacific and small firms are backing apps to dominate

à North America, along with mid-size firms think the browser will be the more dominant

à Overall, and specifically in large companies, the gap is predicted to close

Mobile apps, and mobile generally, have not penetrated into much of research yet, as can be seen from our tracking questions on the methods being used. However, the mobile app is expected to have an important part to play in the future – and not necessarily at the expense of browser-delivered mobile surveys.

The most likely outcome is that there will be a place for both technologies along side each other – even though the data do not explicitly show this, because the ‘neither will dominate’ group is just 10% overall, and is fairly consistent across all groups – but a large group, by stating not sure, seem to be saying it is too close to call at present.

Q. Which method do you think will dominate in the future, as the technology matures?

N = 250

Figure 6 Predictions for which method will dominate in future

34%!

24%!

38%!

50%!

36%!

25%!

43%!

37%!

42%!

35%!

33%!

31%!

46%!

45%!

10%!

10%!

11%!

8%!

8%!

19%!

2%!

18%!

25%!

16%!

10%!

25%!

10%!

10%!

Total!

North America!

Europe!

Pac Rim!

Small!

Medium!

Large!

App! Browser! Neither will dominate! Not sure!


5 Panel rewards and quality control

Questions

1. To what extent do surveys across the industry come from incentivised panels?

2. What incentive methods are used, and with what frequency?

3. What quality control methods are applied?

4. How much do panel owners use independent panel verification services to control for professional respondents with multiple panel memberships?

Key findings

Penny for your thoughts – most online surveys today are incentivised

§ Nearly six out of 10 (57%) of research companies are using incentivised panels for between two-thirds and 100% of their samples. Only a few (7%) are not using rewards at all.

Independent panel verification is the exception not the norm

§ Around three-quarters (76%) of panel operators do not subscribe to independent panel verification services. Even among large companies 58% do not do this.

Most MR companies run simple fraud prevention checks on online responses

§ Most companies are checking for speeding by respondents (73%) and nearly two-thirds (63%) look for ‘straightlining’: two quality control methods that many data collection tools make easy to apply.

More thorough respondent fraud checks are largely shunned

§ Just over a half surveyed (52%) use challenge questions, and fewer still some of the more high tech methods.

Consumerist rewards are the norm

§ Incentives to participate in surveys appeal to a consumerist appetite, with vouchers being the most frequently used reward (75%).

§ Donation to good causes, gifts, cash payments, entries to sweepstakes or points that can be exchanged for prizes are all rewards that research companies very commonly use.

§ The communitarian approach of rewarding participation by enhanced status, or visibly though special recognition and kudos awards is currently used by only 16% of companies.


5.1 Prevalence of incentivised panel surveys

à Around two-thirds of all survey responses now come from incentivised panels

à There is little difference in behaviour by company size

à N America makes very slightly more use of incentives than other regions

The differences observed between company tiers and global regions are very slight – so a highly consistent picture emerges that virtually two out of every three surveys fielded today are drawn from an incentivised panel.

Q: How much of your quantitative survey work uses panels or other sample sources where participants receive a reward? [0-100%]

N = 250

Figure 7 Extent to which panel participants are incentivised

63%!

66%!

61%!

63%!

64%!

61%!

64%!

Total!

N America!

Europe!

Asia Pacific!

Small!

Medium!

Large!


5.2 Distribution of incentivised panel use

à Practices vary widely between different firms

à The largest sector uses incentivised panels for most of its work (and many of these for all work)

à Another large group use panels sparingly or not at all

à North America is the most dependent on panels

à Asia Pacific is the most diverse in its practices

Q: How much of your quantitative survey work uses panels or other sample sources where participants receive a reward? [0-100%]

N = 250

Figure 8 Distribution of the prevalence of incentivised panel as a sample source

In the graph above, the percentages given have been grouped into terciles, with the exception of those stating 0% or 100%, which are grouped separately. If panel usage were randomly (and therefore evenly) distributed, the green, orange and blue sectors would align with the 33% and 67% vertical grid lines. Instead, this shows that behaviour tends to be clustered at either end of the spectrum, with the middle tercile relatively sparsely populated.

7%!

5%!

8%!

8%!

8%!

5%!

5%!

20 %!

17 %!

20 %!

25 %!

19 %!

24 %!

17 %!

16 %!

15 %!

21 %!

8 %!

14 %!

18 %!

24 %!

32 %!

40 %!

28 %!

23 %!

31 %!

37 %!

26 %!

26 %!

22 %!

24 %!

38 %!

28 %!

18 %!

29 %!

0%! 33%! 67%! 100%!

Total!

N America!

Europe!

Asia Pacific!

Small!

Medium!

Large!

None! Up to 1/3! Between 1/3 and 2/3! Over 2/3! All!


5.3 Incentives used

à Vouchers dominate but points and cash are used the most frequently

à Cash is more common with large firms and in N America

à Donations and lotteries widely used – but not as a major incentive method

à Awarding ‘kudos’ points is still a rarity

Research companies appear to be using the full spectrum of incentive methods available, with the sole exception of awarding enhanced status or privileges – which is a method that is popular on social media sites and which, though especially relevant to communities, is also applicable to panels.

Q. Which of these incentives do you use, and to what extent?

N = 250

Figure 9 Incentives used – frequently or occasionally

There are interesting differences between the methods that are widely used (regardless of how frequently that use is) and the underlying core group for each that use them frequently. Vouchers are widely used, but among those users, just over a quarter them frequently; even more so with donations, which 69% of firms use, but less than a fifth use them frequently. The reverse applies to redeemable points, which overall 54% use, but a majority of these use them frequently.

27 %!

16 %!

12 %!

28 %!

21 %!

32 %!

2 %!

48 %!

53 %!

51 %!

32 %!

38 %!

22 %!

14 %!

Vouchers!

Donation to a good cause!

Non-monetary gifts!

Cash!

Sweepstakes/lotteries!

Points for redemption!

Enhanced member status or privileges !

Frequently! Occasionally!


Total

N America

Europe

Asia Pacific

Small

Med

Large

Vouchers 27% 29% 25% 28% 24% 32% 29%

Donation to a good cause 16% 13% 18% 15% 13% 21% 17%

Non-monetary gifts 12% 9% 12% 23% 9% 13% 24%

Cash 28% 36% 22% 28% 30% 21% 33%

Sweepstakes/lotteries 21% 25% 19% 18% 21% 18% 24%

Points for redemption 32% 28% 35% 38% 29% 32% 45%

Enhanced status or privileges 2% 0% 3% 5% 1% 2% 7%

Table 4 Incentives used frequently – by global region and company size

N=250

Total

N America

Europe

Asia Pacific

Small

Med

Large

Vouchers 75% 81% 69% 80% 72% 81% 76%

Donation to a good cause 69% 75% 66% 63% 63% 78% 76%

Non-monetary gifts 64% 59% 64% 73% 58% 70% 74%

Cash 60% 75% 49% 58% 56% 60% 76%

Sweepstakes/lotteries 59% 69% 53% 50% 57% 60% 62%

Points for redemption 54% 45% 62% 55% 52% 54% 62%

Enhanced status or privileges 16% 14% 13% 28% 10% 21% 29%

Table 5 Incentives used frequently or occasionally – by global region and company size

Some methods also appear to be more popular with different company sizes and global regions.

Perhaps living up to a cultural stereotype, North America is into cash as a reward method, both as a frequent and occasional method, much more than other parts of the world. It is cited less frequently in Europe, where points are more popular (points are less popular in North America).

Companies in North America also seem to use slightly more incentive methods then the other regions. On average, they mentioned 4.2 methods, against 3.8 in Europe and 4.1 in Asia Pacific. Large companies on average mentioned 4.6 methods, against 3.7 mentioned by smaller firms.

Large firms also seem to be ahead in offering enhanced status or other ‘kudos’ rewards – which may relate to their greater engagement with research communities, where this model is particularly relevant.


5.4 Survey quality measures applied

à Simple, immediate measures like speed checks and straight-lining detection are the most popular

à More effective but more costly measures like digital fingerprinting or ID verification are less common

à Most firms seem to be using a combination of methods

Research professional bodies such as ESOMAR and CASRO have recently highlighted many quality issues arising from panels and the effect of incentives on survey quality leading to interview fraud on a grand scale. Having established the degree of incentive usage, we also wished to examine the extent to which research companies were using technology to detect fraud – using the kinds of quality control measures that technology can facilitate relatively easily.

Q. Which of these quality control methods do you apply to prevent inappropriate or fraudulent participation from participants taking incentivized surveys in your panel research?

N = 250

Figure 10 Chart title

73 %!

63 %!

52 %!

49 %!

29 %!

46 %!

43 %!

9 %!

Speed detection!

Straight-lining detection!

Challenge questions!

IP address location checking!

Digital fingerprinting to identify multiple registrations!

Analytics on aggregated responses!

ID verification at the time of registration!

No method used!


5.5 Survey quality measures at a detailed level

à Larger companies apply more QC measures than smaller ones

à Europe behind N America and Asia Pac on ‘stronger’ fingerprinting, IP checking and ID verify measures

There are several differences in detail, at a company size and global regional level, in the both the actual quality control (QC) measures applied and the number of measures applied.

We calculated the number of measures by counting the number of mentions, after excluding “no method used”.

Large firms are taking the lead with the number of measures they apply (on average more than four were cited), and also the extent to which they use IP address checking, digital fingerprinting and ID verification, all of which are technically advanced methods. These may require additional software to achieve, and usually involve some additional effort and know-how to achieve.

There also appears to be slightly more attention given to quality control measures in North America (average of four methods cited) against Europe (3.2) and Asia Pacific (3.4).

Europe lies behind both North America and Asia Pacific on the three more high-tech and therefore ‘stronger’ fraud checking measures of IP address location checking, digital fingerprinting (which identifies multiple registrations) and ID verification.

The 5% of large and medium-sized companies that report no checks possibly indicates that there is some under-reporting in this question, as it is unlikely that the organisations involved in fact apply no such checks – it serves to remind us that a limitation of this survey of companies is that it depends on the extent of knowledge of the individual completing it about activities which other teams may be performing.

N = 250

Total

North Am.

Eur- ope

Asia Pac Small Med Large

Speed detection 73% 76% 71% 70% 67% 83% 79%

Straight-lining detection 63% 77% 57% 48% 61% 65% 67%

Challenge questions 52% 58% 48% 45% 46% 60% 57%

IP address location checking 49% 55% 40% 58% 43% 51% 64%

Digital fingerprinting to identify multiple registrations

29% 35% 22% 33% 24% 32% 41%

Analytics on aggregated responses 46% 49% 45% 43% 43% 43% 60%

ID verification at the time of registration 43% 49% 38% 43% 38% 51% 50%

No method used 9% 6% 11% 10% 12% 5% 5%

Average number of measures (excl. none)

3.54 3.98 3.22 3.38 3.23 3.84 4.17

Table 6 Quality control measures by global region and company size


5.6 Use of independent panel verification services

à Overall, only a quarter of research companies with their own panels use independent verification services

à Large firms are more than twice as likely to use them than small firms

à Only one in five firms in Europe use IPV services

While the quality control measures discussed previously can help to detect fraudulent behaviour in the moment (either at the moment of the survey, or at the time of joining the panel), they do not help to combat the professional respondent problem, identified in studies by ESOMAR and CASRO, amongst others. For this, individuals need to be checked for membership of other panels – using measures such as digital fingerprinting, to recognise multiple registrants. Use of these services could eliminate this problem to a large extent, although there is an additional cost associated, and some additional effort involved.

This survey identifies the same trend observed with other fraud reduction measures – that the more powerful measures, which are also more costly to apply, are used less frequently. In this case, they are used much less frequently.

Q. Do you subscribe to any independent panel verification services?

N = 110 Base: those who operate their own panels

Figure 11 Adoption of independent panel verification services (among panel owners)

24%!

29%!

19%!

28%!

17%!

21%!

42%!

Total!

North America!

Europe!

Asia Pac!

Small!

Medium!

Large!


6 Communities Questions

§ How many communities are research firms operating today (versus 3 years ago)?

§ Is more growth in research community work anticipated?

§ What kind of software do community operators use to run them?

§ What particular software features are sought in a community platform?

§ How good is existing software support for these features?

Key findings

Communities are still rare

§ As in 2009, only one in six companies (17%) are operating a community. Even among large companies, only 31% run communities.

Companies that run communities don’t run many of them

§ Of the 41 companies in our sample who are running communities, over half run 4 or fewer communities: figures broadly similar to those we obtained in 2009.

Still optimistic about growth, despite lack of growth

§ Those running communities, and especially large companies, mostly predict that they will increase their number of communities over the next year.

Discussion boards and polls are most important requirements for community software

§ 79% said they need discussion forums and 75% need polls within their community platform. Over 60% also said they required diaries, blogs and real-time groups, making the requirements for community platforms quite diverse.

Existing software supports most important features

§ The simpler and more established capabilities like polls, diaries and forums seem well supported, but poor support was identified for co-creation tools, text analytics and mobile community apps

§ Four out of ten community operators wants to have a mobile app for their community members to use

There is a place for specialist community software

§ Around a quarter of firms use the same software for panel and community administration, but the majority either use specialist panel tools, or blend these with their panel platform.


6.1 Who runs communities?

à Communities are a specialised activity among a small minority of firms

à Communities much more likely to be found in N America and among large companies

à Uptake is lower in Asia Pacific and among small firms

The number of firms operating communities is probably fewer than is expected, and remains very much the preserve of specialist operators rather than a mainstream activity offered by a majority of research firms. In fact, the substantial majority do not offer communities, nor do they show any intention of doing so.

Q: Does your company operate any online research communities?

N=250

Figure 12 Communities operated in 2012 by region and company size

16%!

21%!

15%!

8%!

9%!

24%!

31%!

18%!

10%!

22%!

23%!

21%!

11%!

17%!

66%!

68%!

63%!

70%!

70%!

65%!

52%!

Total!

North America!

Europe!

Asia Pac!

Small!

Medium!

Large!

Operate at least one community! Developing or plan to introduce! No current plans!


6.2 Change in uptake since 2009

à No overall growth detected in communities operated

à Fewer companies now planning to introduce communities than three years ago

This same question was asked in 2009 and in 2012, which allows direct comparison of the responses. Although the sample size was smaller in 2009, the sampling method and sample composition is broadly similar.

The proportion of firms that offer communities is virtually the same then and now. Those with no plans to introduce communities have hardened since that time, from just over half of firms to two-thirds now.

Q: Does your company operate any online research communities?

N=188 (2009) and 250 (2012)

17%!

16%!

27%!

18%!

56%!

66%!

2009!

2012!

Operate at least one community!Developing or plan to introduce!No current plans!


6.3 Number of communities operated

à The majority of firms run 4 communities or fewer

à Since 2009, slow growth in the number of communities firms are running

à A very small number of niche players have achieved scale with 20+ communities

Q: How many communities do you currently run?

Figure 13 Number of communities operated in 2009 and 2012

63%!

13%!

16%!

9%!

0%!

56%!

17%!

12%!

10%!

5%!

1-4!

5-9!

10-19!

20-49!

50+!

2009 N=32 !

2012 N=41!


6.4 Predicted growth for communities

à Weak growth anticipated overall

à A minority anticipate strong growth

à Growth predictions now more cautious than in 2009

This question was also asked in 2009 as well as 2012, which allows direct comparison. As this question was asked only of those operating or planning to operate communities, the sample size is too small to analyse it by region or company size.

Compared to 2009, the sentiment in 2012 is more cautious, with fewer predicting strong growth and more considering modest or no growth.

Q: Do you expect the number of communities you run to increase over the next 12 months?

Base: Those currently or planning to operate communities

Figure 14 Predicted growth for communities

34%!

50%!

7%!

2%!

6%!

20%!

60%!

11%!

4%!

6%!

Increase strongly!

Increase slightly!

Remain the same!

Decrease!

Cannot say!2009 N=82!

2012 N=85!


6.5 Software used to administer communities

à A quarter of community operators use panel management software to run their communities

à Most use specialist community software

à Some combine specialist software with panel management software

From a technical perspective, communities and panels share many characteristics that make their requirements distinctive from that of conventional survey data collection. These include having a database of members or participants, repeated contacts to the same individual for different purposes, and the accumulation of response history over time. For that reason, some panel software offers community capabilities as an extension to panel management capabilities. This question aimed to discover whether this hybrid approach is popular among community operators, or if they prefer to use tools dedicated to managing communities.

It appears that only a minority use the same tools as for panel management exclusively; some supplement their panel management software with specialist community tools, while the majority only use specialist software for communities.

Q: Do you use the same software or different software for communities as for managing panels?

N = 85

Figure 15 Software used to administer communities

Use the same

software!27%!

Use different software!

55%!

Use a mixture!

18%!


6.6 Required capabilities in community software tools

à Simple ‘panel plus’ tools are those most sought

à Around 60% want diaries, real-time groups and blogs

à 45% want a mobile app for community members to use

There are many different forms of engagement possible with a community, from the addition of simple discussion forums and polls to a panel (which could be considered ‘panel plus’ rather than a full community) to more intensive and unconventional research activities. This survey set out seven such methods that are frequently cited. Of these, only co-creation tools were requested by less than half of those responding (by 41%). Communities tend to generate a lot of text, which can be painstaking to analyse. Despite this, only 35% saw this as a requirement. However, almost half (45%) want their community tool to offer a mobile app so that community members can participate from smartphones, tablets and the like.

Q: Which of these specific capabilities do you require in your research community software platform?

N = 85

Figure 16 Required capabilities in community software tools

79%!

75%!

62%!

61%!

61%!

45%!

41%!

35%!

8%!

Discussion boards or forums!

Polls!

Diaries!

Real-time groups!

Blogs!

Mobile app (for participants)!

Co-creation tools!

Text analytics!

Other capabilities!


6.7 Satisfaction with community software capabilities

à Blogs, discussion boards and diaries appear to be well-supported

à Support for text analytics and co-creation tools considered least satisfactory

à Mobile app support considered poor by nearly a fifth

As these questions relate to features required by users – and not all companies require these features – the base sizes are inevitably quite small for these responses, and are clearly shown in the axis of the chart.

A mixed picture emerges with some features well supported, and others less so. Blogs, discussion boards (or forums) and diaries all fare reasonably well – almost all rated the support good or fair. The more technically demanding requirements of real-time focus groups and mobile apps have a significant minority finding them poor, however. As do the more novel and unconventional options for text analysis and the more nebulous co-creation tools. These also give greater cause for dissatisfaction, with only 30% and 23% respectively rating them ‘good’.

Q: How would you describe the support for these capabilities in the software tools that you currently use?

Figure 17 Satisfaction with community software capabilities: ranked from good to poor

52%!

52%!

43%!

42%!

39%!

30%!

23%!

42%!

40%!

47%!

40%!

46%!

53%!

51%!

6%!

8%!

9%!

18%!

15%!

17%!

26%!

Blogs (N=52)!

Discussion boards (N=67)!

Diaries (N=53)!

Mobile app (N=38)!

Real-time groups (N=52)!

Text analytics (N=30)!

Co-creation tools (N=35)!

Good! Fair! Poor!


Part 2: Trends

There is a central core of questions in this survey that we have been asking for several years – and some since the inception of this project in 2004. With several years of data, several clear and stable trends have emerged. This shows continued consistency in the survey as a whole, since the sample this year is made up largely of different companies and individuals than in the previous year.

Topics covered in this section

§ Research activities § Sources of online sample § Mixed-mode § Analysis and results reporting § Changing software


7 Research activities

Questions

§ Volume of work that derives from quantitative, qualitative or other research activities?

§ Volume of work by research mode (CATI, CAWI etc)?

§ Anticipated changes in the amount of work?

Key findings

Mobile research growing rapidly

§ The number of companies offering mobile research (self completion, but not including SMS) is increasing rapidly. When we first measured this in 2009, 6% of companies offered this as a service. In 2012 it is 18%.

Most other modes are fairly stable, except for paper and CAPI

§ Paper appears to be declining and CAPI increasing. Volumes of mobile research may be increasing too – in previous years, the volume was 1% or less, in 2012 it is 2%.

Tablets may be fuelling CAPI growth

§ The number of companies offering mobile CAPI appears to be stable, but there is growth in the number of companies offering laptop/tablet CAPI in 2011 and 2012.

Significant minority of Web surveys completed on smartphones

§ Companies now report that 13% of those taking part in their online surveys are using their smartphones.

Volumes of paper and CATI research dwindling, Web strong

§ Web research represents just over half (51%) of all quantitative research and this proportion may now be stabilising.

Research companies expecting Web and mobile research to be main growth areas

§ The participants in this survey say that mobile and Web research are the area they expect to grow the most (the latter seemingly contradicting the finding above)


7.1 Research mix – 6-year trend

à Almost a quarter of revenues derive from qualitative

à Qualitative vs. quantitative split is very stable over the years

à Research firms do very little non-research work

Over the years, the proportion of quantitative research revenues is to qualitative is remarkably stable. Quantitative represents around 70% and qualitative between 20% and 24%. There is a discontinuity in the chart, at 2009, because “non-research activities” was added as a new answer category.

Q: Please indicate the approximate proportion of your company revenues that derive from quantitative, qualitative or other research activities.

N = 250

Figure 18 Proportion of company revenues that derive from research activities, annual trend

0%!

10%!

20%!

30%!

40%!

50%!

60%!

70%!

80%!

2007! 2008! 2009! 2010! 2011! 2012!

Quantitative research!

Qualitative research!

Other!

Other research!

Non-research activities!


7.2 Company revenues from research in 2012

à Smaller companies do more qualitative work, compared to larger companies

à The mix of work varies little between global regions

These results are much the same for all regions. However large companies do more quantitative and less qualitative work than small and medium-sized companies.

Q: Please indicate the approximate proportion of your company revenues that derive from quantitative, qualitative or other research activities.

N = 250

Figure 19 Proportion of company revenues that derive from research activities, by region and company size

69%! 69%! 71%! 66%! 66%! 68%!79%!

23%! 24%! 22%!25%! 26%! 24%!

14%!

4%! 4%! 5%!4%! 4%! 5%! 5%!3%! 4%! 2%! 6%! 4%! 4%! 2%!

Total! Americas! Europe! Asia Pacific! Small! Medium! Large!

Quantitative research! Qualitative research! Other research! Non-research activities!


7.3 Volume of work

à Web + CATI + paper = 92% of work

à Web was growing until 2010 but may now have reached a plateau

à Steep decline in paper, gradual decline in CATI

The volume of Web research has risen by 11 percentage points between 2006 and 2012. In the same time, paper research has declined by exactly the same amount.

The volume of CATI work has also declined a little over the years, from 27% in 2006 to 21% in 2012. Despite the large increase in the number of companies offering laptop/tablet CAPI, it remains at 6% of total quantitative research volume in 2012. In 2006, it was 5%.

For details on the more minor modes, see the table overleaf.

Q: Focusing on your quantitative research activities, please indicate the approximate proportion of your work represented by each of these interviewing modes or combinations. N = 250

Figure 20 Volume of work by mode

0%!

10%!

20%!

30%!

40%!

50%!

60%!

2006! 2007! 2008! 2009! 2010! 2011! 2012!

Web!

CATI!

Paper!

laptop/tablet CAPI!

Mixed mode!

mCAPI!


2006 2007 2008 2009 2010 2011 2012

N 213 233 215 188 213 230 250

Web 40% 43% 48% 46% 47% 51% 51%

CATI 27% 25% 26% 23% 27% 23% 21%

Paper 21% 19% 14% 16% 13% 14% 10%

CAPI total 5% 7% 5% 8% 5% 6% 9%

Laptop or tablet CAPI 5% 5% 4% 5% 3% 5% 6%

mCAPI 0% 2% 1% 3% 2% 1% 3%

Mixed mode total 7% 6% 6% 6% 6% 6% 6%

Mixed mode CATI and web 4% 4% 4% 3% 4% 5% 4%

Any other mixed mode 2% 2% 2% 3% 2% 1% 2%

Self-completion on mobile devices (not SMS)

0% 0% 1% 1% 0% 2%

IVR 0% 1% 1% 1% 1% 1%

SMS text messaging (self completion) 1% 0% 0% 0% 0% 0% 0%

Table 7 Proportion of quantitative research by mode


7.4 Research modes offered – mainstream modes

à More companies offering CAPI research now

à Fewer companies doing paper-based research

à Online research practised by most – but not all firms

CAPI seems to be making inroads at a time that paper-based research seems to be on its way out – paper is the only mode that is clearly being offered by fewer companies over time. Web research is part of the portfolio for nearly all companies (92% in 2012) and looks set to remain dominant.

The results for this are derived from the question in the previous section (‘please indicate the approximate volume of your work represented by each of these interviewing modes’). It is calculated by counting the number of people who give a non-zero response.

Q: All mentions of each research mode N = 250

Figure 21 Percentage of market research firms using the main modes of research

0%!

10%!

20%!

30%!

40%!

50%!

60%!

70%!

80%!

90%!

100%!

2006! 2007! 2008! 2009! 2010! 2011! 2012!

Web!

CATI!

Paper!

Mixed mode!

CAPI!


7.5 Research modes offered – minority modes

à Mobile self-complete becoming mainstream

à Laptop/tablet CAPI growing very rapidly, but mobile CAPI stable

à SMS not gained foothold in marketplace; IVR is dwindling

Here, we look at the percentage of companies offering the more minor modes of research, for some of which we only started collecting data in 2009. Mobile self-complete is rapidly becoming a common offering among research companies – just 6% offered this mode in 2009 compared with 18% in 2012. Although, as reported earlier in this chapter, volumes of research conducted in this mode are still very low. SMS research is down to 2% from its 2009 ‘peak’ of 8%.

We measure laptop and tablet CAPI separately from mobile CAPI (mCAPI), and can see that the growth in CAPI (reported in the previous section) is entirely from laptop and tablet CAPI, although we assume that it is actually tablets, and not laptops, that are fuelling the growth.

Q: All mentions of each research mode N = 250

Figure 22 Percentage of market research firms using the minor modes of research

0%!

5%!

10%!

15%!

20%!

25%!

30%!

35%!

40%!

45%!

50%!

2006! 2007! 2008! 2009! 2010! 2011! 2012!

Mixed mode CATI & Web!

laptop/tablet CAPI!

mCAPI!

Mixed mode other!

Self complete - mobile!

IVR!

SMS!


7.6 Changes in the amount of work

à Participants expect main areas for growth to be mobile and web

à Paper, CATI, IVR expected to decline

The fact that the participants in this survey expect mobile self-complete to grow fairly modestly is to be expected, although it seems quite pessimistic when considering the growth observed between the 2011 and 2012 studies. Also, there is a contradiction between expected growth in web, and the growth experienced in reported volumes of work over the last few years (as presented earlier, on p. 38) which has increased sharply.

Q What changes do you foresee in the amount of work you will handle over the next three years in these research activities?

N = 250

+2= major growth; +1= modest growth; 0= no change; -1 = decline

Figure 23 Anticipated changes in the amount of work over next three years

-1!

-0.5!

0!

0.5!

1!

1.5!

2!

2006! 2007! 2008! 2009! 2010! 2011! 2012!

Web! Self complete - mobile! mobile CAPI! Mixed mode CATI & Web!

laptop or tablet CAPI! SMS! Other mixed mode! IVR!

CATI! Paper!


7.7 Reported smartphone usage in web surveys

à Reported participation in normal online surveys via mobile devices has nearly doubled since 2011

à Significant proportion of online surveys are now completed on smartphones

à A large proportion of participants don’t know the figure

The purpose of this question is to capture the extent to which firms are measuring and experiencing ‘unintentional’ mobile participation in their online surveys (i.e. the survey has not been designed around a mobile methodology as such). In 2011, the reported total was 6.7%. By 2012, this estimate had risen to 13.1%. However, of those eligible to answer this question (because they do web surveys), 30% said they did not know what the estimate was. Hence, the estimate is based on 163 responses.

Given the surge in the use of smartphones around the globe, it is not surprising that the figure has increased, but what is more worrying is the 30% of those interviewed who were unable to give an estimate.

Q: Thinking just about your web surveys, what proportion of participants are taking these surveys on small format mobile devices such as smartphones?

2012: N=233

Those providing an estimate versus those who don’t know

Figure 24 Percentage of web survey participants taking surveys on smartphones and other small format mobile devices: 2011 and 2012

13.1%!

6.7%!

2012!

2011!

Answer given 70%

Don't know 30%

14.7%!

11.3%!

13.5%!

11.1%!

14.9%!

16.1%!

Americas!

Europe!

Asia Pacific!

Small!

Medium!

Large!

2012!N=163!


8 Sources of online sample

Question

§ Proportion of online samples from each identified source

Access panels favoured over in-house panels

§ Over the years, there has been a trend towards the use of access panels and away from in-house panels.

§ The proportion of client sample in use has remained steady.

§ Recently, researchers seem to be turning to less conventional sources for their sample


8.1 Proportion of online sample in use - trend

à Researchers becoming more reliant on access panels

à Own panels supplying lower proportion of sample

These figures provide evidence for our assumption that researchers are finding it increasingly difficult to source sample for online surveys, either in the quantity or quality they require. The volume of client sample has never declined, despite predictions to the contrary in previous years in this study. The amount of sample from access panels has slowly risen, whereas the volume from companies’ own panels has gradually declined.

Q: What proportion of your online samples come from each of these sources?

N = 250

Figure 25 Proportion of online sample in use

0%!

5%!

10%!

15%!

20%!

25%!

30%!

35%!

40%!

2006! 2008! 2009! 2010! 2011! 2012!

Third party/access panels!

Sample provided by client!

Own panels!

Specialist sample providers!

Others!


8.2 Proportion of online sample in use in 2012

à Large companies best at supplying own sample

à Even large companies source nearly three-fifths (59%) of sample externally

à North America: less use of own panels

The results for large companies are markedly different to those of small and medium-sized companies. Although, it is probably to be expected that due to their greater resources, large companies manage to source more sample from their own panels and are therefore less reliant on client data, access panels and sample providers.

Looking at the global regions, North American companies make greater use of access panels, and source less of their sample from own panels.

Q: What proportion of your online samples come from each of these sources?

N = 250

Figure 26 Proportion of online sample in use, by region and company size

35%!

39%!

30%!

37%!

35%!

39%!

24%!

27%!

27%!

28%!

21%!

29%!

25%!

21%!

24%!

19%!

27%!

27%!

20%!

23%!

37%!

11%!

12%!

10%!

13%!

12%!

9%!

14%!

4%!

3%!

5%!

3%!

3%!

4%!

4%!

Total!

North America!

Europe!

Asia Pac!

Small!

Medium!

Large!

Third party/access panels! Sample provided by client! Own panels!

Specialist sample providers! Others!


8.3 Online sample in use - trend

à Substantial growth in use of access panels since 2004

à Slightly more companies using client-provided sample

Most (85%) research companies use sample provided by their clients, perhaps indicating a growth in customer satisfaction work, over market-wide or gen. pop. studies. Just over three-quarters (77%) use access panels and this has risen from just over half (54%) in 2004. There has been no change in the proportion of companies using specialist sample providers and own panels.

An intriguing observation this year is the rise of ‘other’ as a sample in both 2011 and 2012. While it does not account for a large proportion, ‘other’ is now cited by nearly a quarter (24%) of companies.

We felt we had to investigate further, so we make follow-up contact with some in this group to ask what they meant by ‘other’ sources. From their varied responses, it appears there is no single missing source from the list. Rather, it seems that researchers are becoming more resourceful in finding ways of obtaining sample. Examples included directories from industry associations and magazines, using click-through ads or links in social media, online magazines or websites related to the topic of the survey.

Q: Which sources of online sample do you use?

N = 250

Figure 27 Online sample sources in use

0%!

10%!

20%!

30%!

40%!

50%!

60%!

70%!

80%!

90%!

100%!

2004! 2005! 2006! 2007! 2008! 2009! 2010! 2011! 2012!

Sample provided by client!

Third party/access panels!

Specialist sample providers!

Own panels!

Others!


9 Mixed mode

Questions

§ The importance of mixed mode support in data collection software?

§ Whether the company uses integrated or disparate software for mixed-mode work?

§ The nature of the mixed mode capabilities required?

Most companies think mixed mode an important capability

§ Nearly 90% of our participants think that the capability to conduct mixed mode research is an important feature when looking for new software – this has changed little since 2008.

Integrated platforms increasingly the norm

§ There is a clear upward trend over recent years towards companies using integrated mixed mode platforms for their data collection.

Sticking to simpler forms of mixed mode requirements

§ Only a quarter (24%) require mixed mode with switching (the most advanced type of multimode research), and this has remained constant since 2005.


9.1 Importance of mixed mode

à Nearly 90% of respondents think mixed mode is important

à Around one third think it is essential

This has been the case since 2008 and there is no sign that there is any weakening in this response.

Q: If you were choosing new software, or reviewing your current solution, how much importance would you place on the tool’s ability to mix and combine different data collection modes?

N = 250

Figure 28 Importance of mixed mode

27%!17%!

35%!30%! 28%!

31%!43%!

31%!

30%! 33%!

29%! 25%!

26%!28%! 25%!

9%! 14%!5%! 9%! 11%!

4%! 2%! 3%! 2%! 2%!

2008! 2009! 2010! 2011! 2012!

Don't know!

Unimportant!

Moderately important!Very important!

Essential!


9.2 Use of integrated platforms

à A clear trend towards integrated platforms

This chart clearly shows that in the last seven years, there has been a clear and gradual shift away from switching between platforms and towards integrated software platforms. In 2012 nearly three-fifths use integrated platforms.

Q: Do you use an integrated software platform for your multimode interviewing, or do you need to switch between different software platforms to combine modes?

N = 250

Figure 29 Use of integrated platform versus switching between different platforms

38%!44%! 50%!

59%! 53%!60%! 57%!

63%!56%! 50%!

41%! 47%!40%! 43%!

2006! 2007! 2008! 2009! 2010! 2011! 2012!

Integrated platform! Switch between different platforms!


9.3 Level of mixed mode support required

à Having a common authoring platform is the most popular mixed-mode need

à A very consistent 21-25% of companies need advanced support, where interviews switch between modes

It is clear that there is no increase in demand for multi-mode with switching. The requirement for this has remained at around a quarter of companies since we first asked this question in 2005. What is less clear is what is happening with common authoring and mixed modes in parallel. It looks as if the former could be becoming more important the later declining, but it is too early to be sure.

Q: What level of mixed mode or multi-mode capabilities do you require?

N = 250

Figure 30 Level of mixed mode

35%! 38%!27%! 27%! 28%!

41%! 42%!

40%! 40%!49%! 51%!

39%!

36%! 34%!

25%! 23%! 24%! 21%!34%!

23%! 24%!

2005! 2006! 2007! 2008! 2010! 2011! 2012!

Common authoring! Mixed modes in parallel! Multi-mode with switching!


10 Analysis and results reporting

Questions

§ Percentage of projects involving each deliverable or distribution method?

§ Anticipated change in demand for research delivery methods?

§ Importance of ability to produce volumes of cross-tabular reports in future?

Large companies using more high tech reporting

§ It is noticeable that in 2012 large companies are making significantly greater use than small and medium companies of high tech results distribution methods, such as dashboards. This was not the case in 2011.

Large companies more optimistic about future of high tech reporting

§ Overall, in both 2011 and 2012 our participants anticipate a modest increase in demand for all newer technology-based methods for delivering results, although in both years large companies are particularly optimistic.

Bad news for trees?

§ Over 80% of the participants in this survey said that the ability to print out large volumes of tables is ‘essential’ or ‘moderately important’. This has changed little since 2004.

§ However, in 2012 very few projects are actually delivered on printed tabs


10.1 Distribution methods in use

à Microsoft PowerPoint is leading delivery method – by far

à Within seven years printed tabs have gone from mainstream to exceptional

à No clear rise in the use of the most high tech methods

Against our expectations, the more high tech results distribution methods (digital dashboards, interactive analysis and online static reports) do not seem to be growing in use. There is a slight shifting of the sands between the various Microsoft products, with Word declining in importance and PowerPoint and Excel increasing in use. It does seem logical that Excel is becoming more frequently used since researchers can include raw data and charts in Excel and allow further manipulation of data, whereas this is not possible in Word.

Printed tables have quickly drifted from a mainstream client deliverable to a specialist one. In 2006, they were used for 23% of projects and in 2012, this had reduced to 6%. However, their use internally, we believe, is undiminished.

Q: What percentage of projects currently involves the following deliverables or distribution methods to the client?

N = 250

Figure 31 Distribution methods in use

0%!

10%!

20%!

30%!

40%!

50%!

60%!

2006! 2007! 2008! 2009! 2010! 2011! 2012!

MS PowerPoint!

MS Excel!

Acrobat PDF!

Online static reports!

MS Word!

Interactive analysis!

Printed tabs!

Digital dashboards !


10.2 Distribution methods in use, by company size

à New in 2012: use of high tech reporting increases by company size

à First sign that high tech reporting is about to increase?

The ‘low tech’ methods, such as Microsoft PowerPoint, Excel, Word or Acrobat PDF, are used in similar proportions by all size of company. However, in 2012 it is noticeable that large companies are making greater use of the more high tech methods of results delivery. This was not the case in 2011.

In fact, large companies are using interactive analysis and dashboards more than Microsoft Word. Dashboards are very rarely used in small companies (3% of projects) but are used by over one in eight (13%) of projects in large companies. There is also a difference in the use of interactive analysis – 8% at small companies compared with 21% at large companies. These are more resource intensive methods that demand both the technical infrastructure and in-house skills, which larger companies tend to have.

Q: What percentage of projects currently involves the following deliverables or distribution methods to the client?

N = 250

Figure 32 Distribution methods in use, by company size

53%! 51%!58%! 56%!

29%! 29%! 29%! 29%!

21%! 24%!

15%!

22%!19%! 20%!

27%!

14%! 14%!14%!

12%!10%! 8%! 7%!

21%!

4%!8%! 8%!6%! 3%! 7%!

13%!

2012! Small! Medium! Large!

MS PowerPoint! Microsoft Excel documents! Acrobat PDF! Online static reports!

MS Word! Interactive analysis! Printed tabs! Digital dashboards !


10.3 Change in demand for high tech delivery

à Modest increase in demand for all high tech delivery methods is anticipated

à Dashboard delivery remains a hot future topic

Companies of all sizes anticipate a modest increase in demand for all high tech delivery methods. Although not shown here, for every method, larger companies are slightly more optimistic. The same was true in 2011.

Q: What kind of change in demand do you anticipate for the following research delivery methods over the coming year?

N = 250

Figure 33 Anticipated change in demand for high tech delivery methods

22%!

14%!

19%!

17%!

23%!

51%!

45%!

46%!

43%!

47%!

27%!

40%!

34%!

39%!

30%!

1%!

2%!

0%!

0%! 20%! 40%! 60%! 80%! 100%!

Fixed reports online!

Clients create analyses online!

Info portals!

External portals (eg Sharepoint)!

Digital dashboards!

Major increase! Modest increase! No increase! Decrease!


10.4 Importance of producing cross tabs in volume

à Nearly everyone needs to produce cross tabs in volume

Even though very few projects are now delivered on printed tables, nearly all our survey participants (87%) say that is essential or moderately important to be able to produce cross tabs in volume. This can be explained by a continuing need to use tables as a primary internal source for analysis, and also as a means to cross-check other deliverables, as a quality control method. The need for cross-tab packages is not about to disappear.

Q: When considering analysis and reporting tools for the future, how important is it that these should be able to produce volumes of cross-tabular reports?

N = 250

Figure 34 Importance of being able to produce volumes of printed cross tabs

59%!55%! 55%! 52%!

57%!50%! 53%! 52%! 53%!

27%!27%!

33%! 36%! 28%!34%! 30%! 33%! 34%!

13%!14%!

10%! 11%! 13%! 13%! 16%! 13%! 10%!

1%! 4%! 2%! 1%! 2%! 3%! 1%! 2%! 2%!

0%!

10%!

20%!

30%!

40%!

50%!

60%!

70%!

80%!

90%!

100%!

2004$ 2005$ 2006$ 2007$ 2008$ 2009$ 2010$ 2011$ 2012$

Essential! Moderately important! Relatively unimportant! Not something future tools should provide!


11 Changing software

Questions

§ Propensity to replace the software currently in use, in the near future?

§ Type of software being considered for replacement?

§ Reasons for changing software?

Key findings

Big appetite for new research software

§ Over a third of respondents say that their companies are planning on changing their research software over the next two years.

All types of research software on shopping list

§ Between 51% and 70% of all of those companies who wish to change their research software want to change core research software modules (data collection, data processing, analysis and report publishing).

Competitive edge considerations top the list for wanting new software

§ ‘Seeking more features’ is the most common reason for wanting to change

§ Beyond this, most companies cite operational or efficiency reasons for needing new software


11.1 Planning on changing software in 2012

à Strong appetite for new software

à Especially in Asia Pacific

We see this question as a barometer both of research companies’ willingness to invest money in as well as their satisfaction with their existing tools. There are some wide regional variations, with 43% of those in Asia Pacific planning on buying new software against 38% in North America and 28% in Europe.

Q: Are you considering changing the software you are using for your data collection or data analysis in the next one to two years?

N = 250

Figure 35 Planning on changing software, by region and company size

34%! 38%!28%!

43%!33%! 35%! 36%!

23%! 12%! 32%!

25%!

23%! 22%!26%!

43%!50%!

40%!33%!

44%! 43%! 38%!

Total! N America! Europe! Asia Pac! Small! Medium! Large!

Yes! Not sure! No!


11.2 Planning on changing software - trend

à 2013 is a relatively favourable year for software developers with new tools to offer

With over a third of companies saying they plan to buy new software, the current year (i.e. the period that follows the 2012 survey) appears to offer more opportunities to software developers for new business than some recent years.

Q: Are you considering changing the software you are using for your data collection or data analysis in the next one to two years?

N = 250

Figure 36 Planning on changing software, by year

26%! 22%!

40%!32%!

25%!

46%!34%!

25%! 34%!

18%!26%!

23%!

20%!

23%!

49%! 44%! 42%! 43%!52%!

35%!43%!

2006! 2007! 2008! 2009! 2010! 2011! 2012!

Yes! Not sure! No!


11.3 Software type to change

à Data collection remains at the top of the shopping list

The biggest ticket items in any research software developer’s portfolio are usually the data collection tools, and participants to this survey have long expressed an appetite to replace them both in this and previous years.

Report publishing remains another hot ticket item, though data analysis seems to have retreated slightly.

The high scores across all categories also indicate that many companies are considering replacing tools in more than one area.

The lower scores for panel management tools is simply an indication that they are a specialised tool which is not used universally. In that context, demand for these also appears to be strong.

Q: What software are you considering changing?

N 2010 = 54; 2011 = 105; N 2012 = 85

Figure 37 Software type wishing to change

43%!

65%!

48%!

67%! 67%!

30%!

70%!

61%!65%! 65%!

31%!

69%!

51%!

57%!

62%!

Panel management! Data collection! Data processing! Data analysis! Report publishing and delivery!

2010! 2011! 2012!


11.4 Reasons for changing software

à Client-oriented reasons around maintaining a competitive edge are the most important

à Costs and other internal issues, less relevant

In this question we allowed participants to select up to three reasons why they wanted to change the software. Here in our chart, we show both where a reason has been selected as one of the top three and also where it has been selected as the main reason.

Most companies seem to be most interested in changing their software for more client-facing or outer-directed reasons. Resolving internal issues that do not affect the client experience, such as IT cost reduction, ease of use, concerns with suppliers seem to be of little importance.

Q: What are your reasons for considering a change in the software used?

N = 85

Figure 38 Reasons for wishing to change software

1.2%!

2.8%!

8.4%!

9.2%!

9.2%!

12.4%!

14.4%!

17.6%!

22.8%!

0.4%!

0.8%!

1.6%!

0.8%!

2.0%!

5.6%!

7.6%!

5.6%!

9.2%!

Company policy reasons!

Concerns with the existing supplier!

Seeking something that is easier to use!

To reduce software or IT costs!

Dissatisfaction or concerns with the existing software!

Consolidate all activities on a single integrated platform!

Move to a more modern platform!

To achieve efficiency improvements through increased automation!

Seeking more flexibility, more capabilities or better functionality!

Main reason!

Top 3 reasons!

the 2012 confirmit market research software...

Documents