beyond academia: social media analysis in social and market research

1

Version 1 | Public © Ipsos MORI

Version 1 | Internal Use Only

Beyond academia: social media analysis in social and market researchBobby DuffyManaging Director, Ipsos MORI Social Research Institute Visiting Senior Research Fellow, King’s College London

Click here to insert cover image

Version 1 | Internal Use Only© Ipsos MORI 2

To manage expectations…

What we’re doing is not that unusual/special…

…but illustrative of issues faced

…and how we’re trying to develop approaches: social media research in researchers’ control


Ipsos MORI…

Public Affairs

The Social Research & Corporate Reputation

Specialists

Loyalty The Customer &

Employee Research Specialists

MediaThe Media, Content, & Technology Research

Specialists

MarketingThe Innovation & Brand Research Specialists

AdvertisingThe Advertising

Research Specialists

Political polling = 0.16% of our business


ONLINE

TELEPHONEFACE TO FACE

POSTAL

QUALITATIVE DEPTH

WORKSHOPSFOCUS

GROUPS

SOCIAL MEDIA

ANALYSIS

Neuro-science

DELIBERATIONETHNOGRAPHY

Has been quite a traditional industry…

Passive data

analysis


“Traditional” social media analysis allows us to capture basic metrics for evaluating campaigns or identifying trends

1. Define the search term

2. Define the base

3. Dashboard metrics

4. Site specific metrics

• Timeframe• Sample of users (eg Twitter users that

follow a specific account)

This can be set by:• Country • Type of media/site

All mentions of the search term are then pulled into one single database. The database is then used to explore different metrics:

• Number of mentions• Where (eg. blog, news, Twitter)• History of mentions over time• Topic (word cloud over time

• Mentions by individual site• Location (world map)• Automated sentimentAll metrics can be filtered.

Specific metrics are set up for Twitter (and news sites), exploring:• Top stories• Top hashtag• Top tweeters

• Number of ‘impressions’ (including followers/re-tweets)

• Most mentioned accounts

Individual entries can be viewed at any time (based on any of the filters above). These can be coded to consider context, tone, manual sentiment.

5. Manual coding

Define which phrases are to be included and which should be excluded.Including definition of word combinations and distance between sets of words.


Public attitudes to science evaluated engagement in science related topics over the course of a year

Source: Public Attitudes to Science 2014, BIS/ Ipsos MORI http://www.ipsos-mori.com/Assets/Docs/Polls/pas-2014-social-listening-climate-change-and-

animal-research.pdf

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec0

500

1000

1500

Horsemeat

Meteor

Measles

GM food

Fracking

Badger cull

Climate change (reaction to IPCC report)

Animal research

7

© Ipsos MORI | Final Version | Public

What do people write to MPs about?

Asylum/Immigration/ refugees

Benefits

Housing

Health Service

Badger culls

Child Support/Child Support Agency

Care of the elderly

Education/schools

Animal Research/ Experimentation

Social security

Famine/overseas aid

Pensions

Tax Credits

Hunting with dogs/fox hunting

66

61

58

56

54

43

42

39

33

30

23

23

21

20

All MPs | % Top mentions

Q Which of the subjects on this list, if any, do you receive most letters about in your post bag, or receive most approaches about from individuals in clinics or other ways?

Base: All MPs (143), Conservative MPs (58), Labour MPs (66) asked, Summer 2014: Source: Ipsos MORI MPs survey


Or much more specific: how particular audiences discuss issues – eg child abuse

1. Top tweets related to: campaigns & awareness raising, toolkits and applied information, and sharing of information of general professional interest.

2. We searched on 61 pre-specified terms of interest across four different

samples of interest

Sample 1 Sample 2 Sample 3 Sample 40

1

2

3

% tweets in sample relating to ‘Abuse’

How to deal with instances of child sexual

abuse

Research into reporting of child abuse


Or use network analysis to identify relationships and key influencers

These maps show the influence of a node or user in a network. In this case the network is the @CharityX account. So, in the case of @CharityX we can see that it’s connected to everyone else because everyone else is following @CharityX. @Educationgovuk is the second most influential user in this network…

@Educationgovuk

@CharityX @Individual

We identified the top influencers in the sample. They tended to be either in senior / high profile positions, have some training or consultancy role, be academics, practitioners or some combination of all of these.


A lot of event-based analysis…

The speech23rd September

>800 p/mtweets36,12

4

12,145Uniquevoices

How did Ed do?

How did Ed do?Tweets over time

Devolution/ constitutional reform/votes for 16-17 year olds

How did Ed do?Who tweeted, and how?

How did Ed do?Who tweeted, and how?

41 Positive1367 Negative

1598 Positive5644 Negative


•8 x more –ve than +ve tweets

•72% on personality, 22% on the political debate

Like Ben Swain, Farage is looking more like a sweaty octopus trying to unhook a bra.

Farage proving what a weapons grade doucheknuckle he is on subject of gay marriage. What a tit.

The white haired lady in the audience just flashed Nigel a serious come to bed smile


But lots of challenges and limitations

Version 1 | Internal Use Only© Ipsos MORI 21 21

Some of the challenges…

1. Calibration: understanding what good looks like

2. Sampling: A number of proxies can be established for identifying types of user. But, these remain assumptive and subjective.

3. Limited profiling: profiling data is limited. Small proportion of tweets have GPS tag, gender algorithms are around 80% accurate, but age profiling is considerably lower.

4. Quality of the data: most current tools are based on creating large search terms to identify relevant entries on social media. Limited ability to quality check and refine the data.

5. Analysis: no comprehensive single solution, requires using a number of different software packages. Resource intensive / expensive to understand sentiment or conduct text analysis. Guidance on ethical reporting is limited.

6. Representativeness…


Twitter is a bit weird…


Paste co-brand logo

here


% Visited in last 3 months

Trends In Social Networking Sites Visited

Base: circa 1,000 GB adults aged 15+ per wave Source: Ipsos MORI

Q2 '12 Q3 '12 Q4 '12 Q1 '13 Q2 '13 Q3 '13 Q4 '13 Q1 '14 Q2 '14 Q3 '140%

10%

20%

30%

40%

50%

60%

44

51

13

1817

117

9


ALL ADULTS

49%

51%

16%

17%

16%

17%

35%

26%

28%

22%

24%

61%

35%

Profile of Twitter Users

Base: circa GB adults (1,000) / All visiting / using Twitter in last 3 months (151): Q3 2014 Source: Ipsos MORI

Twitter users are young: Nearly two thirds are agedunder 35.

They are also more likely to be AB or C1 social grade and quite mobile: 88% of them own a Smartphone, 58% a Tablet.

Male

Female

15-24

25-34

35-44

45-54

55+

AB

C1

C2

DE

Own Smartphone

Own Tablet

5347

3924

2114

3

3333

2211

8858

Twitter users are young: Nearly two thirds are agedunder 35.

They are also more likely to be AB or C1 social grade and quite mobile: 88% of them own a Smartphone, 58% a Tablet.


Social Networking – accessing Twitter in past 3 months 2014

Base: circa 4,000 GB adults aged 15+: Q4 2013 / Q1 / Q2/ Q3 2014 Source: Ipsos MORI

Females 14 36 19 16 11 7 2Females AB 14 46 15 15 17 7 4

Females C1 17 45 25 20 7 12 1

Females C2 14 33 13 21 10 7 3

Females DE 12 26 19 10 12 1 0

All 15-24 25-34 35-44 45-54 55-64 65+

Males 18 35 25 26 13 9 2

Males AB 23 46 35 36 20 16 4

Males C1 22 39 32 32 10 2 3

Males C2 15 31 21 18 11 12 0

Males DE 10 26 13 9 5 3 0

40-100%20-39%0-19%

Version 1 | Public© Ipsos MORI

All data points represent > 200 responses

In which of these ways have you used the Internet in the last three months?

Has penetration started to flatten out?

09 10 11 12 130%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Pre war (born before 1945) Baby boomers (born 1945-1965) Generation X (born 1966-1979) Generation Y (born 1980-)

% To visit social networking sites (such as Facebook, Twitter etc), or to look at or/and to take part in discussion forums or blogs

Source: Ipsos MORI Observer

Version 1 | Public© Ipsos MORI

Map of how IPCC report from Sept 2013 was discussed online

Challenge: Identifying relevant discussion can be difficult

Armed police officer reinstated because sex on duty is 'like a tea break' - MIRROR http://bit.ly/1eHvK6R #ipcc

http://t.co/7auywDbxBD

https://twitter.com/hashtag/ipcc?src=hash

https://twitter.com/hashtag/ipcc?src=hash


Challenge: Identifying ‘public opinion’ isn’t easy either…

Climate change (Sept 13-Dec 13)

Traditional news 43%

Animal testing

Forums8%

Blogs14%

GM Measles

Twitter 35%

Fracking

Fracking

Badger culling

Badger cullMeteor

Horse-meat

The Guardian Newspaper

British Medical Journal

European Commissioner for Climate Action

Media agencies, charities, environmental organisations and politicians all have

voices on social media

Source: Public Attitudes to Science 2014, BIS/ Ipsos MORI http://www.ipsos-mori.com/Assets/Docs/Polls/pas-2014-social-listening-climate-change-and-animal-research.pdf


Challenge: Automating sentiment

13

46

41

60

37

4

200 sampleAutomated sentiment:

positive, neutral, negative

Same 200 sampleManual coding: pro, anti, neutral climate change Only a 55%

accuracy rate when trying

to use machine

learning to automate manual coding

RT @[name]: Way to go, elected officials! #idiots House Votes To Deny Climate Science And Ties Hands On Climate Change http://tu2026


Developing our approach


Partnership


Paste co-brand logo

here

Existing approaches

• Create complex search term with combinations of keywords

• Returned data only as good as the query

• Standard analysis of metadata (eg time, retweets)

• Manual coding/qual analysis• Data export to text analytics

programme• Lack of subgroup analysis• “Blackbox” for most

researchers

Proposed TSB approach

• Create simple search term• Manual coding to define

categories (tangible or attitudinal)

• Natural language processing (NLP) applied to remainder

• Iterative machine learning process to improve accuracy

• Standard analysis of metadata

• Additional subgroup analysis• Researchers “own” data

Developing our approach…


Developments over the next 12 months…

• Drawing new insights: demographic profiling (age, gender, location); can we produce aggregated profiles of the demographic split of gathered social media datasets?

• Credible research:oa framework for understanding the representativeness of

social media attitudes through tests against conventional research

oa confidence scoring system with which to judge the performance of social media analysis

oa corrective weighting programme from which to generalise social media attitudes onto wider constituencies/ social groups

• Ethical research: best practice ethics guide for social media research conducted by the social/market research community


Version 1 | Internal Use Only

Thank [email protected]

@BobbyIpsosMORI

Click here to insert cover image

beyond academia: social media analysis in social and market research

Documents

social media research

pa social research

research findings

senior research fellow

successful career ipsos

internal use

public ipsos mori1to

mori polls