Online experiment:
Data and Method
Media in Context and the 2015 General Election:
How Traditional and Social Media Shape Elections and Governing
(ES/M010775/1)
University of Exeter
What has an online experiment done for us?
Survey questions help estimating exposure to the specific media
sources, which can then be linked to the content of that media
Online experiment help examining causal linkages between media
exposure, information seeking and attitudes in constituencies with
varying contexts
Together they provide a robustness check on the findings from
observational data and allow detecting media effects and
estimating their size, as well as under what conditions they emerge.
11/09/2015
2
Design of the experiment
Online panel
Competitive (marginal) constituency
Guardian Telegraph Local/regional Control
Safe constituency
Guardian Telegraph Local/regional Control
11/09/2015
3
Wave 1-2 – 3 exposures a week followed by a survey & W3 – one shot exposure + a survey
Seat type by Wave
11/09/2015
4
All
Wave 1 Wave 2 Wave 3
2010 GE 2015 GE 2010 GE 2015 GE 2010 GE 2015 GE
Marginal 1297 32.0% 20.4% 32.8% 21.2% 33.1% 21.6%
Safe 2729 68.0% 79.6% 67.2% 78.8% 66.9% 78.4%
4026 3024 3279 1830
Treatment groups be Wave
Wave 1 Wave 2 Wave 3
Guardian marginal 227 7.5% 259 7.9% 136 7.4%
Telegraph marginal 247 8.2% 263 8.0% 146 8.0%
Regional marginal 246 8.1% 281 8.6% 163 8.9%
Control marginal 248 8.2% 272 8.3% 160 8.7%
Guardian safe 516 17.1% 546 16.7% 300 16.4%
Telegraph safe 496 16.4% 558 17.0% 297 16.2%
Regional safe 526 17.4% 553 16.9% 307 16.8%
Control safe 518 17.1% 547 16.7% 321 17.5%
3024 3279 1830
11/09/2015
5
Treatments
Panel members were randomly assigned to four treatment groups
the week before the General Election.
Three of them were exposed to different political news stories –
covering “horse race” and policy issues – from different newspapers
– The Guardian, The Telegraph, or relevant regional newspapers - 3
times a week treatment (before and after the election) and a ‘one
shot’ treatment (after the election). The fourth – control – group will
be exposed to “placebo” (non-political) news stories.
All treatment groups completed post-treatment online surveys
(Waves 1-3).
11/09/2015
6
Examples of treatments for W1-2
11/09/2015
7
Treatments for Wave 3
11/09/2015
8
Sample vs. UK population: Demographics
Variable Experimental subjects General population
Average age (in years) 53.22 47.78
Female (%) 48.31 50.89
White (%) 95.44 86.84
Education: GSCE (or equivalent) or below (%) 26.11 52.34
Education: A-levels or equivalent (%) 13.25 12.12
Education: University degree or professional
qualification (%)49.04 27.12
Employed full-time (%) 32.87 38.56
Employed part-time (%) 14.03 13.70
Unemployed (%) 5.48 4.41
Household income: < £20,000 31.76 30.00
Household income: > £80,000 4.26 10.00
11/09/2015
9
Sample vs. UK population: Regional distribution
Variable Experimental subjects General population
Region: East (%) 9.02 9.53
Region: East Midlands (%) 8.36 7.39
Region: London (%) 8.85 13.32
Region: North East (%) 5.14 4.23
Region: North West (%) 11.69 11.49
Region: South East (%) 15.03 14.07
Region: South West (%) 10.44 8.62
Region: West Midlands (%) 9.02 9.13
Region: Yorkshire and The Humber (%) 9.51 8.61
Wales (%) 5.03 4.99
Scotland (%) 7.92 8.63
11/09/2015
10
Sample vs. UK population: Voter preferences
Variable Experimental subjects General population
Election: Conservative voters (%) 36.94 36.81
Election: Labour voters (%) 29.14 30.44
Election: Lib. Dem. voters (%) 9.34 7.88
Election: UKIP voters (%) 14.91 12.65
11/09/2015
11
Party holding the participant's seat after the 2010 GE
Con Lab Lib Dem SNP
Plaid
Cymru Green Other
Wave 1 % 51.8 38.1 8.7 0.7 0.3 0.3 0.1
Wave 2 % 51.5 38.5 8.4 0.9 0.2 0.3 0.2
Wave 3 % 51.0 38.8 8.9 0.6 0.3 0.3 0.2
11/09/2015
Party holding the participant's seat after the 2015 GE
Con Lab Lib Dem SNP
Plaid
Cymru UKIP Green
Wave 1 % 56.9 33.8 1.3 7.3 0.3 0.1 0.3
Wave 2 % 56.4 34.5 1.3 7.3 0.2 0.1 0.3
Wave 3 % 56.6 34.3 1.0 7.5 0.3 0.1 0.3
Survey questions (all waves)
Socio-demographics
Political engagement
Political preferences (voter preferences, like/dislike of
parties/party leaders, closeness to political parties)
Political efficacy and interest in politics
Engagement with and recall of political information
Issues positions
11/09/2015
13
Survey questions – altering W1-3
Political knowledge (e.g., % of women MPs) – W1
Expectation of the GE outcome (nationally and in the constituency) – W1-2
Expectation of the closeness of the GE and its outcomes – W1
Likelihood to vote – W1
Media endorsements – W2-3
Trust in media – W2
Voted/not voted in the 2015 GE and party choice – W2-3
Assessment of the election outcomes – W2-3
Assessment of the previous/newly elected Government performance – W3
Assessment of the legitimacy of the electoral outcome and the decisiveness of the Conservative majority in 2015 – W3
11/09/2015
14
Linking the data to other data sources
Statistical matching - integrating two or more data sources (usually data from sample surveys) referred to the same target population (D’Orazio, 2015).
Statistical matching implies the matching of similar rather than identical units is not only acceptable but is expected.
Statistical matches are made on the basis of similarity of characteristics rather than uniquely identifying information as is the case with exact and probability matching (ONS, 2004).
See D’Orazio et al., 2006b; Singh et al., 1993 for more details11/09/2015
15