annoyed users: ads and ad-block usage in the wildenric/publications/imc230-pujola... · annoyed...

Annoyed Users:

Ads and Ad-Block Usage in the Wild

IMC’15 Tokyo, Japan

Oliver Hohlfeld

RWTH Aachen

Anja Feldmann

TU Berlin

Enric Pujol

TU Berlin

2

3

http://www.journalism.org/2015/04/29/digital-news-revenue-fact-sheet

Page Fair and Adobe report on Ad Blocking

4

Mozilla Add-on statistics

Publishers reaction

5

The Guardian

6(Why can I not see BILD.de?)

Acceptable ads program(non-intrusive ads)

7

White-listing within AdBlockPlus (ABP)

Our work

8

Online advertisingResidential Broadband Network

Methodology based on AdBlockPlus

Characterize ad trafficAd-blocker usage

9

10

11

URL

||ads.example.com^ $ script, image, domain=bar.com

www.bar.com

#document HTML HEAD BODY … IMG

DOM Tree

Content-Type Base URL

Is this information available in HTTP header traces?

http://ads.example.com/ad.gif

www.foo.com

Content Policy

12

||ads.example.com ^ $ script, image, domain=foo.com

13

||ads.example.com ^ $ script, image, domain=foo.com

HTTP logsBro IDS

14

||ads.example.com ^ $ script, image, domain=foo.com

HTTP logsBro IDS

Map Web PagesReferrerMap

15

||ads.example.com ^ $ script, image, domain=foo.com

Classify requestslibadblockplus + filter lists

HTTP logsBro IDS

Map Web PagesReferrerMap

Limitations

16

HTTPS trafficContent-Type

HTTP logs

Referrer Map Web page

Classification Hidden ads (CSS)List sanity

Ad-blocker usage indicators

17

Low ratio of ad requests Easylist updates

AdBlock Plus

Caveat: ad-free content

Any ad-blocker

Caveat: HTTPS + NATs

How to find a suitable threshold?

Active measurements

Values for the complete set of Alexa top 1K

Browser Mode #HTTP ad-requests ratio

Vanilla 57.8K 16 %

Adblock Plus + all lists 48.5K <0.1 %

1828.10.2015

When can we apply ad-ratio indicator?

At least 1K requests. Set threshold to 5%1928.10.2015

Vanilla: no pluginAdBP-Pa: adblockPlus + all lists

Data sets

Trace RBN-1 RBN-2

Duration 4 days 15 and ½ hours

Subscribers 7.5 K 19.7 K

HTTPbytes 18.8 TB 11.4 TB

HTTPrequests 131.95 M 85.09 M

Characterize ad traffic

Assess ad-blocker prevalence

2028.10.2015

How much ad-related traffic?

2128.10.2015

Ad-related traffic

EasyList and derivatives (55 %) EasyPrivacy (35 %) Acceptable ads (10 %)

0 % 18 % 100 %

Black-listed (57%)

“Loose filters” @@||gstatic.com^$document

How many ABP-capable browsers?

Requests: 50 % (all)72 % (ads)

2228.10.2015

500K pairs<IP, UserAgent>

1.6K unique UserAgent strings 9.2 K users

Heavy hitters

MobileFirefox Chrome Safari IE

33 % 24 % 15 % 21 % 7 %

Known browsers

23

Ad Ratio EasyList

✘ ✘

✘ ✔

✔ ✘

✔ ✔

28.10.2015

Browsers with no evidence of an adblocker: 46% of the ad traffic

Inconsistent indicatorsa) various users in the same household

2428.10.2015

Ad Ratio EasyList

✘ ✘

✘ ✔

✔ ✘

✔ ✔

Inconsistent indicatorsa) different plugin, or b) ad-free content

2528.10.2015

Ad Ratio EasyList

✘ ✘

✘ ✔

✔ ✘

✔ ✔

Evidence of Adblock Plus for 22% of the users Still generate ad traffic!

2628.10.2015

Ad Ratio EasyList

✘ ✘

✘ ✔

✔ ✘

✔ ✔

?

27

Most AdBlockPlus users do not change default settings

82% 11 % 7%

EasyPrivacyAcceptable ads

EasyList

No ad blocker

EasyPrivacy

Acceptable ads

99%

87%

ABP users

90%

70%

How do users configure ABP?AGGREGATED

Summary

Can we use existing functionality to classify ad traffic?

Yes, we can! (but limitations apply)

2828.10.2015

Summary

Can we use existing functionality to classify ad traffic?

Yes, we can! (but limitations apply)

Which traffic is potentially affected by Adblock Plus?

~18% requests and ~1% HTTP bytes

Google, CDNs, Cloud Providers and dedicated ASes

10% of the ad-related traffic is white listed

2928.10.2015

Summary

Can we use existing functionality to classify ad traffic?

Yes, we can! (but limitations apply)

Which traffic is potentially affected by Adblock Plus?

~18% requests and ~1% HTTP bytes

Google, CDNs, Cloud Providers and dedicated ASes

10% of the ad-related traffic conforms Acceptable Ads

Can we measure how end users adopt ad-block plugins?

Yes, in our traces ~20% of users

Out-of-the-box i.e., acceptable ads / no EasyPrivacy3028.10.2015

Thank you!

Can we use existing functionality to classify ad traffic?

Yes, we can! (but limitations apply)

Which traffic is potentially affected by Adblock Plus?

~18% requests and ~1% HTTP bytes

Google, CDNs, Cloud Providers and dedicated ASes

10% of the ad-related traffic conforms Acceptable Ads

Can we measure how end users adopt ad-block plugins?

Yes, in our traces ~20% of users

Out-of-the-box i.e., acceptable ads / no EasyPrivacy3128.10.2015

3228.10.2015