I Always Feel Like

Somebody’s Watching Me Measuring Online Behavioural

Advertising

Juan Miguel Carrascosa (Univ. Carlos III de Madrid)

Jakub Mikians (Universitat Polytechnic Catalunya)

Rubén Cuevas (Univ. Carlos III de Madrid)

Vijay Erramilli (Guavus)

Nikolaos Laoutaris (Telefonica Research)

WHAT? • A methodology for detecting behavioural

advertising for specific topics, for example: o Cars, sports or computers

o Sensitive topics: health, politics or sexual orientation

• Eventually it will become a tool for auditing specific

websites and trackers

2

WHY? • Online Behavioural Advertising (OBA)

…is rising and is here to stay

3

… but PRIVACY concerns are rising too

• Tracking, Cookies, Fingerprinting…

• Behavioural targeted Advertising is the

process of: o Collecting information (from users)

o Show ads (based on the information)

Because of tracking

4

Main idea of our methodology

5

CLEAN PROFILE/BROWSER

WEBPAGES

NEUTRAL WEBPAGE

6

Challenges

• Complexity of the ecosystem o Several aggregators, Ad Exchanges, Ad Networks, etc.

7

Challenges

• Complexity of the ecosystem o Several aggregators, Ad Exchanges, Ad Networks, etc.

• Different types of advertising

• How to obtain datasets and automate the process

Summary of Finding

• More expensive categories are more targeted

• Sensitive categories (e.g. health, sexual

orientation… ) seem to be targeted

• Do-Not-Track flag seems to have no effect on

whether you are targeted

8

METHODOLOGY

9

Methodology

10

SELECTION

COLLECTING ADS

TAGGING SYSTEM

FILTER PHASE

MEASURING OBA

Methodology

• Personas

• Training Pages

• Control Pages

11

SELECTION

COLLECTING ADS

TAGGING SYSTEM

FILTER PHASE

MEASURING OBA

12

1. Reduce profile contamination Personas with a very specific

behavioural trait

Methodology Selection of Personas and Training Pages

13

1. Reduce profile contamination Personas with a very specific

behavioural trait

2. Wide range of Personas

3. Automated and systematic

manner

Methodology Selection of Personas and Training Pages

• Google Ad Words' hierarchical category system o More than 2000 categories (i.e., behavioural traits)

14

Methodology Selection of Personas and Training Pages

+2000 • All Google Ad Words’ categories

240 • Google Ad Words CATEGORIZATION SYSTEM

202 • Google's ads PREFERENCES PROFILE

104 • Personas with at least 10 training pages

51 • 51 Personas with 10 training pages

15

Methodology Selection of Personas and Training Pages

• Reduce profile contamination

16

Methodology Selection of Control Pages

• Requirements: o Popular

o Show Ads

o Low number of identifiable tags

5 WEATHER PAGES

Methodology

17

SELECTION

COLLECTING ADS

TAGGING SYSTEM

FILTER PHASE

MEASURING OBA

18

Methodology Collecting Ads

Specific case: SWIMMING POOLS & SPAS

Methodology Collecting Ads

19

CONTROL PAGE!!!

COLLECT ADS (landing pages)

Methodology

SOURCES o Google

o Cyren

o McAfee

20

SELECTION

COLLECTING ADS

TAGGING SYSTEM

FILTER PHASE

MEASURING OBA

Methodology: Tagging Phase

21

• To detect systematically relations between training

and landing pages. o Based on keywords

• 3 sources/dictionaries o INPUT: Urls /Pages

o OUTPUT: Keywords/Categories

TRAINING PAGES

LANDING PAGES (ADs)

Cyren KEYWORDS

McAfee KEYWORDS

Google KEYWORDS

Methodology

22

SELECTION

COLLECTING ADS

TAGGING SYSTEM

FILTER PHASE

MEASURING OBA

• Used to identify and remove landing pages

associated with non-OBA ads:

23

Methodology Filtering different types of ads

Landing Page

Retargeting

Static & Contextual

Demographic & Geographical

Training Pages

Control Pages

Clean Profile

Personas’ Similarity

Methodology

METRICS

o TTK

o BAiLP

24

SELECTION

COLLECTING ADS

TAGGING SYSTEM

FILTER PHASE

MEASURING OBA

Methodology Measuring OBA: METRICS

• TTK (Targeted Training Keywords)

o MEASURES IF OBA IS HAPPENING

o MEASURES HOW MUCH A PERSONA IS

TARGETED

• BAiLP (Behavioural Advertising in Landing Page)

o MEASURES PERCENTAGE OF ADS THAT

ARE LIKELY ASSOCIATED TO OBA

o HOW MANY OF THE ADs SERVED TO A

PERSONA ARE TARGETED.

25

Training Keywords

Landing Keywords

Training Keywords Ads

EXPERIMENTAL ANALYSIS

26

How frequent is OBA? Are some personas more targeted than others?

27

In summary, • TTK measures if OBA is happening

• BAiLP captures what percentage is due to OBA.

CORRELATION received OBA (BAiLP) and its value for the advertisers

How frequent is OBA? Are some personas more targeted than others?

28

In summary, • TTK measures if OBA is happening

• BAiLP captures what percentage is due to OBA.

CORRELATION received OBA (BAiLP) and its value for the advertisers

Categories where the bid price is higher

MORE TARGETED

Is OBA applied to sensitive topics?

• Same methodology 21 sensitive personas

29

Is OBA applied to sensitive topics?

• Same methodology 21 sensitive personas

30

OBA in sensitive topics is LOWER than previous cases

BUT still EXIST

European Law

• To become effective on January 2016

31

Geographical and Do-Not-Track

• Same methodology: o Geographical: US and ES

o Do-Not-Track: ON|OFF

32

Geographical and Do-Not-Track

• Same methodology: o Geographical: US and ES

o Do-Not-Track: ON|OFF

33

Geographical regions and Do-Not-Track flag

DO NOT HAVE ANY IMPACT on OBA

Conclusions

• Detecting Behavioural targeting on specific

categories is difficult but doable

• Categories that have a higher value for the

advertisers get more targeted ads

34

THANK YOU

35

Juan Miguel Carrascosa Univ. Carlos III de Madrid

jcarrasc@it.uc3m.es

TYPES EUROPEAN PROJECT http://www.types-project.eu/

DATA TRANSPARENCY LAB http://www.datatransparencylab.org/