ad fraud blocking analytics webinar

Ad Fraud / Ad Blockingand Polluted Analytics

December 2015Augustine Fou, [email protected] 212.203.7239

mailto:[email protected]

November 2015 / marketing.scienceconsulting group, inc.

Dr. Augustine Fou

Brief Agenda

• Ad Fraud Background

• What is Ad Fraud

• Impact of Ad Blocking

• How Fraud Pollutes Analytics

• Low Hanging Fruit – You Can Do NOW!

Ad Fraud Background


Dr. Augustine Fou

Percent of digital ad spend in programmatic: 70 - 75%

1995Hundreds of major sites.

2005Thousands of mainstream blogs.

2015Millions of “long-tail” websites.


Dr. Augustine Fou

Fraud continues upward as digital ad spend goes up

Digital ad fraudHigh / Low Estimates

plus best-guess

Published estimates

Digital ad spendSource: IAB 2014 FY Report

$ billions

E


Dr. Augustine Fou

UPDATED: Full Year 2014 Digital Ad Spend – $50B

Impressions(CPM/CPV)

Clicks(CPC)

Leads(CPL)

Sales(CPA)

Search 38%$18.8B

Video 7%$3.5B

Lead Gen 4%$2.0B

10% Other$5.0B

Source: IAB, FY 2014 Internet Advertising Report, May 2015$42.5B

Display 16%$7.9B

Mobile 25%$6.2B$6.2B

CPM Performance

• classifieds• sponsorship• rich media

$7.0B

http://www.iab.net/media/file/IAB_Internet_Advertising_Revenue_Report_FY_20142.pdf


Dr. Augustine Fou

Bad guys go where the money is – impressions/clicks

Impressions(CPM/CPV)

Clicks(CPC)

Search$18.8B

86% digital spend

Display$7.9B

Video$3.5B Mobile

$6.2B$6.2B

Leads(CPL)

Sales(CPA)

Lead Gen$2.0B

Other$5.0B

• classifieds• sponsorship• rich media

estimated fraud

not at risk

(up from 84% in 2013)


Dr. Augustine Fou

retail finance automotive telecom CPG entertainment pharma travel cons. electronics0

10

20

30

40

50

60

70

80

90

100

indexed spend share

indexed fraud rate

Ad fraud impacts every industry vertical

High CPC industries

Source: Ad spend share data from IAB, May 2015 | Fraud rate data from Integral Ad Science Q2 2014 Fraud Report

What is Ad Fraud?


Dr. Augustine Fou

Two main types of fraud and how each is generated

Impression (CPM) Fraud

(includes mobile display, video ads)

1. Put up fake websites and load tons of ads on the pages

Search Click (CPC) Fraud

(includes mobile search ads)

2. Use bots to repeatedly load pages to generate fake ad impressions (launder the origins of the ads to avoid detection)

1. Put up fake websites and participate in search networks

2a. Use bots to type keywords to cause search ads to load

2b. Use bots to click on the ad to generate the CPC revenue


Dr. Augustine Fou

Bots are the cause of all automated ad fraud

Headless BrowsersSeleniumPhantomJSZombie.jsSlimerJS

Mobile Simulators35 listed

Bots are made from malware compromised PCs or headless browsers (no screen) in datacenters.

http://www.mobilexweb.com/emulators


Dr. Augustine Fou

Bots range from simple to advanced; do different tasks

Malware (on PCs)Botnets (from datacenters)

Toolbars (in-browser)Javascript (on webpages)


Dr. Augustine Fou

Bot fraud observed as high as 100%

Source: ANA / White Ops Study Published December 2014 [PDF]

display ads

11%

25%

video ads

23%

50%

sourced traffic

52%

100%

ANA/WhiteOps Study

What We’ve Seen

Case 1 Case 2

http://t.co/izYaT0swC9

Why are bad bots so hard to identify?


Dr. Augustine Fou

Bad guy’s advanced bots are not on any industry list

10,000bots observed

in the wild

user-agents.org

bad guys’ bots3%

Dstillery, Oct 9, 2014_“findings from two independent third parties,

Integral Ad Science and White Ops”

3.7%Rocket Fuel, Sep 22, 2014

“Forensiq results confirmed that ... only 3.72% of impressions categorized as high risk.”

2 - 3%comScore, Sep 26, 2014

“most campaigns have far less; more in the 2% to 3% range.”

detect based on industry bot list

“not on any list”disguised as normal browsers –

Internet Explorer; constantly adapting to avoid detection

http://finance.yahoo.com/news/dstillery-leads-industry-1-inventory-173635834.html

http://finance.yahoo.com/news/dstillery-leads-industry-1-inventory-173635834.html

http://rocketfuel.com/blog/forensiq-confirms-the-quality-of-rocket-fuel-programs

http://rocketfuel.com/blog/forensiq-confirms-the-quality-of-rocket-fuel-programs

http://adexchanger.com/platforms/fraud-day-with-comscore-an-ad-impression-is-a-terrible-thing-to-waste/






Dr. Augustine Fou

Example of filtering using bot lists – good, but not enough

Google Analytics filters visits using official bot lists

Bad guy bots are not on those lists and don’t declare themselves honestly; they pretend to be browsers like Internet Explorer, Safari, etc.

“bad guy bots are not on industry lists”


Dr. Augustine Fou

Humans vs “honest” bots vs fraudulent bots

Confirmed humans• found page via search• observed events (mouse

click with coordinates)

“Honest” bots• search engine crawlers• declare user agent honestly• observed to be 1 – 5% of

websites’ traffic

Fraud bots• come from data centers• malware compromised PCs• deliberately disguise user

agent as human users


Dr. Augustine Fou

Mitigation does not require developers or statisticians

Sites or ad networks that have high percentage of confirmed bots are blacklisted from ad-serving or ad spend to those sites is reduced

In-ad (display ads served)On-site (clients’ websites)

Sources of traffic that have high incidence of bots are added to ad-serving blacklists and filtered in analytics reports

Impact of Ad Blocking


Dr. Augustine Fou

Ad blocking user growth continues to soar

Source: PageFair / Adobe Aug 2015

http://blog.pagefair.com/2015/ad-blocking-report/



Dr. Augustine Fou

Ad blocking as a percent of users


Europe: 8% - 38%U.S.: 8% - 17%




Dr. Augustine Fou

Estimated economic impact of ad blocking


Global economic impact: $41BU.S. economic impact: $20B




Dr. Augustine Fou

Directly measured ad blocking rate

Non-mobile Mobile

Ad Block

No Ad Block 53.6%

15.4%

25.6%

5.4%

29% 21%

Overall Average

79.2%

20.8%

26% Ad Blocking Rate

* percentages represent portion of data from N = 10 million sample

69.0% 31.0%Column Totals

Pollution of Analytics


Dr. Augustine Fou

Bot activity pollutes quantity metrics

• Bots can be programmed to send as much traffic and generate as many impresisons and clicks as the advertiser wants

By systematically reducing ad spend to ad networks and sites that had the highest bots, and increasing allocation to premium publishers, the advertiser increased ad impressions served to humans, and lowered those served to bots.


Dr. Augustine Fou

Bot activity pollutes quality metrics

• Bots can manipulate bounce rates, click through rates, time on site, pages per visit; These engagement metrics appear to be tuned to 47 – 63%; pages per session averaged 2.03; and time on site was 1 – 2 minutes.


Dr. Augustine Fou

Bot activity pollutes conversion metrics

378411

357 361

512 495525

409

595

536 552596

437 452

380425

532489

592 584

403 416 415

587 570

490463

516

400 389418

Avg. 475 conversions /day

Avg. 3,526 sessions /dayAvg. 6,636 sessions /day

24% confirmed humans

-47%

Avg. 473 conversions /day

40% confirmed humans

0%

5%

10%

15%

20%

25%

Avg. 7.1% conversion rate

Avg. 13.5% conversion rate

“doubling humans, doubles conversion rates”


Dr. Augustine Fou

Bad guys hide fraud by passing fake parameters

Click thru URL passing fake source “utm_source=msn”

fake campaign“utm_campaign=Olay_Search”

http://www.olay.com/skin-care-products/OlayPro-X?utm_source=msn&utm_medium=cpc&utm_campaign=Olay_Search_Desktop_Category+Interest+Product.Phrase&utm_term=eye%20cream&utm_content=TZsrSzFz_eye%20cream_p_2990456911









Dr. Augustine Fou

Bad guys fake KPIs, trick measurement systemsBad guys have higher CTR Bad guys have higher viewability

AD

Bad guys stack ads above the fold to fake 100% viewability

Good guys have to array ads on the page – e.g. lower average viewability.

What you can do NOW


Dr. Augustine Fou

Recommendations

1. Don’t panic; but also don’t be complacent – directly measure the amount of fraud that is impacting your digital ad spend and continuously mitigate.

2. Focus on the details – don’t assume someone else has taken care of the problem; take small, simple steps at low to no cost – e.g. look in analytics for referring sites that have 100% bounce and 0:00 time on site.

3. Update KPIs to focus on things that are not easily faked (i.e. don’t focus on number of impressions, clicks, or visits); focus on “conversion events” like purchases or other human actions.


Dr. Augustine Fou

Normal Weekday vs Weekend Traffic Patterns

weekends weekends weekends weekends

weekdays weekdays weekdays weekdays

Natural website pattern is weekends have lower traffic


Dr. Augustine Fou

Typical Hour-of-Day Pattern

humans sleeping humans awake; visiting websites


Dr. Augustine Fou

Humans Sleep At Night

Hourly traffic charts show lower traffic at night (as expected because humans sleep at night)

Unusual traffic patterns with no normal night time trends visible, likely due to bot activity


Dr. Augustine Fou

Humans Visit via Search

humans find sites via search, during waking hours

Bot traffic adds anomalous spikes to pattern


Dr. Augustine Fou

Search vs Non-Human Traffic

notice the timing

hour-of-day pattern


Dr. Augustine Fou

Closeup by Hour of Day

6 am5 am 2 am 3 am 3 am 2 am 3 am18396 sessions 162 184 178 159 156

Sunday

85% avg bounce rate; 100% peak bounce rate


CONFIDENTIAL

These advanced bots also faked some Goal EventsGoal events that are based on page visits and video plays, could be (and were) faked.

page visit goal page visit goal video play goal


CONFIDENTIAL

But, there was no motive to fake other goals – e.g. pledges

Other goals like pledges and downloads were not faked (faking downloads would cost them server resources).

make a pledge curriculum download

“Bots don’t make donations!”


CONFIDENTIAL

Despite traffic loss, real human goals did not changeDespite losing all of the traffic from these fake/fraud sites, there was no change to the number of pledges and downloads, during the same period of time.

102,231 sessions

0 sessions

Conversion event 1

Conversion event 2

About the Author


CONFIDENTIAL

Dr. Augustine Fou – Recognized Expert on Ad Fraud

2013

2014

2015SPEAKING ENGAGEMENTS / PANELS4A’s Webinar on Ad Fraud – October Digital Ad Fraud Podcast – JanuaryProgrammatic Ad Fraud Webinar – MarchAdCouncil Webinar on Ad Fraud - AprilTelX Marketplace Live – JuneARF Audience Measurement – JuneIAB Webinar on Ad Fraud / Botnets - September [email protected] | 212.203.7239

mailto:[email protected]


CONFIDENTIAL

Harvard Business Review – October 2015

Excerpt:

Hunting the Bots

Fou, a prodigy who earned a Ph.D. from MIT at 23, belongs to the generation that witnessed the rise of digital marketers, having crafted his trade at American Express, one of the most successful American consumer brands, and at Omnicom, one of the largest global advertising agencies. Eventually stepping away from corporate life, Fou started his own practice, focusing on digital marketing fraud investigation.

Fou’s experiment proved that fake traffic is unproductive traffic. The fake visitors inflated the traffic statistics but contributed nothing to conversions, which stayed steady even after the traffic plummeted (bottom chart). Fake traffic is generated by “bad-guy bots.” A bot is computer code that runs automated tasks.

ad fraud blocking analytics webinar

Marketing