![Page 1: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/1.jpg)
PerceptualAd-Blocking:MeetAdversarialMachineLearning
FlorianTramèrStanfordComputerForum– SecurityWorkshop
April8th 2019
JointworkwithPascalDupré,GiliRusak,GiancarloPellegrinoandDanBoneh
![Page 2: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/2.jpg)
2
TheFutureofAd-Blocking?easylist.txt
…markup……URLs…
???
Thisisanad
Humandistinguishabilityofads> Legalrequirement(U.S.FTC,EUE-Commerce)> Industryself-regulationonad-disclosure
![Page 3: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/3.jpg)
§ AdHighlighter byStorey etal.> Visuallydetectsad-disclosures> TraditionalComputerVisiontechniques> SimplifiedversionimplementableinAdblock Plus
§ Sentinel byAdblock Plus> LocatesadsinFacebookscreenshotsusing neuralnetworks> Notyetdeployed
PerceptualAd-Blocking
3
![Page 4: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/4.jpg)
§ AdHighlighter byStorey etal.> Visuallydetectsad-disclosures> TraditionalComputerVisiontechniques> SimplifiedversionimplementableinAdblock Plus
§ Sentinel byAdblock Plus> LocatesadsinFacebookscreenshotsusing neuralnetworks> Notyetdeployed
PerceptualAd-Blocking
4
![Page 5: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/5.jpg)
5
HowSecureisPerceptualAd-Blocking?
Jerry uploads malicious content
…
… so that Tom’s post
gets blocked
![Page 6: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/6.jpg)
§ Perceptualad-blockers:howtheywork
§ Attackingperceptualad-blockers
§ Whydefendingishard
6
Outline
![Page 7: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/7.jpg)
§ Perceptualad-blockers:howtheywork
§ Attackingperceptualad-blockers
§ Whydefendingishard
7
Outline
![Page 8: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/8.jpg)
HowdoesaPerceptualAd-BlockerWork?
8
https://www.example.com Ad Disclosure
Data Collection and Training Page Segmentation Action
Classifier Classifier
Ad
Classification
Ø Element-based (e.g.,findall<img>tags)[Storey etal.2017]Ø Frame-based (segmentrenderedwebpageinto“frames”)Ø Page-based (unsegmentedscreenshotsà-la-Sentinel)
Templatematching,OCR,DNNs,Objectdetectornetworks
![Page 9: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/9.jpg)
BuildingaPage-BasedAd-Blocker
9Video taken from 5 websites not used during training
We trained a neural network to detect ads on news websites from all G20 nations
![Page 10: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/10.jpg)
§ Perceptualad-blockers:howtheywork
§ Attackingperceptualad-blockers
§ Whydefendingishard
10
Outline
![Page 11: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/11.jpg)
MLworkswellonaverage≠
MLworkswellonadversarialdata
11
TheCurrentStateofML
*aslongasthereisnoadversary
*
![Page 12: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/12.jpg)
AdversarialExamples
§ How?> Training⟹ “tweakmodelparameters suchthat𝑓( ) = 𝑝𝑎𝑛𝑑𝑎”> Attacking⟹ “tweakinputpixels suchthat𝑓( ) = 𝑔𝑖𝑏𝑏𝑜𝑛”
12
Szegedy etal.,2014Goodfellow etal.,2015
𝜀 ≈ ⁄2 255
![Page 13: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/13.jpg)
13
AdversarialExamples:APervasivePhenomenon
(Carlini et al. 2016, Cisse et al. 2017,
Carlini & Wagner 2018)
(a) (b) (c) (d)
Figure 4: Examples of successful impersonation and dodging attacks. Fig. (a) shows SA (top) and SB (bottom) dodgingagainst DNNB . Fig. (b)–(d) show impersonations. Impersonators carrying out the attack are shown in the top row andcorresponding impersonation targets in the bottom row. Fig. (b) shows SA impersonating Milla Jovovich (by Georges Biard/ CC BY-SA / cropped from https://goo.gl/GlsWlC); (c) SB impersonating SC ; and (d) SC impersonating Carson Daly (byAnthony Quintano / CC BY / cropped from https://goo.gl/VfnDct).
Figure 5: The eyeglass frames used by SC for dodging recog-nition against DNNB .
postors) never occurs, while true acceptance remains high.Following a similar procedure, we found that a threshold of0.90 achieved a reasonable tradeo↵ between security and us-ability for DNNC ; the true acceptance rate became 92.01%and the false acceptance rate became 4e�3. Attemptingto decrease the false acceptance rate to 0 reduced the trueacceptance rate to 41.42%, making the FRS unusable.
Using thresholds changes the definition of successful im-personation: to successfully impersonate the target t, theprobability assigned to ct must exceed the threshold. Eval-uating the previous impersonation attempts under this def-inition, we found that success rates generally decreased butremained high enough for the impersonations to be consid-ered a real threat (see Table 2). For example, SB ’s successrate when attempting to fool DNNB and impersonate SC
decreased from 88.00% without threshold to 75.00% whenusing a threshold.
Time Complexity The DNNs we use in this work arelarge, e.g., the number of connections in DNNB , the small-est DNN, is about 3.86e8. Thus, the main overhead whensolving the optimization problem via GD is computing thederivatives of the DNNs with respect to the input images.For NI images used in the optimizations and NC connec-tions in the DNN, the time complexity of each GD iterationis O(NI ⇤NC). In practice, when using about 30 images, oneiteration of GD on a MacBook Pro (equipped with 16GB ofmemory and a 2.2GHz Intel i7 CPU) takes about 52.72 sec-onds. Hence, running the optimization up to 300 iterationsmay take about 4.39 hours.
6. EXTENSION TO BLACK-BOX MODELSSo far we have examined attacks where the adversary has
access to the model she is trying to deceive. In general,previous work on fooling ML systems has assumed knowl-edge of the architecture of the system (see Sec. 2). In thissection we demonstrate how similar attacks can be appliedin a black-box scenario. In such a scenario, the adversarywould typically have access only to an oracle O which out-puts a result for a given input and allows a limited number ofqueries. The threat model we consider here is one in whichthe adversary has access only to the oracle.We next briefly describe a commercial FRS that we use in
our experiments (Sec. 6.1), and then describe and evaluatepreliminary attempts to carry out impersonation attacks ina black-box setting (Sec. 6.2–6.3).
6.1 Face++: A Commercial FRSFace++ is a cross-platform commercial state-of-the-art
FRS that is widely used by applications for facial recog-nition, detection, tracking, and analysis [46]. It has beenshown to achieve accuracy over 97.3% on LFW [8]. Face++allows users to upload training images and labels and trainsan FRS that can be queried by applications. Given an im-age, the output from Face++ is the top three most proba-ble classes of the image along with their confidence scores.Face++ is marketed as“face recognition in the cloud.” Usershave no access to the internals of the training process andthe model used, nor even to a precise explanation of themeaning of the confidence scores. Face++ is rate-limited to50,000 free queries per month per user.To train the Face++ model, we used the same training
data used for DNNB in Sec. 4.1 to create a 10-class FRS.
6.2 Impersonation Attacks on Face++The goal of our black-box attack is for an adversary to
alter an image to which she has access so that it is mis-classified. We attempted dodging attacks with randomlycolored glasses and found that it worked immediately forseveral images. Therefore, in this section we focus on theproblem of impersonation from a given source to a target .
(Sharif et al. 2016)
(Kurakin et al. 2016)
(Athalye et al. 2018)
(Eykholt et al. 2017)(Eykholt et al. 2018)
![Page 14: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/14.jpg)
14
(Meaningful)Defenses
![Page 15: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/15.jpg)
15
AdversarialExamplesforPage-BasedPerceptualAd-Blockers
![Page 16: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/16.jpg)
§ Goal:Makeadsunrecognizablebyad-blocker
§ Adversary=Websitepublisher
§ Otheradversariesexist(e.g.,Ad-Network)
16
Ad-BlockEvasion
![Page 17: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/17.jpg)
Evasion:UniversalTransparentOverlay
17
UseHTMLtiling tominimizeperturbationsize(20KB)
Ø 100%successrateon20webpagesnotusedtocreatetheoverlayØ Theattackisuniversal: theoverlayiscomputedonceandworks
forall(ormost)websitesØ AttackcanbemademorestealthywithoutrelyingonCSS
§ Webpublisherperturbseveryrenderedpixel
![Page 18: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/18.jpg)
18
Ad-BlockDetection§ Goal:Triggerad-blockeron“honeypot”content
> Detectad-blockinginclient-sideJavaScriptoronserver> Applicabilityoftheseattacksdependsonad-blockertype
§ Adversary=Websitepublisher> Useclient-sideJavaScripttodetectDOMchanges
![Page 19: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/19.jpg)
Detection:Perturbfixedpagelayout
19
original
§ Publisheraddshoneypotinpage-regionwithfixedlayout> E.g.,pageheader
Withhoneypotheader
![Page 20: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/20.jpg)
20
NewThreats:PrivilegeAbuse
… so that Tom’s post gets blocked
Jerry uploads malicious content
…
Whathappened?Ø ObjectdetectormodelgeneratesboxpredictionsfromfullpageinputsØ ContentfromoneusercanaffectpredictionsanywhereonpageØ Model’ssegmentationisnotalignedwithweb-securityboundaries
§ Ad-blockevasion&detectionisawell-knownarmsrace.Butthere’smore!
![Page 21: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/21.jpg)
§ Perceptualad-blockers:howtheywork
§ Attackingperceptualad-blockers
§ Whydefendingishard
21
Outline
![Page 22: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/22.jpg)
Ø Adversaryhaswhite-boxaccess toad-blocker
Ø AdversarycanexploitFalseNegativesandFalsePositivesinclassificationpipeline
Ø Adversarypreparesattacksofflineó
Ø Adversarycantakepartincrowd-sourced datacollectionfortrainingthead-blocker
22
AChallengingThreatModel
Thead-blockermustdefendagainstattacksinreal-timeintheuser’sbrowser
![Page 23: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/23.jpg)
§ AttacksareeasyiftheadversaryhasaccesstotheMLmodel> Solution:hidemodelfromadversary?
§ Idea1:Obfuscatethead-blocker?> Itisn’thardtocreateadversarialexamplesforblack-boxclassifiers
§ Idea2:Randomizethead-blocker?> Deploydifferentmodels
- Adversarialexamplesthatworkagainstmultiplemodels> Randomly changepage beforeclassifying
- Adversarialexamplesrobusttorandomtransformations
23
DefenseStrategy1:ObfuscatetheModel
https://www.example.com AdDisclosure
DataCollectionandTraining (1)PageSegmentation (3)Action
Classifier Classifier
Ad
(2)Classification
https://www.example.com AdDisclosure
DataCollectionandTraining (1)PageSegmentation (3)Action
Classifier Classifier
Ad
(2)Classification
![Page 24: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/24.jpg)
§ Ifad-blockerisattacked(evasionordetection),collectadversarialsamplesandre-trainthemodel> Ortrainonadversarialexamplesproactively
§ ThisiscalledAdversarialTraining(Szegedy’14)> Newarms-race:Theadversaryfindsnewattacksandad-blockerre-trains> Mountinganewattackismucheasierthanupdatingthemodel> On-goingresearch:sofartheadversaryalwayswins!
24
DefenseStrategy2:AnticipateandAdapt
![Page 25: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/25.jpg)
§ Storey etal:recognizead-disclosures> Simplercomputervisionproblemthan
full-pagead-detection> Light-weightandmaturetechniques
(OCR,perceptualhashing,SIFT)
§ AdversarialExamplesstillexist
25
DefenseStrategy3:SimplifytheProblem
![Page 26: Perceptual Ad-Blocking: Meet Adversarial Machine Learning · 20 New Threats: Privilege Abuse … so that Tom’s post gets blocked Jerry uploads malicious content … What happened?](https://reader033.vdocuments.us/reader033/viewer/2022041723/5e4ff6c73f87326ee2449c77/html5/thumbnails/26.jpg)
TakeAway
§ Emulatinghumandetectionofadscouldbe theend-gameforad-blockers
§ But veryhardwithcurrentcomputervisiontechniques> Resistingadversarialexamplesisachallengingopenproblem
§ Perceptualad-blockershavetosurvivea strongthreatmodel> Evasion&detectionwithadversarialexamples> Privilegeabuseattacksfromarbitrarycontentproviders> Similarattackfornon-Webad-blockers(e.g.,adblock radio)
26https://github.com/ftramer/ad-versarial
Ø Trainapage-basedad-blockerØ Downloadpre-trainedmodelsØ Attackdemos
http://arxiv.org/abs/1811.03194