the state of physical attacks on deep learning systems · lujo bauer mila jovovich (87%) sharif et...
TRANSCRIPT
TheStateofPhysicalAttacksonDeepLearningSystems
EarlenceFernandes
Collaborators:IvanEvtimov,KevinEykholt,Chaowei Xiao,AmirRahmati,FlorianTramer,BoLi,AtulPrakash,Tadayoshi Kohno,DawnSong
ImagerecognitionObjectdetection
ScenesegmentationDNAvariantcalling
GameplayingSpeechrecognition
Re-enactingpoliticiansColorizingphotosPoseestimationDescribingphotosGeneratingphotos
TranslationMusiccompositions
CreatingartCreatingDNNs
PredictingearthquakesParticlephysics
QuantumchemistryRecommendationsCreatingfakenewsFightingfakenews
NLPAutomatedSurveillance
…
DeepLearning+Cyber-PhysicalSystems
AirborneCollisionAvoidanceSystemXunmanned(ACASXu)
Apollo(Baidu)Self-DrivingCar
TheGibbon-ImpersonatingPandaaka,AdversarialExamples
“panda”57.7%confidence
“gibbon”99.3%confidence
ImageCredit:OpenAI
=+ ε
ExplainingandHarnessingAdversarialExamples,Goodfellow etal.,arXiv 1412.6572,2015
But,anattackerrequirespixel-leveldigitalaccesstothemodel’sinput
Howcanattackerscreatephysicalattacks?
ACompendiumofPhysicalAttacksPrintingoutadigitallycreatedadversarialexampleworks,butislessrobusttoenvironmentalconditions
Printedpatternsoneyeglass-shapedcut-outscancompromisefacerecognition
clean adversarial
Kurakin etal.,AdversarialExamplesinthePhysicalWorld,arXiv 1607.02533,2016
FastGradientSignMethod(FGSM)approach
Lujo Bauer MilaJovovich(87%)
Sharifetal.,AccessorizetoaCrime:RealandStealthyAttacksonState-of-the-ArtFaceRecognition,CCS2016
Optimizationapproach
ACompendiumofPhysicalAttacksStickersonStopsignscanfoolobjectclassifiersand
detectorsinarangeofphysicalconditions
Optimizationapproach
Eykholt etal.,RobustPhysical-WorldAttacksonDeepLearningVisualClassification,CVPR2018
Eykholt etal.,PhysicalAdversarialExamplesforObjectDetectors,WOOT2018
Mywork
Chenetal.,RobustPhysicalAdversarialAttackonFaster-RCNNObjectDetector,arXiv 1804.05810,2018
AttackerscanbackdoorDNNssothatspecialstickerscausespecificbehavior
Training-timeattack
Guetal.,BadNets:IdentifyingVulnerabilitiesintheMachineLearningModelSupplyChain,arXiv1708.06733,2017
ACompendiumofPhysicalAttacks3Dprintedturtlescanberiflestoastate-of-the-artclassifier
AdversarialExamplescanhideinmusic
Patchesthatcamouflageanyobjectasatoasterexist
Athalye etal.,SynthesizingRobustAdversarialExamples,ICML2018
Expectation-over-Transformationsapproach(optimization)
Expectation-over-Transformationsapproach(optimization)
Brownetal.,AdversarialPatch,arXiv 1712.09665,May2018
Carlini etal.,AudioAdversarialExamples:TargetedAttacksonSpeech-
to-Text,DLSWorkshop2018
Yuanetal.,CommanderSong:ASystematicApproachforPractical
AdversarialVoiceRecognition,USENIXSecurity2018
OpenQuestions
• Arethereotherphysicaldomainswherewecanexploreadversarialexamples?• Currentattacksonlylookatasinglemodel.But,amodelisonlyapartofthewholeCPS.Dotheseattackshavesystem-wideeffects?• Isthereanythingspecificaboutphysicaladversarialexamplesthatmakethemeasierormoredifficulttodefendagainst?• Shouldweonlydependon“pureML”techniquesfordefense?• WhataspectsofCPSscanweleveragetodefend(defenseindepth)?
Thankyou!
• Arethereotherphysicaldomainswherewecanexploreadversarialexamples?• Currentattacksonlylookatasinglemodel.But,amodelisonlyapartofthewholeCPS.Dotheseattackshavesystem-wideeffects?• Isthereanythingspecificaboutphysicaladversarialexamplesthatmakethemeasierormoredifficulttodefendagainst?• Shouldweonlydependon“pureML”techniquesfordefense?• WhataspectsofCPSscanweleveragetodefend(defenseindepth)?
EarlenceFernandes,[email protected],earlence.com